2024-08-06T20:30:58.5216672Z Current runner version: '2.318.0' 2024-08-06T20:30:58.5226909Z Runner name: 'i-0b13d1134efc770c9' 2024-08-06T20:30:58.5228460Z Runner group name: 'Default' 2024-08-06T20:30:58.5229899Z Machine name: 'ip-10-0-52-148' 2024-08-06T20:30:58.5236821Z ##[group]GITHUB_TOKEN Permissions 2024-08-06T20:30:58.5240410Z Actions: read 2024-08-06T20:30:58.5241467Z Attestations: read 2024-08-06T20:30:58.5242530Z Checks: read 2024-08-06T20:30:58.5243509Z Contents: read 2024-08-06T20:30:58.5244477Z Deployments: read 2024-08-06T20:30:58.5245482Z Discussions: read 2024-08-06T20:30:58.5246476Z Issues: read 2024-08-06T20:30:58.5247435Z Metadata: read 2024-08-06T20:30:58.5248469Z Packages: read 2024-08-06T20:30:58.5249468Z Pages: read 2024-08-06T20:30:58.5250417Z PullRequests: read 2024-08-06T20:30:58.5251456Z RepositoryProjects: read 2024-08-06T20:30:58.5252555Z SecurityEvents: read 2024-08-06T20:30:58.5253760Z Statuses: read 2024-08-06T20:30:58.5254772Z ##[endgroup] 2024-08-06T20:30:58.5259813Z Secret source: Actions 2024-08-06T20:30:58.5261251Z Prepare workflow directory 2024-08-06T20:30:58.9190868Z Prepare all required actions 2024-08-06T20:30:58.9246317Z Getting action download info 2024-08-06T20:30:59.1093506Z Download action repository 'pytorch/test-infra@main' (SHA:a1f5a89251fc4258ab59806094fe3108f7d6741a) 2024-08-06T20:30:59.5452907Z Download action repository 'pytorch/pytorch@main' (SHA:de00c7958301ce81b9716bdef5731ed40d4d14ca) 2024-08-06T20:31:03.1773548Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-06T20:31:03.3345339Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-06T20:31:03.5911993Z Getting action download info 2024-08-06T20:31:03.6963111Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-06T20:31:03.9603836Z Getting action download info 2024-08-06T20:31:04.0622759Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-06T20:31:04.2773059Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/132710/merge (bf5bb5a1585a03379137fab341e87c02c77e76cd) 2024-08-06T20:31:04.2775930Z ##[group] Inputs 2024-08-06T20:31:04.2776686Z build-environment: linux-focal-cuda11.8-py3.10-gcc9 2024-08-06T20:31:04.2778365Z test-matrix: {"include": [{"config": "distributed", "shard": 1, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 2, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 3, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}]} 2024-08-06T20:31:04.2780477Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:31:04.2781560Z sync-tag: 2024-08-06T20:31:04.2782920Z timeout-minutes: 360 2024-08-06T20:31:04.2783559Z use-gha: 2024-08-06T20:31:04.2784031Z dashboard-tag: 2024-08-06T20:31:04.2784563Z s3-bucket: gha-artifacts 2024-08-06T20:31:04.2785101Z aws-role-to-assume: 2024-08-06T20:31:04.2785846Z ##[endgroup] 2024-08-06T20:31:04.2786650Z Complete job name: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 1, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:31:04.3551932Z A job started hook has been configured by the self-hosted runner administrator 2024-08-06T20:31:04.3709585Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-06T20:31:04.3722065Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:31:04.3723126Z ##[endgroup] 2024-08-06T20:31:05.9155337Z Runner Type: linux.8xlarge.nvidia.gpu 2024-08-06T20:31:05.9155975Z Instance Type: g3.8xlarge 2024-08-06T20:31:05.9156402Z AMI Name: amzn2-ami-hvm-2.0.20240306.2-x86_64-ebs 2024-08-06T20:31:05.9156875Z AMI ID: ami-0ce0c36d7a00b20e2 2024-08-06T20:31:10.9519817Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-06T20:31:10.9520615Z with: 2024-08-06T20:31:10.9521481Z github-secret: *** 2024-08-06T20:31:10.9522408Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-06T20:31:10.9523338Z activate-with-label: false 2024-08-06T20:31:10.9523698Z label: with-ssh 2024-08-06T20:31:10.9524028Z remove-existing-keys: true 2024-08-06T20:31:10.9524403Z fail-silently: true 2024-08-06T20:31:10.9524717Z env: 2024-08-06T20:31:10.9525007Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:31:10.9525359Z ##[endgroup] 2024-08-06T20:31:11.0645551Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-06T20:31:11.3661505Z Grabbing public ssh keys from https://github.com/drisspg.keys 2024-08-06T20:31:11.4354616Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-06T20:31:11.4376533Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-06T20:31:11.4419199Z Login using: ssh ec2-user@ec2-54-210-110-210.compute-1.amazonaws.com 2024-08-06T20:31:11.4419972Z All testing is done inside the container, to start an interactive session run: 2024-08-06T20:31:11.4420621Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-06T20:31:11.4557143Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-06T20:31:11.4557741Z with: 2024-08-06T20:31:11.4558031Z submodules: recursive 2024-08-06T20:31:11.4558387Z fetch-depth: 0 2024-08-06T20:31:11.4558699Z env: 2024-08-06T20:31:11.4559052Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:31:11.4559409Z ##[endgroup] 2024-08-06T20:31:11.4663931Z ##[group]Run retry () { 2024-08-06T20:31:11.4664344Z retry () { 2024-08-06T20:31:11.4664796Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-06T20:31:11.4665295Z } 2024-08-06T20:31:11.4665945Z echo "${GITHUB_WORKSPACE}" 2024-08-06T20:31:11.4666372Z if [ -z "${NO_SUDO}" ]; then 2024-08-06T20:31:11.4666821Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-06T20:31:11.4667256Z else 2024-08-06T20:31:11.4667585Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-06T20:31:11.4667999Z fi 2024-08-06T20:31:11.4668375Z mkdir "${GITHUB_WORKSPACE}" 2024-08-06T20:31:11.4676944Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:31:11.4677462Z env: 2024-08-06T20:31:11.4677748Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:31:11.4678101Z NO_SUDO: 2024-08-06T20:31:11.4678396Z ##[endgroup] 2024-08-06T20:31:11.4704462Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:31:14.7818824Z ##[group]Run malfet/checkout@silent-checkout 2024-08-06T20:31:14.7819264Z with: 2024-08-06T20:31:14.7819576Z ref: b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:31:14.7820005Z fetch-depth: 0 2024-08-06T20:31:14.7820331Z submodules: recursive 2024-08-06T20:31:14.7820676Z quiet-checkout: true 2024-08-06T20:31:14.7821025Z repository: pytorch/pytorch 2024-08-06T20:31:14.7821533Z token: *** 2024-08-06T20:31:14.7821841Z ssh-strict: true 2024-08-06T20:31:14.7822154Z persist-credentials: true 2024-08-06T20:31:14.7822511Z clean: true 2024-08-06T20:31:14.7822837Z sparse-checkout-cone-mode: true 2024-08-06T20:31:14.7823220Z lfs: false 2024-08-06T20:31:14.7823527Z set-safe-directory: true 2024-08-06T20:31:14.7823858Z env: 2024-08-06T20:31:14.7824146Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:31:14.7824489Z ##[endgroup] 2024-08-06T20:31:14.9129021Z Syncing repository: pytorch/pytorch 2024-08-06T20:31:14.9130855Z ##[group]Getting Git version info 2024-08-06T20:31:14.9131478Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-06T20:31:14.9132353Z [command]/usr/bin/git version 2024-08-06T20:31:14.9135281Z git version 2.40.1 2024-08-06T20:31:14.9171386Z ##[endgroup] 2024-08-06T20:31:14.9193800Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/96de4b94-dc70-4d6a-b5f5-490e231f8084' before making global git config changes 2024-08-06T20:31:14.9195259Z Adding repository directory to the temporary git global config as a safe directory 2024-08-06T20:31:14.9201026Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:31:14.9246336Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-06T20:31:14.9252720Z ##[group]Initializing the repository 2024-08-06T20:31:14.9256785Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:31:14.9289116Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-06T20:31:14.9289877Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-06T20:31:14.9290574Z hint: of your new repositories, which will suppress this warning, call: 2024-08-06T20:31:14.9291101Z hint: 2024-08-06T20:31:14.9291682Z hint: git config --global init.defaultBranch 2024-08-06T20:31:14.9292175Z hint: 2024-08-06T20:31:14.9292887Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-06T20:31:14.9293698Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-06T20:31:14.9294479Z hint: 2024-08-06T20:31:14.9294823Z hint: git branch -m 2024-08-06T20:31:14.9295460Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-06T20:31:14.9303142Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-06T20:31:14.9336868Z ##[endgroup] 2024-08-06T20:31:14.9337432Z ##[group]Disabling automatic garbage collection 2024-08-06T20:31:14.9341427Z [command]/usr/bin/git config --local gc.auto 0 2024-08-06T20:31:14.9374535Z ##[endgroup] 2024-08-06T20:31:14.9375081Z ##[group]Setting up auth 2024-08-06T20:31:14.9384198Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-06T20:31:14.9417781Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-06T20:31:14.9705416Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-06T20:31:14.9737731Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-06T20:31:15.0024777Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-06T20:31:15.0075603Z ##[endgroup] 2024-08-06T20:31:15.0076167Z ##[group]Fetching the repository 2024-08-06T20:31:15.0084110Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-06T20:31:17.2733644Z remote: Enumerating objects: 1008383 2024-08-06T20:31:17.2734316Z remote: Enumerating objects: 1008693, done. 2024-08-06T20:31:17.2736535Z remote: Counting objects: 0% (1/310) 2024-08-06T20:31:17.2737035Z remote: Counting objects: 1% (4/310) 2024-08-06T20:31:17.2737534Z remote: Counting objects: 2% (7/310) 2024-08-06T20:31:17.2738013Z remote: Counting objects: 3% (10/310) 2024-08-06T20:31:17.2738463Z remote: Counting objects: 4% (13/310) 2024-08-06T20:31:17.2738927Z remote: Counting objects: 5% (16/310) 2024-08-06T20:31:17.2739395Z remote: Counting objects: 6% (19/310) 2024-08-06T20:31:17.2739848Z remote: Counting objects: 7% (22/310) 2024-08-06T20:31:17.2740315Z remote: Counting objects: 8% (25/310) 2024-08-06T20:31:17.2740777Z remote: Counting objects: 9% (28/310) 2024-08-06T20:31:17.2741224Z remote: Counting objects: 10% (31/310) 2024-08-06T20:31:17.2742072Z remote: Counting objects: 11% (35/310) 2024-08-06T20:31:17.2742535Z remote: Counting objects: 12% (38/310) 2024-08-06T20:31:17.2742976Z remote: Counting objects: 13% (41/310) 2024-08-06T20:31:17.2743437Z remote: Counting objects: 14% (44/310) 2024-08-06T20:31:17.2743902Z remote: Counting objects: 15% (47/310) 2024-08-06T20:31:17.2744347Z remote: Counting objects: 16% (50/310) 2024-08-06T20:31:17.2744812Z remote: Counting objects: 17% (53/310) 2024-08-06T20:31:17.2745273Z remote: Counting objects: 18% (56/310) 2024-08-06T20:31:17.2746002Z remote: Counting objects: 19% (59/310) 2024-08-06T20:31:17.2746467Z remote: Counting objects: 20% (62/310) 2024-08-06T20:31:17.2746993Z remote: Counting objects: 21% (66/310) 2024-08-06T20:31:17.2747457Z remote: Counting objects: 22% (69/310) 2024-08-06T20:31:17.2747897Z remote: Counting objects: 23% (72/310) 2024-08-06T20:31:17.2748357Z remote: Counting objects: 24% (75/310) 2024-08-06T20:31:17.2748830Z remote: Counting objects: 25% (78/310) 2024-08-06T20:31:17.2749272Z remote: Counting objects: 26% (81/310) 2024-08-06T20:31:17.2749829Z remote: Counting objects: 27% (84/310) 2024-08-06T20:31:17.2750483Z remote: Counting objects: 28% (87/310) 2024-08-06T20:31:17.2750962Z remote: Counting objects: 29% (90/310) 2024-08-06T20:31:17.2751425Z remote: Counting objects: 30% (93/310) 2024-08-06T20:31:17.2751882Z remote: Counting objects: 31% (97/310) 2024-08-06T20:31:17.2752327Z remote: Counting objects: 32% (100/310) 2024-08-06T20:31:17.2752799Z remote: Counting objects: 33% (103/310) 2024-08-06T20:31:17.2753269Z remote: Counting objects: 34% (106/310) 2024-08-06T20:31:17.2753720Z remote: Counting objects: 35% (109/310) 2024-08-06T20:31:17.2754184Z remote: Counting objects: 36% (112/310) 2024-08-06T20:31:17.2754652Z remote: Counting objects: 37% (115/310) 2024-08-06T20:31:17.2755115Z remote: Counting objects: 38% (118/310) 2024-08-06T20:31:17.2755584Z remote: Counting objects: 39% (121/310) 2024-08-06T20:31:17.2756050Z remote: Counting objects: 40% (124/310) 2024-08-06T20:31:17.2756503Z remote: Counting objects: 41% (128/310) 2024-08-06T20:31:17.2756978Z remote: Counting objects: 42% (131/310) 2024-08-06T20:31:17.2757449Z remote: Counting objects: 43% (134/310) 2024-08-06T20:31:17.2757898Z remote: Counting objects: 44% (137/310) 2024-08-06T20:31:17.2758368Z remote: Counting objects: 45% (140/310) 2024-08-06T20:31:17.2758833Z remote: Counting objects: 46% (143/310) 2024-08-06T20:31:17.2759323Z remote: Counting objects: 47% (146/310) 2024-08-06T20:31:17.2759804Z remote: Counting objects: 48% (149/310) 2024-08-06T20:31:17.2760273Z remote: Counting objects: 49% (152/310) 2024-08-06T20:31:17.2760729Z remote: Counting objects: 50% (155/310) 2024-08-06T20:31:17.2761207Z remote: Counting objects: 51% (159/310) 2024-08-06T20:31:17.2761675Z remote: Counting objects: 52% (162/310) 2024-08-06T20:31:17.2762124Z remote: Counting objects: 53% (165/310) 2024-08-06T20:31:17.2762594Z remote: Counting objects: 54% (168/310) 2024-08-06T20:31:17.2763071Z remote: Counting objects: 55% (171/310) 2024-08-06T20:31:17.2763525Z remote: Counting objects: 56% (174/310) 2024-08-06T20:31:17.2763994Z remote: Counting objects: 57% (177/310) 2024-08-06T20:31:17.2764469Z remote: Counting objects: 58% (180/310) 2024-08-06T20:31:17.2764919Z remote: Counting objects: 59% (183/310) 2024-08-06T20:31:17.2765391Z remote: Counting objects: 60% (186/310) 2024-08-06T20:31:17.2765957Z remote: Counting objects: 61% (190/310) 2024-08-06T20:31:17.2766412Z remote: Counting objects: 62% (193/310) 2024-08-06T20:31:17.2766919Z remote: Counting objects: 63% (196/310) 2024-08-06T20:31:17.2767519Z remote: Counting objects: 64% (199/310) 2024-08-06T20:31:17.2767988Z remote: Counting objects: 65% (202/310) 2024-08-06T20:31:17.2768435Z remote: Counting objects: 66% (205/310) 2024-08-06T20:31:17.2768900Z remote: Counting objects: 67% (208/310) 2024-08-06T20:31:17.2769370Z remote: Counting objects: 68% (211/310) 2024-08-06T20:31:17.2769820Z remote: Counting objects: 69% (214/310) 2024-08-06T20:31:17.2770283Z remote: Counting objects: 70% (217/310) 2024-08-06T20:31:17.2770745Z remote: Counting objects: 71% (221/310) 2024-08-06T20:31:17.2771190Z remote: Counting objects: 72% (224/310) 2024-08-06T20:31:17.2771656Z remote: Counting objects: 73% (227/310) 2024-08-06T20:31:17.2772126Z remote: Counting objects: 74% (230/310) 2024-08-06T20:31:17.2772573Z remote: Counting objects: 75% (233/310) 2024-08-06T20:31:17.2773036Z remote: Counting objects: 76% (236/310) 2024-08-06T20:31:17.2773509Z remote: Counting objects: 77% (239/310) 2024-08-06T20:31:17.2773954Z remote: Counting objects: 78% (242/310) 2024-08-06T20:31:17.2774419Z remote: Counting objects: 79% (245/310) 2024-08-06T20:31:17.2774885Z remote: Counting objects: 80% (248/310) 2024-08-06T20:31:17.2775423Z remote: Counting objects: 81% (252/310) 2024-08-06T20:31:17.2775908Z remote: Counting objects: 82% (255/310) 2024-08-06T20:31:17.2776372Z remote: Counting objects: 83% (258/310) 2024-08-06T20:31:17.2776818Z remote: Counting objects: 84% (261/310) 2024-08-06T20:31:17.2777293Z remote: Counting objects: 85% (264/310) 2024-08-06T20:31:17.2777757Z remote: Counting objects: 86% (267/310) 2024-08-06T20:31:17.2778206Z remote: Counting objects: 87% (270/310) 2024-08-06T20:31:17.2778674Z remote: Counting objects: 88% (273/310) 2024-08-06T20:31:17.2779141Z remote: Counting objects: 89% (276/310) 2024-08-06T20:31:17.2779603Z remote: Counting objects: 90% (279/310) 2024-08-06T20:31:17.2780069Z remote: Counting objects: 91% (283/310) 2024-08-06T20:31:17.2780536Z remote: Counting objects: 92% (286/310) 2024-08-06T20:31:17.2780986Z remote: Counting objects: 93% (289/310) 2024-08-06T20:31:17.2781460Z remote: Counting objects: 94% (292/310) 2024-08-06T20:31:17.2781929Z remote: Counting objects: 95% (295/310) 2024-08-06T20:31:17.2782400Z remote: Counting objects: 96% (298/310) 2024-08-06T20:31:17.2782852Z remote: Counting objects: 97% (301/310) 2024-08-06T20:31:17.2783320Z remote: Counting objects: 98% (304/310) 2024-08-06T20:31:17.2783787Z remote: Counting objects: 99% (307/310) 2024-08-06T20:31:17.2784235Z remote: Counting objects: 100% (310/310) 2024-08-06T20:31:17.2784735Z remote: Counting objects: 100% (310/310), done. 2024-08-06T20:31:17.2785240Z remote: Compressing objects: 0% (1/177) 2024-08-06T20:31:17.2789324Z remote: Compressing objects: 1% (2/177) 2024-08-06T20:31:17.2811271Z remote: Compressing objects: 2% (4/177) 2024-08-06T20:31:17.2833991Z remote: Compressing objects: 3% (6/177) 2024-08-06T20:31:17.2901193Z remote: Compressing objects: 4% (8/177) 2024-08-06T20:31:17.2995198Z remote: Compressing objects: 5% (9/177) 2024-08-06T20:31:17.3049441Z remote: Compressing objects: 6% (11/177) 2024-08-06T20:31:17.3071721Z remote: Compressing objects: 7% (13/177) 2024-08-06T20:31:17.3096631Z remote: Compressing objects: 8% (15/177) 2024-08-06T20:31:17.3128608Z remote: Compressing objects: 9% (16/177) 2024-08-06T20:31:17.3246951Z remote: Compressing objects: 10% (18/177) 2024-08-06T20:31:17.3247416Z remote: Compressing objects: 11% (20/177) 2024-08-06T20:31:17.3247914Z remote: Compressing objects: 12% (22/177) 2024-08-06T20:31:17.3248396Z remote: Compressing objects: 13% (24/177) 2024-08-06T20:31:17.3249336Z remote: Compressing objects: 14% (25/177) 2024-08-06T20:31:17.3249811Z remote: Compressing objects: 15% (27/177) 2024-08-06T20:31:17.3250289Z remote: Compressing objects: 16% (29/177) 2024-08-06T20:31:17.3250747Z remote: Compressing objects: 17% (31/177) 2024-08-06T20:31:17.3251230Z remote: Compressing objects: 18% (32/177) 2024-08-06T20:31:17.3251716Z remote: Compressing objects: 19% (34/177) 2024-08-06T20:31:17.3252175Z remote: Compressing objects: 20% (36/177) 2024-08-06T20:31:17.3252655Z remote: Compressing objects: 21% (38/177) 2024-08-06T20:31:17.3253134Z remote: Compressing objects: 22% (39/177) 2024-08-06T20:31:17.3253594Z remote: Compressing objects: 23% (41/177) 2024-08-06T20:31:17.3254076Z remote: Compressing objects: 24% (43/177) 2024-08-06T20:31:17.3254581Z remote: Compressing objects: 25% (45/177) 2024-08-06T20:31:17.3255064Z remote: Compressing objects: 26% (47/177) 2024-08-06T20:31:17.3255532Z remote: Compressing objects: 27% (48/177) 2024-08-06T20:31:17.3256010Z remote: Compressing objects: 28% (50/177) 2024-08-06T20:31:17.3256490Z remote: Compressing objects: 29% (52/177) 2024-08-06T20:31:17.3256955Z remote: Compressing objects: 30% (54/177) 2024-08-06T20:31:17.3257561Z remote: Compressing objects: 31% (55/177) 2024-08-06T20:31:17.3258374Z remote: Compressing objects: 32% (57/177) 2024-08-06T20:31:17.3262236Z remote: Compressing objects: 33% (59/177) 2024-08-06T20:31:17.3262886Z remote: Compressing objects: 34% (61/177) 2024-08-06T20:31:17.3265674Z remote: Compressing objects: 35% (62/177) 2024-08-06T20:31:17.3269598Z remote: Compressing objects: 36% (64/177) 2024-08-06T20:31:17.3270087Z remote: Compressing objects: 37% (66/177) 2024-08-06T20:31:17.3272099Z remote: Compressing objects: 38% (68/177) 2024-08-06T20:31:17.3272601Z remote: Compressing objects: 39% (70/177) 2024-08-06T20:31:17.3274923Z remote: Compressing objects: 40% (71/177) 2024-08-06T20:31:17.3275417Z remote: Compressing objects: 41% (73/177) 2024-08-06T20:31:17.3276903Z remote: Compressing objects: 42% (75/177) 2024-08-06T20:31:17.3277407Z remote: Compressing objects: 43% (77/177) 2024-08-06T20:31:17.3277896Z remote: Compressing objects: 44% (78/177) 2024-08-06T20:31:17.3280745Z remote: Compressing objects: 45% (80/177) 2024-08-06T20:31:17.3281276Z remote: Compressing objects: 46% (82/177) 2024-08-06T20:31:17.3281758Z remote: Compressing objects: 47% (84/177) 2024-08-06T20:31:17.3282236Z remote: Compressing objects: 48% (85/177) 2024-08-06T20:31:17.3282719Z remote: Compressing objects: 49% (87/177) 2024-08-06T20:31:17.3283203Z remote: Compressing objects: 50% (89/177) 2024-08-06T20:31:17.3283666Z remote: Compressing objects: 51% (91/177) 2024-08-06T20:31:17.3284148Z remote: Compressing objects: 52% (93/177) 2024-08-06T20:31:17.3284637Z remote: Compressing objects: 53% (94/177) 2024-08-06T20:31:17.3285101Z remote: Compressing objects: 54% (96/177) 2024-08-06T20:31:17.3285578Z remote: Compressing objects: 55% (98/177) 2024-08-06T20:31:17.3286072Z remote: Compressing objects: 56% (100/177) 2024-08-06T20:31:17.3286543Z remote: Compressing objects: 57% (101/177) 2024-08-06T20:31:17.3287026Z remote: Compressing objects: 58% (103/177) 2024-08-06T20:31:17.3287503Z remote: Compressing objects: 59% (105/177) 2024-08-06T20:31:17.3287985Z remote: Compressing objects: 60% (107/177) 2024-08-06T20:31:17.3288445Z remote: Compressing objects: 61% (108/177) 2024-08-06T20:31:17.3288994Z remote: Compressing objects: 62% (110/177) 2024-08-06T20:31:17.3289476Z remote: Compressing objects: 63% (112/177) 2024-08-06T20:31:17.3289942Z remote: Compressing objects: 64% (114/177) 2024-08-06T20:31:17.3290613Z remote: Compressing objects: 65% (116/177) 2024-08-06T20:31:17.3291096Z remote: Compressing objects: 66% (117/177) 2024-08-06T20:31:17.3291558Z remote: Compressing objects: 67% (119/177) 2024-08-06T20:31:17.3292037Z remote: Compressing objects: 68% (121/177) 2024-08-06T20:31:17.3292526Z remote: Compressing objects: 69% (123/177) 2024-08-06T20:31:17.3292992Z remote: Compressing objects: 70% (124/177) 2024-08-06T20:31:17.3293470Z remote: Compressing objects: 71% (126/177) 2024-08-06T20:31:17.3293950Z remote: Compressing objects: 72% (128/177) 2024-08-06T20:31:17.3294412Z remote: Compressing objects: 73% (130/177) 2024-08-06T20:31:17.3294894Z remote: Compressing objects: 74% (131/177) 2024-08-06T20:31:17.3295371Z remote: Compressing objects: 75% (133/177) 2024-08-06T20:31:17.3295833Z remote: Compressing objects: 76% (135/177) 2024-08-06T20:31:17.3296348Z remote: Compressing objects: 77% (137/177) 2024-08-06T20:31:17.3296840Z remote: Compressing objects: 78% (139/177) 2024-08-06T20:31:17.3297308Z remote: Compressing objects: 79% (140/177) 2024-08-06T20:31:17.3297790Z remote: Compressing objects: 80% (142/177) 2024-08-06T20:31:17.3298387Z remote: Compressing objects: 81% (144/177) 2024-08-06T20:31:17.3298872Z remote: Compressing objects: 82% (146/177) 2024-08-06T20:31:17.3299361Z remote: Compressing objects: 83% (147/177) 2024-08-06T20:31:17.3299841Z remote: Compressing objects: 84% (149/177) 2024-08-06T20:31:17.3300324Z remote: Compressing objects: 85% (151/177) 2024-08-06T20:31:17.3300791Z remote: Compressing objects: 86% (153/177) 2024-08-06T20:31:17.3301277Z remote: Compressing objects: 87% (154/177) 2024-08-06T20:31:17.3301761Z remote: Compressing objects: 88% (156/177) 2024-08-06T20:31:17.3302296Z remote: Compressing objects: 89% (158/177) 2024-08-06T20:31:17.3302762Z remote: Compressing objects: 90% (160/177) 2024-08-06T20:31:17.3303258Z remote: Compressing objects: 91% (162/177) 2024-08-06T20:31:17.3303741Z remote: Compressing objects: 92% (163/177) 2024-08-06T20:31:17.3304206Z remote: Compressing objects: 93% (165/177) 2024-08-06T20:31:17.3304697Z remote: Compressing objects: 94% (167/177) 2024-08-06T20:31:17.3305183Z remote: Compressing objects: 95% (169/177) 2024-08-06T20:31:17.3305899Z remote: Compressing objects: 96% (170/177) 2024-08-06T20:31:17.3306393Z remote: Compressing objects: 97% (172/177) 2024-08-06T20:31:17.3306875Z remote: Compressing objects: 98% (174/177) 2024-08-06T20:31:17.3307343Z remote: Compressing objects: 99% (176/177) 2024-08-06T20:31:17.3307829Z remote: Compressing objects: 100% (177/177) 2024-08-06T20:31:17.3308353Z remote: Compressing objects: 100% (177/177), done. 2024-08-06T20:31:44.1593635Z remote: Total 1008693 (delta 148), reused 259 (delta 131), pack-reused 1008383 2024-08-06T20:32:06.9208020Z [command]/usr/bin/git rev-parse --verify --quiet b9d86fa89636e301796d4201f36d86c73f6e49bc^{object} 2024-08-06T20:32:06.9236594Z b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:32:06.9242724Z ##[endgroup] 2024-08-06T20:32:06.9243342Z ##[group]Determining the checkout info 2024-08-06T20:32:06.9244756Z ##[endgroup] 2024-08-06T20:32:06.9245286Z ##[group]Checking out the ref 2024-08-06T20:32:06.9249306Z [command]/usr/bin/git checkout --quiet --force b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:32:08.7657580Z ##[endgroup] 2024-08-06T20:32:08.7658169Z ##[group]Setting up auth for fetching submodules 2024-08-06T20:32:08.7661891Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-06T20:32:08.7718755Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-06T20:32:08.7751232Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-06T20:32:08.7809208Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-06T20:32:08.7814931Z ##[endgroup] 2024-08-06T20:32:08.7815381Z ##[group]Fetching submodules 2024-08-06T20:32:08.7821246Z [command]/usr/bin/git submodule sync --recursive 2024-08-06T20:32:08.8151872Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-06T20:32:08.8464977Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-06T20:32:08.8467427Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-06T20:32:08.8470769Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-06T20:32:08.8474001Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-06T20:32:08.8477793Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-06T20:32:08.8481294Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-06T20:32:08.8484633Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-06T20:32:08.8488806Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-06T20:32:08.8492530Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-06T20:32:08.8496803Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-06T20:32:08.8500918Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-06T20:32:08.8505257Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-06T20:32:08.8510245Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-06T20:32:08.8514809Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-06T20:32:08.8519501Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-06T20:32:08.8524479Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-08-06T20:32:08.8529615Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:32:08.8534697Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-06T20:32:08.8540090Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-06T20:32:08.8545426Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-06T20:32:08.8551790Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-06T20:32:08.8557540Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-06T20:32:08.8564202Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-06T20:32:08.8570300Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-06T20:32:08.8576848Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-06T20:32:08.8583834Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-06T20:32:08.8592992Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-06T20:32:08.8599196Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-06T20:32:08.8605848Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-06T20:32:08.8612655Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-06T20:32:08.8619776Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-06T20:32:08.8626827Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-06T20:32:08.8634152Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-06T20:32:08.8641556Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-06T20:32:08.8648913Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-06T20:32:08.8676962Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-06T20:32:09.1006058Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-06T20:32:09.2782749Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-06T20:32:09.5405049Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-06T20:32:09.7769511Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-06T20:32:11.6851189Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-06T20:32:23.5889337Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-06T20:32:24.0248223Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-06T20:32:24.4333196Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-06T20:32:25.0622564Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-06T20:32:26.4480374Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-06T20:32:28.8223320Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-06T20:32:36.0205364Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-06T20:32:37.5955567Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-06T20:32:39.4489967Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-06T20:32:40.7138785Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-08-06T20:32:40.8708054Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-06T20:32:41.3091980Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-06T20:32:41.6113373Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-06T20:32:42.7918659Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-06T20:32:43.3155605Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-06T20:32:43.6000395Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-06T20:32:45.4303219Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-06T20:32:46.2015877Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-06T20:32:46.9155532Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-06T20:32:53.7983167Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-06T20:32:55.9792759Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-06T20:32:59.6287180Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-06T20:32:59.8531067Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-06T20:33:09.5722589Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-06T20:33:09.7101706Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-06T20:33:09.8878888Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-06T20:33:10.8244924Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-06T20:33:11.1108766Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-06T20:33:11.7595657Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-06T20:33:12.1801220Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-06T20:33:12.1929109Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-06T20:33:12.2023126Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-06T20:33:12.2303773Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-06T20:33:12.2758155Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-06T20:33:13.4638260Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-06T20:33:13.4902708Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-06T20:33:13.5443575Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-06T20:33:13.6659769Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-06T20:33:13.7052524Z Submodule path 'third_party/cudnn_frontend': checked out '98ca4e1941fe3263f128f74f10063a3ea35c7019' 2024-08-06T20:33:14.3496948Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-06T20:33:14.6611014Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-06T20:33:14.7581563Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-06T20:33:14.7598913Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:33:14.7602091Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:33:14.7605116Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:33:14.7608548Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:33:14.7612173Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:33:14.7638764Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-06T20:33:15.8548557Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-06T20:33:16.5268917Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-06T20:33:18.8485918Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-06T20:33:20.0198711Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-06T20:33:20.3563759Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-06T20:33:20.4763761Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-06T20:33:20.9956790Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-06T20:33:21.0732165Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-06T20:33:21.0867419Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-06T20:33:21.2307475Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-06T20:33:21.2794770Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-06T20:33:21.2898297Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-08-06T20:33:21.3363846Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-06T20:33:21.3654520Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-06T20:33:21.4186956Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-06T20:33:21.4326889Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-06T20:33:21.4344061Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-06T20:33:21.4370994Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-06T20:33:36.2365141Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-06T20:33:36.2569656Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-06T20:33:36.3706057Z Submodule path 'third_party/kineto': checked out 'da2f2682cabaf95d601fa2a9b7e0979f84fe7667' 2024-08-06T20:33:36.3724972Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:33:36.3727857Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:33:36.3731210Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:33:36.3757652Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-06T20:33:38.0386871Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-06T20:33:39.3042211Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-06T20:33:40.5828767Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-06T20:33:40.5845651Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:33:40.5848518Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:33:40.5851664Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:33:40.5855082Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:33:40.5858484Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:33:40.5862208Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:33:40.5865907Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:33:40.5870469Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:33:40.5897702Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-06T20:33:41.5661850Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-06T20:33:42.0238035Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-06T20:33:43.2578468Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-06T20:33:43.5601357Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-06T20:33:44.1538706Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-06T20:33:46.2985371Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-06T20:33:54.0574844Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-06T20:33:54.4903059Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-06T20:33:54.5110243Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-06T20:33:54.5534089Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-06T20:33:54.5679805Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-06T20:33:54.5696199Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:33:54.5724042Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-06T20:33:54.8819582Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-06T20:33:54.9024303Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-06T20:33:54.9500889Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-06T20:33:55.0718143Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-06T20:33:55.0899017Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-06T20:33:55.1358493Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-06T20:33:55.2049290Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-06T20:33:55.2492020Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-06T20:33:55.2765204Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-06T20:33:55.4055681Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-06T20:33:55.8339300Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-06T20:33:55.8376213Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-06T20:33:55.8379413Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-06T20:33:55.8406777Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-06T20:33:56.2715022Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-06T20:33:57.2142158Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-06T20:33:57.2538781Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-06T20:33:57.3347578Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-06T20:33:57.3359247Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:33:57.3362107Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:33:57.3365408Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:33:57.3368805Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:33:57.3372511Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:33:57.3376068Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:33:57.3379964Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:33:57.3383720Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:33:57.3410129Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-06T20:33:57.8003646Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-06T20:33:58.9669723Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-06T20:33:59.2965780Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-06T20:34:06.1803860Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-06T20:34:06.4361725Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-06T20:34:06.6444151Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-06T20:34:06.9740782Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-06T20:34:15.3277998Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-06T20:34:15.3761850Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-06T20:34:15.3934665Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-06T20:34:15.5175914Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-06T20:34:15.5317641Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-06T20:34:15.5474909Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-06T20:34:15.5648048Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-06T20:34:15.5665080Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:34:15.5669062Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:34:15.5695697Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-06T20:34:17.5935379Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-06T20:34:19.0621540Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-06T20:34:19.1169451Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-06T20:34:19.7057438Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-06T20:34:19.7183175Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-06T20:34:20.0477280Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-06T20:34:20.0499472Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:34:20.0502657Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-06T20:34:20.0530678Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-06T20:34:20.4738539Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-06T20:34:21.6276157Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-06T20:34:21.7134975Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-06T20:34:21.7228333Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-06T20:34:21.7361597Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-06T20:34:21.7793061Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-06T20:34:21.8120735Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-06T20:34:21.8615464Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-06T20:34:21.8908932Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-06T20:34:21.8926025Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:34:21.8929309Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:34:21.8932361Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:34:21.8935672Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:34:21.8962543Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-06T20:34:23.0432238Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-06T20:34:23.2763988Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-06T20:34:24.5787461Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-06T20:34:25.5940708Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-06T20:34:25.6108093Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-06T20:34:25.6945940Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-06T20:34:25.7274317Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-06T20:34:25.7290025Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:34:25.7317760Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-06T20:34:25.9087136Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-06T20:34:25.9123432Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-06T20:34:25.9443299Z Entering 'android/libs/fbjni' 2024-08-06T20:34:25.9485940Z Entering 'third_party/FP16' 2024-08-06T20:34:25.9527352Z Entering 'third_party/FXdiv' 2024-08-06T20:34:25.9568076Z Entering 'third_party/NNPACK' 2024-08-06T20:34:25.9610884Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:34:25.9652786Z Entering 'third_party/XNNPACK' 2024-08-06T20:34:25.9711584Z Entering 'third_party/benchmark' 2024-08-06T20:34:25.9752211Z Entering 'third_party/cpp-httplib' 2024-08-06T20:34:25.9793742Z Entering 'third_party/cpuinfo' 2024-08-06T20:34:25.9836260Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:34:25.9876992Z Entering 'third_party/cutlass' 2024-08-06T20:34:25.9926582Z Entering 'third_party/eigen' 2024-08-06T20:34:25.9969936Z Entering 'third_party/fbgemm' 2024-08-06T20:34:26.0011878Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:34:26.0051805Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:34:26.0092713Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:34:26.0139294Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:34:26.0179915Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:34:26.0221988Z Entering 'third_party/flatbuffers' 2024-08-06T20:34:26.0266445Z Entering 'third_party/fmt' 2024-08-06T20:34:26.0307953Z Entering 'third_party/foxi' 2024-08-06T20:34:26.0348311Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:34:26.0389726Z Entering 'third_party/gloo' 2024-08-06T20:34:26.0431701Z Entering 'third_party/googletest' 2024-08-06T20:34:26.0473493Z Entering 'third_party/ideep' 2024-08-06T20:34:26.0512451Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:34:26.0563579Z Entering 'third_party/ittapi' 2024-08-06T20:34:26.0604911Z Entering 'third_party/kineto' 2024-08-06T20:34:26.0646568Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:34:26.0687529Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:34:26.0729596Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:34:26.0770520Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:34:26.0811171Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:34:26.0851181Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:34:26.0894784Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:34:26.0935255Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:34:26.0975359Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:34:26.1017499Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:34:26.1060044Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:34:26.1101142Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:34:26.1142662Z Entering 'third_party/mimalloc' 2024-08-06T20:34:26.1183584Z Entering 'third_party/nccl/nccl' 2024-08-06T20:34:26.1224668Z Entering 'third_party/nlohmann' 2024-08-06T20:34:26.1267355Z Entering 'third_party/onnx' 2024-08-06T20:34:26.1325541Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:34:26.1366809Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:34:26.1410724Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:34:26.1453227Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:34:26.1493065Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:34:26.1533572Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:34:26.1574253Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:34:26.1616677Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:34:26.1657545Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:34:26.1697679Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:34:26.1736734Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:34:26.1779402Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:34:26.1821794Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:34:26.1884343Z Entering 'third_party/pocketfft' 2024-08-06T20:34:26.1924725Z Entering 'third_party/protobuf' 2024-08-06T20:34:26.1969052Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:34:26.2009277Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:34:26.2052822Z Entering 'third_party/psimd' 2024-08-06T20:34:26.2093722Z Entering 'third_party/pthreadpool' 2024-08-06T20:34:26.2135431Z Entering 'third_party/pybind11' 2024-08-06T20:34:26.2176158Z Entering 'third_party/python-peachpy' 2024-08-06T20:34:26.2217449Z Entering 'third_party/sleef' 2024-08-06T20:34:26.2259585Z Entering 'third_party/tensorpipe' 2024-08-06T20:34:26.2301108Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:34:26.2341118Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:34:26.2381251Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:34:26.2420459Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:34:26.2459491Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:34:26.2514772Z ##[endgroup] 2024-08-06T20:34:26.2519682Z ##[group]Persisting credentials for submodules 2024-08-06T20:34:26.2522620Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-06T20:34:26.2830227Z Entering 'android/libs/fbjni' 2024-08-06T20:34:26.2882818Z Entering 'third_party/FP16' 2024-08-06T20:34:26.2933933Z Entering 'third_party/FXdiv' 2024-08-06T20:34:26.2985260Z Entering 'third_party/NNPACK' 2024-08-06T20:34:26.3037164Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:34:26.3088803Z Entering 'third_party/XNNPACK' 2024-08-06T20:34:26.3157467Z Entering 'third_party/benchmark' 2024-08-06T20:34:26.3209787Z Entering 'third_party/cpp-httplib' 2024-08-06T20:34:26.3261563Z Entering 'third_party/cpuinfo' 2024-08-06T20:34:26.3313596Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:34:26.3365950Z Entering 'third_party/cutlass' 2024-08-06T20:34:26.3426452Z Entering 'third_party/eigen' 2024-08-06T20:34:26.3479593Z Entering 'third_party/fbgemm' 2024-08-06T20:34:26.3530585Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:34:26.3581192Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:34:26.3631939Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:34:26.3688331Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:34:26.3738710Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:34:26.3790188Z Entering 'third_party/flatbuffers' 2024-08-06T20:34:26.3843475Z Entering 'third_party/fmt' 2024-08-06T20:34:26.3893759Z Entering 'third_party/foxi' 2024-08-06T20:34:26.3944845Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:34:26.3996397Z Entering 'third_party/gloo' 2024-08-06T20:34:26.4047427Z Entering 'third_party/googletest' 2024-08-06T20:34:26.4097677Z Entering 'third_party/ideep' 2024-08-06T20:34:26.4147891Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:34:26.4205104Z Entering 'third_party/ittapi' 2024-08-06T20:34:26.4255780Z Entering 'third_party/kineto' 2024-08-06T20:34:26.4306270Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:34:26.4357165Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:34:26.4409665Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:34:26.4460189Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:34:26.4512673Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:34:26.4562726Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:34:26.4616141Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:34:26.4666206Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:34:26.4716607Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:34:26.4770115Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:34:26.4821313Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:34:26.4873209Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:34:26.4924177Z Entering 'third_party/mimalloc' 2024-08-06T20:34:26.4975163Z Entering 'third_party/nccl/nccl' 2024-08-06T20:34:26.5026434Z Entering 'third_party/nlohmann' 2024-08-06T20:34:26.5078225Z Entering 'third_party/onnx' 2024-08-06T20:34:26.5145712Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:34:26.5196578Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:34:26.5249269Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:34:26.5301629Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:34:26.5353078Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:34:26.5402235Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:34:26.5453360Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:34:26.5504774Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:34:26.5554602Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:34:26.5604773Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:34:26.5653521Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:34:26.5706436Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:34:26.5758540Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:34:26.5828966Z Entering 'third_party/pocketfft' 2024-08-06T20:34:26.5881613Z Entering 'third_party/protobuf' 2024-08-06T20:34:26.5936433Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:34:26.5986820Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:34:26.6038761Z Entering 'third_party/psimd' 2024-08-06T20:34:26.6089473Z Entering 'third_party/pthreadpool' 2024-08-06T20:34:26.6139803Z Entering 'third_party/pybind11' 2024-08-06T20:34:26.6191733Z Entering 'third_party/python-peachpy' 2024-08-06T20:34:26.6243126Z Entering 'third_party/sleef' 2024-08-06T20:34:26.6293737Z Entering 'third_party/tensorpipe' 2024-08-06T20:34:26.6343569Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:34:26.6394068Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:34:26.6443261Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:34:26.6493455Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:34:26.6542359Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:34:26.6609725Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-06T20:34:26.6916917Z Entering 'android/libs/fbjni' 2024-08-06T20:34:26.6965191Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-06T20:34:26.6983632Z Entering 'third_party/FP16' 2024-08-06T20:34:26.7032948Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-06T20:34:26.7052331Z Entering 'third_party/FXdiv' 2024-08-06T20:34:26.7099753Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-06T20:34:26.7119782Z Entering 'third_party/NNPACK' 2024-08-06T20:34:26.7166793Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-06T20:34:26.7186283Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:34:26.7234379Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-06T20:34:26.7253145Z Entering 'third_party/XNNPACK' 2024-08-06T20:34:26.7300100Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-06T20:34:26.7336529Z Entering 'third_party/benchmark' 2024-08-06T20:34:26.7384047Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-06T20:34:26.7403259Z Entering 'third_party/cpp-httplib' 2024-08-06T20:34:26.7450322Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-06T20:34:26.7469264Z Entering 'third_party/cpuinfo' 2024-08-06T20:34:26.7517754Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-06T20:34:26.7537180Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:34:26.7584773Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-06T20:34:26.7603933Z Entering 'third_party/cutlass' 2024-08-06T20:34:26.7651928Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-06T20:34:26.7679984Z Entering 'third_party/eigen' 2024-08-06T20:34:26.7727934Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-06T20:34:26.7748773Z Entering 'third_party/fbgemm' 2024-08-06T20:34:26.7796905Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-06T20:34:26.7815921Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:34:26.7864155Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-06T20:34:26.7882692Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:34:26.7929191Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-06T20:34:26.7948419Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:34:26.7996380Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-06T20:34:26.8021507Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:34:26.8068803Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:34:26.8087010Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:34:26.8134869Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-06T20:34:26.8154243Z Entering 'third_party/flatbuffers' 2024-08-06T20:34:26.8202268Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-06T20:34:26.8223512Z Entering 'third_party/fmt' 2024-08-06T20:34:26.8271655Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-06T20:34:26.8290060Z Entering 'third_party/foxi' 2024-08-06T20:34:26.8338494Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-08-06T20:34:26.8357417Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:34:26.8405059Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-06T20:34:26.8423139Z Entering 'third_party/gloo' 2024-08-06T20:34:26.8471368Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-06T20:34:26.8489883Z Entering 'third_party/googletest' 2024-08-06T20:34:26.8537333Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:34:26.8556728Z Entering 'third_party/ideep' 2024-08-06T20:34:26.8604671Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-06T20:34:26.8621918Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:34:26.8669787Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-06T20:34:26.8695649Z Entering 'third_party/ittapi' 2024-08-06T20:34:26.8743122Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-06T20:34:26.8761658Z Entering 'third_party/kineto' 2024-08-06T20:34:26.8809712Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-06T20:34:26.8829695Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:34:26.8877432Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-06T20:34:26.8895237Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:34:26.8944080Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-06T20:34:26.8963621Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:34:26.9011466Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-06T20:34:26.9030018Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:34:26.9077690Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-06T20:34:26.9096054Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:34:26.9143710Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-06T20:34:26.9160974Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:34:26.9209137Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-06T20:34:26.9229949Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:34:26.9278251Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-06T20:34:26.9296689Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:34:26.9344998Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:34:26.9363001Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:34:26.9410370Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-06T20:34:26.9430279Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:34:26.9477114Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-06T20:34:26.9496920Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:34:26.9545360Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-06T20:34:26.9563409Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:34:26.9610637Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-06T20:34:26.9630897Z Entering 'third_party/mimalloc' 2024-08-06T20:34:26.9678851Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-06T20:34:26.9697835Z Entering 'third_party/nccl/nccl' 2024-08-06T20:34:26.9744725Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-06T20:34:26.9764309Z Entering 'third_party/nlohmann' 2024-08-06T20:34:26.9811455Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-06T20:34:26.9831822Z Entering 'third_party/onnx' 2024-08-06T20:34:26.9879682Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-06T20:34:26.9915782Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:34:26.9963832Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-06T20:34:26.9982214Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:34:27.0030303Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-06T20:34:27.0050681Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:34:27.0100021Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-06T20:34:27.0120031Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:34:27.0167035Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-06T20:34:27.0185117Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:34:27.0233138Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:34:27.0252106Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:34:27.0299619Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-06T20:34:27.0317441Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:34:27.0365345Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-06T20:34:27.0384805Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:34:27.0432581Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-06T20:34:27.0450224Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:34:27.0498146Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-06T20:34:27.0517138Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:34:27.0564072Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-06T20:34:27.0580978Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:34:27.0628983Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-06T20:34:27.0648861Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:34:27.0696166Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-06T20:34:27.0716133Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:34:27.0763546Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-06T20:34:27.0804229Z Entering 'third_party/pocketfft' 2024-08-06T20:34:27.0852679Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-06T20:34:27.0871192Z Entering 'third_party/protobuf' 2024-08-06T20:34:27.0919647Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-06T20:34:27.0941450Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:34:27.0990071Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-06T20:34:27.1008826Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:34:27.1057259Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:34:27.1078026Z Entering 'third_party/psimd' 2024-08-06T20:34:27.1124832Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-06T20:34:27.1143635Z Entering 'third_party/pthreadpool' 2024-08-06T20:34:27.1191564Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-06T20:34:27.1210074Z Entering 'third_party/pybind11' 2024-08-06T20:34:27.1256962Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-06T20:34:27.1276138Z Entering 'third_party/python-peachpy' 2024-08-06T20:34:27.1323511Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-06T20:34:27.1341870Z Entering 'third_party/sleef' 2024-08-06T20:34:27.1389954Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-06T20:34:27.1408429Z Entering 'third_party/tensorpipe' 2024-08-06T20:34:27.1455984Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-06T20:34:27.1474480Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:34:27.1521015Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:34:27.1538972Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:34:27.1586547Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-06T20:34:27.1604745Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:34:27.1651718Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-06T20:34:27.1670171Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:34:27.1716663Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-06T20:34:27.1733827Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:34:27.1781555Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-06T20:34:27.2837768Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-06T20:34:27.3145070Z Entering 'android/libs/fbjni' 2024-08-06T20:34:27.3186876Z Entering 'third_party/FP16' 2024-08-06T20:34:27.3229780Z Entering 'third_party/FXdiv' 2024-08-06T20:34:27.3271217Z Entering 'third_party/NNPACK' 2024-08-06T20:34:27.3312797Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:34:27.3354437Z Entering 'third_party/XNNPACK' 2024-08-06T20:34:27.3413143Z Entering 'third_party/benchmark' 2024-08-06T20:34:27.3455089Z Entering 'third_party/cpp-httplib' 2024-08-06T20:34:27.3496842Z Entering 'third_party/cpuinfo' 2024-08-06T20:34:27.3538443Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:34:27.3580121Z Entering 'third_party/cutlass' 2024-08-06T20:34:27.3629421Z Entering 'third_party/eigen' 2024-08-06T20:34:27.3672922Z Entering 'third_party/fbgemm' 2024-08-06T20:34:27.3715724Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:34:27.3756201Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:34:27.3797181Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:34:27.3843607Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:34:27.3886644Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:34:27.3928153Z Entering 'third_party/flatbuffers' 2024-08-06T20:34:27.3974219Z Entering 'third_party/fmt' 2024-08-06T20:34:27.4015967Z Entering 'third_party/foxi' 2024-08-06T20:34:27.4056757Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:34:27.4099287Z Entering 'third_party/gloo' 2024-08-06T20:34:27.4141266Z Entering 'third_party/googletest' 2024-08-06T20:34:27.4182166Z Entering 'third_party/ideep' 2024-08-06T20:34:27.4223314Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:34:27.4271701Z Entering 'third_party/ittapi' 2024-08-06T20:34:27.4314649Z Entering 'third_party/kineto' 2024-08-06T20:34:27.4355794Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:34:27.4396782Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:34:27.4439671Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:34:27.4480757Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:34:27.4523431Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:34:27.4563477Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:34:27.4606671Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:34:27.4647450Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:34:27.4688533Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:34:27.4730174Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:34:27.4772185Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:34:27.4813273Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:34:27.4854708Z Entering 'third_party/mimalloc' 2024-08-06T20:34:27.4896855Z Entering 'third_party/nccl/nccl' 2024-08-06T20:34:27.4938619Z Entering 'third_party/nlohmann' 2024-08-06T20:34:27.4981431Z Entering 'third_party/onnx' 2024-08-06T20:34:27.5040594Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:34:27.5082579Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:34:27.5125611Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:34:27.5169498Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:34:27.5210419Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:34:27.5251111Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:34:27.5291577Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:34:27.5332968Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:34:27.5373346Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:34:27.5413045Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:34:27.5453237Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:34:27.5496109Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:34:27.5540091Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:34:27.5602677Z Entering 'third_party/pocketfft' 2024-08-06T20:34:27.5645225Z Entering 'third_party/protobuf' 2024-08-06T20:34:27.5691401Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:34:27.5732677Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:34:27.5775291Z Entering 'third_party/psimd' 2024-08-06T20:34:27.5816642Z Entering 'third_party/pthreadpool' 2024-08-06T20:34:27.5857851Z Entering 'third_party/pybind11' 2024-08-06T20:34:27.5899506Z Entering 'third_party/python-peachpy' 2024-08-06T20:34:27.5940922Z Entering 'third_party/sleef' 2024-08-06T20:34:27.5982369Z Entering 'third_party/tensorpipe' 2024-08-06T20:34:27.6025051Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:34:27.6066053Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:34:27.6105763Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:34:27.6146352Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:34:27.6185955Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:34:27.6243504Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-06T20:34:27.6554825Z Entering 'android/libs/fbjni' 2024-08-06T20:34:27.6596310Z Entering 'third_party/FP16' 2024-08-06T20:34:27.6638119Z Entering 'third_party/FXdiv' 2024-08-06T20:34:27.6680000Z Entering 'third_party/NNPACK' 2024-08-06T20:34:27.6721196Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:34:27.6763011Z Entering 'third_party/XNNPACK' 2024-08-06T20:34:27.6822027Z Entering 'third_party/benchmark' 2024-08-06T20:34:27.6864281Z Entering 'third_party/cpp-httplib' 2024-08-06T20:34:27.6905953Z Entering 'third_party/cpuinfo' 2024-08-06T20:34:27.6947862Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:34:27.6989762Z Entering 'third_party/cutlass' 2024-08-06T20:34:27.7039712Z Entering 'third_party/eigen' 2024-08-06T20:34:27.7083473Z Entering 'third_party/fbgemm' 2024-08-06T20:34:27.7125229Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:34:27.7165795Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:34:27.7206728Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:34:27.7254735Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:34:27.7295127Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:34:27.7337240Z Entering 'third_party/flatbuffers' 2024-08-06T20:34:27.7382355Z Entering 'third_party/fmt' 2024-08-06T20:34:27.7423684Z Entering 'third_party/foxi' 2024-08-06T20:34:27.7464918Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:34:27.7506447Z Entering 'third_party/gloo' 2024-08-06T20:34:27.7550346Z Entering 'third_party/googletest' 2024-08-06T20:34:27.7592945Z Entering 'third_party/ideep' 2024-08-06T20:34:27.7633340Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:34:27.7682014Z Entering 'third_party/ittapi' 2024-08-06T20:34:27.7724136Z Entering 'third_party/kineto' 2024-08-06T20:34:27.7765176Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:34:27.7806435Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:34:27.7850599Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:34:27.7892220Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:34:27.7932779Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:34:27.7973020Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:34:27.8015340Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:34:27.8055407Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:34:27.8096556Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:34:27.8138113Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:34:27.8180626Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:34:27.8221265Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:34:27.8263456Z Entering 'third_party/mimalloc' 2024-08-06T20:34:27.8306393Z Entering 'third_party/nccl/nccl' 2024-08-06T20:34:27.8348344Z Entering 'third_party/nlohmann' 2024-08-06T20:34:27.8391372Z Entering 'third_party/onnx' 2024-08-06T20:34:27.8449778Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:34:27.8491081Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:34:27.8534438Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:34:27.8576839Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:34:27.8617444Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:34:27.8657847Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:34:27.8698834Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:34:27.8740376Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:34:27.8780519Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:34:27.8820864Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:34:27.8859651Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:34:27.8902150Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:34:27.8944889Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:34:27.9006707Z Entering 'third_party/pocketfft' 2024-08-06T20:34:27.9047970Z Entering 'third_party/protobuf' 2024-08-06T20:34:27.9093716Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:34:27.9134073Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:34:27.9176790Z Entering 'third_party/psimd' 2024-08-06T20:34:27.9218441Z Entering 'third_party/pthreadpool' 2024-08-06T20:34:27.9261952Z Entering 'third_party/pybind11' 2024-08-06T20:34:27.9304356Z Entering 'third_party/python-peachpy' 2024-08-06T20:34:27.9345302Z Entering 'third_party/sleef' 2024-08-06T20:34:27.9386755Z Entering 'third_party/tensorpipe' 2024-08-06T20:34:27.9428102Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:34:27.9469097Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:34:27.9508814Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:34:27.9549761Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:34:27.9589634Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:34:27.9643576Z ##[endgroup] 2024-08-06T20:34:27.9685827Z [command]/usr/bin/git log -1 --format='%H' 2024-08-06T20:34:27.9712891Z 'b9d86fa89636e301796d4201f36d86c73f6e49bc' 2024-08-06T20:34:27.9899153Z Prepare all required actions 2024-08-06T20:34:27.9899911Z Getting action download info 2024-08-06T20:34:28.1228458Z ##[group]Run ./.github/actions/setup-linux 2024-08-06T20:34:28.1228954Z env: 2024-08-06T20:34:28.1229302Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:28.1229721Z ##[endgroup] 2024-08-06T20:34:28.1315903Z ##[group]Run set -euo pipefail 2024-08-06T20:34:28.1316423Z set -euo pipefail 2024-08-06T20:34:28.1317101Z function get_ec2_metadata() { 2024-08-06T20:34:28.1317652Z  # Pulled from instance metadata endpoint for EC2 2024-08-06T20:34:28.1318503Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-06T20:34:28.1319273Z  category=$1 2024-08-06T20:34:28.1319795Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-06T20:34:28.1320410Z  runner_name_str=i-0b13d1134efc770c9 2024-08-06T20:34:28.1320983Z  if [[ -f /.inarc ]]; then 2024-08-06T20:34:28.1321494Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-06T20:34:28.1322062Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-06T20:34:28.1322723Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-06T20:34:28.1323308Z  else 2024-08-06T20:34:28.1323825Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-06T20:34:28.1324422Z  fi 2024-08-06T20:34:28.1324775Z } 2024-08-06T20:34:28.1325198Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-06T20:34:28.1325806Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-06T20:34:28.1326460Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-06T20:34:28.1327057Z echo "system info $(uname -a)" 2024-08-06T20:34:28.1335290Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:28.1335828Z env: 2024-08-06T20:34:28.1336206Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:28.1336630Z ##[endgroup] 2024-08-06T20:34:28.1437630Z ami-id: ami-0ce0c36d7a00b20e2 2024-08-06T20:34:28.1505440Z instance-id: i-0b13d1134efc770c9 2024-08-06T20:34:28.1571694Z instance-type: g3.8xlarge 2024-08-06T20:34:28.1581000Z system info Linux ip-10-0-52-148.ec2.internal 4.14.336-257.562.amzn2.x86_64 #1 SMP Sat Feb 24 09:50:35 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-06T20:34:28.1616361Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-06T20:34:28.1617154Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-06T20:34:28.1624894Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:28.1625373Z env: 2024-08-06T20:34:28.1625962Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:28.1626296Z ##[endgroup] 2024-08-06T20:34:28.1707318Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-06T20:34:28.1707901Z if systemctl is-active --quiet docker; then 2024-08-06T20:34:28.1708380Z  echo "Docker daemon is running..."; 2024-08-06T20:34:28.1708804Z else 2024-08-06T20:34:28.1709259Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-06T20:34:28.1709774Z fi 2024-08-06T20:34:28.1717108Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:28.1717589Z env: 2024-08-06T20:34:28.1717881Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:28.1718240Z ##[endgroup] 2024-08-06T20:34:28.1763816Z Docker daemon is running... 2024-08-06T20:34:28.1819754Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-06T20:34:28.1820259Z with: 2024-08-06T20:34:28.1820534Z shell: bash 2024-08-06T20:34:28.1820832Z timeout_minutes: 5 2024-08-06T20:34:28.1821141Z max_attempts: 3 2024-08-06T20:34:28.1821452Z retry_wait_seconds: 30 2024-08-06T20:34:28.1822915Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-08-06T20:34:28.1824216Z polling_interval_seconds: 1 2024-08-06T20:34:28.1824567Z warning_on_retry: true 2024-08-06T20:34:28.1824913Z continue_on_error: false 2024-08-06T20:34:28.1825244Z env: 2024-08-06T20:34:28.1825999Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:28.1826439Z AWS_RETRY_MODE: standard 2024-08-06T20:34:28.1826782Z AWS_MAX_ATTEMPTS: 5 2024-08-06T20:34:28.1827095Z AWS_DEFAULT_REGION: us-east-1 2024-08-06T20:34:28.1827448Z ##[endgroup] 2024-08-06T20:34:29.1863221Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-06T20:34:29.1863984Z Configure a credential helper to remove this warning. See 2024-08-06T20:34:29.1864696Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-06T20:34:29.1865190Z 2024-08-06T20:34:29.1865370Z Login Succeeded 2024-08-06T20:34:29.2538511Z Command completed after 1 attempt(s). 2024-08-06T20:34:29.2598598Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-06T20:34:29.2599265Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-06T20:34:29.2599826Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-06T20:34:29.2607986Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:29.2608491Z env: 2024-08-06T20:34:29.2608770Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:29.2609125Z ##[endgroup] 2024-08-06T20:34:29.2692366Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-06T20:34:29.2693095Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-06T20:34:29.2693619Z # shellcheck disable=SC2046 2024-08-06T20:34:29.2694058Z docker stop $(docker ps -q) || true 2024-08-06T20:34:29.2694527Z # Prune all of the docker images 2024-08-06T20:34:29.2694936Z docker system prune -af 2024-08-06T20:34:29.2702407Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:29.2702887Z env: 2024-08-06T20:34:29.2703163Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:29.2703511Z ##[endgroup] 2024-08-06T20:34:29.3029080Z "docker stop" requires at least 1 argument. 2024-08-06T20:34:29.3029764Z See 'docker stop --help'. 2024-08-06T20:34:29.3030051Z 2024-08-06T20:34:29.3030247Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-06T20:34:29.3030588Z 2024-08-06T20:34:29.3030762Z Stop one or more running containers 2024-08-06T20:34:29.3210954Z Total reclaimed space: 0B 2024-08-06T20:34:29.3250939Z ##[group]Run set +e 2024-08-06T20:34:29.3251309Z set +e 2024-08-06T20:34:29.3251634Z set -x 2024-08-06T20:34:29.3251938Z  2024-08-06T20:34:29.3252254Z PT_DOMAIN=download.pytorch.org 2024-08-06T20:34:29.3252980Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-06T20:34:29.3253886Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-06T20:34:29.3254514Z # one is returned at random 2024-08-06T20:34:29.3255018Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-06T20:34:29.3255500Z  2024-08-06T20:34:29.3255799Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-06T20:34:29.3256358Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-06T20:34:29.3257004Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-06T20:34:29.3257502Z  2024-08-06T20:34:29.3257805Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-06T20:34:29.3258302Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-06T20:34:29.3258761Z  exit 1 2024-08-06T20:34:29.3259065Z  fi 2024-08-06T20:34:29.3259361Z fi 2024-08-06T20:34:29.3259818Z  2024-08-06T20:34:29.3260192Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-06T20:34:29.3260663Z  # Clean up any old records first 2024-08-06T20:34:29.3261106Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-06T20:34:29.3261532Z fi 2024-08-06T20:34:29.3261822Z  2024-08-06T20:34:29.3262209Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-06T20:34:29.3262875Z cat /etc/hosts 2024-08-06T20:34:29.3270824Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:29.3271306Z env: 2024-08-06T20:34:29.3271595Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:29.3271928Z ##[endgroup] 2024-08-06T20:34:29.3294658Z + PT_DOMAIN=download.pytorch.org 2024-08-06T20:34:29.3299948Z ++ dig -4 +short download.pytorch.org 2024-08-06T20:34:29.3300773Z ++ tail -n1 2024-08-06T20:34:29.3458004Z + RESOLVED_IP=18.160.10.28 2024-08-06T20:34:29.3458418Z + '[' -z 18.160.10.28 ']' 2024-08-06T20:34:29.3458805Z + grep -r download.pytorch.org /etc/hosts 2024-08-06T20:34:29.3467855Z 18.160.10.36 download.pytorch.org 2024-08-06T20:34:29.3468886Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-06T20:34:29.3590866Z + echo '18.160.10.28 download.pytorch.org' 2024-08-06T20:34:29.3591553Z + sudo tee -a /etc/hosts 2024-08-06T20:34:29.3682410Z 18.160.10.28 download.pytorch.org 2024-08-06T20:34:29.3697775Z + cat /etc/hosts 2024-08-06T20:34:29.3704434Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-06T20:34:29.3711234Z ::1 localhost6 localhost6.localdomain6 2024-08-06T20:34:29.3711730Z 18.160.10.28 download.pytorch.org 2024-08-06T20:34:29.3864620Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-06T20:34:29.3865243Z with: 2024-08-06T20:34:29.3866621Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:29.3867614Z docker-build-dir: .ci/docker 2024-08-06T20:34:29.3867998Z working-directory: . 2024-08-06T20:34:29.3868454Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:29.3868966Z force-push: false 2024-08-06T20:34:29.3869267Z env: 2024-08-06T20:34:29.3869559Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:29.3869907Z ##[endgroup] 2024-08-06T20:34:29.3898676Z ##[group]Run set -ex 2024-08-06T20:34:29.3899088Z set -ex 2024-08-06T20:34:29.3899395Z  2024-08-06T20:34:29.3899898Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-06T20:34:29.3900758Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-06T20:34:29.3901445Z # job could then download the pre-built image as usual 2024-08-06T20:34:29.3902078Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-06T20:34:29.3902672Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:29.3903233Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:29.3903732Z  2024-08-06T20:34:29.3904186Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-06T20:34:29.3904730Z  exit 0 2024-08-06T20:34:29.3905021Z else 2024-08-06T20:34:29.3905375Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:29.3906353Z fi 2024-08-06T20:34:29.3906629Z  2024-08-06T20:34:29.3907069Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-06T20:34:29.3907807Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-06T20:34:29.3908447Z  # use it as it is, but first let's extract the tag 2024-08-06T20:34:29.3909050Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-06T20:34:29.3909682Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:29.3910272Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:29.3910784Z else 2024-08-06T20:34:29.3911192Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-06T20:34:29.3911755Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:29.3912777Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:29.3913456Z fi 2024-08-06T20:34:29.3921074Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:29.3921543Z env: 2024-08-06T20:34:29.3921836Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:29.3922182Z REPO_NAME: pytorch 2024-08-06T20:34:29.3923099Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:29.3924075Z DOCKER_BUILD_DIR: .ci/docker 2024-08-06T20:34:29.3924551Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:29.3925024Z ##[endgroup] 2024-08-06T20:34:29.3949168Z + [[ ! -d .ci/docker ]] 2024-08-06T20:34:29.3949556Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-06T20:34:29.3949921Z + echo skip=false 2024-08-06T20:34:29.3951093Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-06T20:34:29.3956487Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:29.3957407Z ++ awk -F '[:,]' '{print $2}' 2024-08-06T20:34:29.3966979Z + DOCKER_TAG=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:29.3967556Z + echo docker-tag=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:29.3968587Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:29.4009314Z ##[group]Run set +e 2024-08-06T20:34:29.4009733Z set +e 2024-08-06T20:34:29.4010032Z set -x 2024-08-06T20:34:29.4010337Z  2024-08-06T20:34:29.4010627Z login() { 2024-08-06T20:34:29.4011222Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-06T20:34:29.4011870Z } 2024-08-06T20:34:29.4012168Z  2024-08-06T20:34:29.4012447Z retry () { 2024-08-06T20:34:29.4012818Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-06T20:34:29.4013231Z } 2024-08-06T20:34:29.4013501Z  2024-08-06T20:34:29.4013815Z retry login "${DOCKER_REGISTRY}" 2024-08-06T20:34:29.4014193Z  2024-08-06T20:34:29.4014623Z # Check if image already exists, if it does then skip building it 2024-08-06T20:34:29.4015234Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-06T20:34:29.4015706Z  exit 0 2024-08-06T20:34:29.4015997Z fi 2024-08-06T20:34:29.4016282Z  2024-08-06T20:34:29.4016716Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-06T20:34:29.4017429Z # be empty. The default action would be to continue rebuild the image 2024-08-06T20:34:29.4018084Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-06T20:34:29.4018647Z  # if we're on the base branch then use the parent commit 2024-08-06T20:34:29.4019160Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-06T20:34:29.4019573Z else 2024-08-06T20:34:29.4019984Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-06T20:34:29.4020576Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-06T20:34:29.4021044Z fi 2024-08-06T20:34:29.4021324Z  2024-08-06T20:34:29.4021643Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-06T20:34:29.4022108Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:29.4022515Z  2024-08-06T20:34:29.4023092Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-06T20:34:29.4023957Z  exit 0 2024-08-06T20:34:29.4024249Z fi 2024-08-06T20:34:29.4024539Z  2024-08-06T20:34:29.4024945Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-06T20:34:29.4026232Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-06T20:34:29.4026956Z  exit 1 2024-08-06T20:34:29.4027263Z fi 2024-08-06T20:34:29.4027540Z  2024-08-06T20:34:29.4028001Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-06T20:34:29.4028798Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-06T20:34:29.4029516Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-06T20:34:29.4030314Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-06T20:34:29.4031236Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-06T20:34:29.4031806Z fi 2024-08-06T20:34:29.4032076Z  2024-08-06T20:34:29.4032584Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:29.4039964Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:29.4040421Z env: 2024-08-06T20:34:29.4040713Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:29.4041073Z DOCKER_BUILD_DIR: .ci/docker 2024-08-06T20:34:29.4041493Z BASE_REVISION: 1736af7cf736184c356be1bb00f59fb2feea6d7d 2024-08-06T20:34:29.4042519Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:29.4043535Z DOCKER_TAG: 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:29.4044084Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:29.4044570Z ##[endgroup] 2024-08-06T20:34:29.4068192Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:29.4068760Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:29.4070477Z + aws ecr get-login-password --region us-east-1 2024-08-06T20:34:29.4071836Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:29.8786020Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-06T20:34:29.8787077Z Configure a credential helper to remove this warning. See 2024-08-06T20:34:29.8787809Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-06T20:34:29.8788258Z 2024-08-06T20:34:29.8788371Z Login Succeeded 2024-08-06T20:34:29.8800310Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:30.1186408Z { 2024-08-06T20:34:30.1187295Z "schemaVersion": 2, 2024-08-06T20:34:30.1188002Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-06T20:34:30.1188549Z "config": { 2024-08-06T20:34:30.1189029Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-06T20:34:30.1189700Z "size": 47454, 2024-08-06T20:34:30.1190256Z "digest": "sha256:8e4e19ae0377c273bc603b24bc49d406d6f755072d4534d040c86afbe761a129" 2024-08-06T20:34:30.1190907Z }, 2024-08-06T20:34:30.1191198Z "layers": [ 2024-08-06T20:34:30.1191483Z { 2024-08-06T20:34:30.1191946Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1192540Z "size": 28580681, 2024-08-06T20:34:30.1193116Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-08-06T20:34:30.1193762Z }, 2024-08-06T20:34:30.1194038Z { 2024-08-06T20:34:30.1194478Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1195047Z "size": 7943353, 2024-08-06T20:34:30.1195625Z "digest": "sha256:09d415c238d76b32a7ea4a6e6add9542db9a5641f7f183af70aae185d0709e58" 2024-08-06T20:34:30.1196605Z }, 2024-08-06T20:34:30.1196897Z { 2024-08-06T20:34:30.1197346Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1197928Z "size": 56226389, 2024-08-06T20:34:30.1198538Z "digest": "sha256:9fe6e2e61518cba6844870c03b285737daec35e62baf25ae7744629ed3a7b470" 2024-08-06T20:34:30.1199137Z }, 2024-08-06T20:34:30.1199520Z { 2024-08-06T20:34:30.1200545Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1201351Z "size": 187, 2024-08-06T20:34:30.1201983Z "digest": "sha256:41f16248e682693ff20b3032c1d5e5541cc87c5af898ae2ff9b24d2940e59100" 2024-08-06T20:34:30.1202685Z }, 2024-08-06T20:34:30.1203067Z { 2024-08-06T20:34:30.1203583Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1204179Z "size": 6885, 2024-08-06T20:34:30.1204802Z "digest": "sha256:95d7b781703928cf3c4eece39d800cccb76728c375fedf51ecd83833fb25e458" 2024-08-06T20:34:30.1205489Z }, 2024-08-06T20:34:30.1206081Z + exit 0 2024-08-06T20:34:30.1206421Z { 2024-08-06T20:34:30.1207143Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1207828Z "size": 1383662475, 2024-08-06T20:34:30.1208445Z "digest": "sha256:8f6c9048534734f4c873935293b7296225846ceb31c1a158400a67ea170dde7f" 2024-08-06T20:34:30.1209133Z }, 2024-08-06T20:34:30.1209517Z { 2024-08-06T20:34:30.1210035Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1239699Z "size": 62282, 2024-08-06T20:34:30.1240403Z "digest": "sha256:ab17245097e491b9368790714f9d90ed447bf0973bd677cfe6f2456d62b72a13" 2024-08-06T20:34:30.1240970Z }, 2024-08-06T20:34:30.1241461Z { 2024-08-06T20:34:30.1241892Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1242429Z "size": 1684, 2024-08-06T20:34:30.1242949Z "digest": "sha256:dfecd7e9912b76ed460b8edd5a85f1943666e38a973ab5458177cf2c7c3110e3" 2024-08-06T20:34:30.1243547Z }, 2024-08-06T20:34:30.1243801Z { 2024-08-06T20:34:30.1244237Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1244739Z "size": 1522, 2024-08-06T20:34:30.1245270Z "digest": "sha256:464a8f74544589bf7b57f9a4cadcb6681e5ed00758f6c35025e691df4e88e890" 2024-08-06T20:34:30.1245866Z }, 2024-08-06T20:34:30.1246124Z { 2024-08-06T20:34:30.1246553Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1247091Z "size": 2445884909, 2024-08-06T20:34:30.1247606Z "digest": "sha256:d67c111fa58886d8904335b5122039816c8e51af5844cb6fe4e14154d9a7c016" 2024-08-06T20:34:30.1248196Z }, 2024-08-06T20:34:30.1248477Z { 2024-08-06T20:34:30.1248888Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1249425Z "size": 85882, 2024-08-06T20:34:30.1249957Z "digest": "sha256:0723abde19c09431db80b5d0f0e003541f6232c2da5c4f35c635c861875937ae" 2024-08-06T20:34:30.1250534Z }, 2024-08-06T20:34:30.1250813Z { 2024-08-06T20:34:30.1251242Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1251748Z "size": 1820, 2024-08-06T20:34:30.1252277Z "digest": "sha256:ceceb076946ee95300822daccb5d3e4e45e6857d34996ef89305d9a078410f8e" 2024-08-06T20:34:30.1252866Z }, 2024-08-06T20:34:30.1253114Z { 2024-08-06T20:34:30.1253541Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1254050Z "size": 654860881, 2024-08-06T20:34:30.1254590Z "digest": "sha256:889eb8142c5187639e02cdc07adc75024843fd85a1f5c77d24c1a1123f5246bb" 2024-08-06T20:34:30.1255170Z }, 2024-08-06T20:34:30.1255423Z { 2024-08-06T20:34:30.1255850Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1256378Z "size": 542, 2024-08-06T20:34:30.1256885Z "digest": "sha256:af8081143f008c5cb1bd61f6d92fde22553cfc007b6a77c2fd2aaf6a9f0088a9" 2024-08-06T20:34:30.1257757Z }, 2024-08-06T20:34:30.1258030Z { 2024-08-06T20:34:30.1258441Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1258968Z "size": 1284, 2024-08-06T20:34:30.1259510Z "digest": "sha256:cc64a79090ad4059e2f3dbc1949b010b2733ab614bc9518ff6256ce2bed52115" 2024-08-06T20:34:30.1260078Z }, 2024-08-06T20:34:30.1260352Z { 2024-08-06T20:34:30.1260781Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1261291Z "size": 483, 2024-08-06T20:34:30.1261804Z "digest": "sha256:ad26542e0d01609840496a12691483f2c50eb8e23002a205becb5b695f9159fb" 2024-08-06T20:34:30.1262389Z }, 2024-08-06T20:34:30.1262642Z { 2024-08-06T20:34:30.1263070Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1263600Z "size": 91711921, 2024-08-06T20:34:30.1264118Z "digest": "sha256:7b42d8ae23671f21f6523a7e71cc50d5f87c4e993027a9bd924985cda6b57de9" 2024-08-06T20:34:30.1264707Z }, 2024-08-06T20:34:30.1264962Z { 2024-08-06T20:34:30.1265380Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1266459Z "size": 3229, 2024-08-06T20:34:30.1267184Z "digest": "sha256:6d46e3b97d9fd2bd0e7ceec44e4884756b26c594a1c1c51d1426ddd13f749db5" 2024-08-06T20:34:30.1267806Z }, 2024-08-06T20:34:30.1268082Z { 2024-08-06T20:34:30.1268520Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1269029Z "size": 1907, 2024-08-06T20:34:30.1269555Z "digest": "sha256:91f9dddffbee46981d9e74382704c1b53c87f1808350cf1dd518687805d3aa59" 2024-08-06T20:34:30.1270147Z }, 2024-08-06T20:34:30.1270407Z { 2024-08-06T20:34:30.1270837Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1271368Z "size": 699, 2024-08-06T20:34:30.1271881Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:34:30.1272480Z }, 2024-08-06T20:34:30.1272737Z { 2024-08-06T20:34:30.1273170Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1273720Z "size": 2813413845, 2024-08-06T20:34:30.1274265Z "digest": "sha256:eeedca475f8cafec096dfa0abc5c6f08938eb47c5d9d96fb001fd7f13a2531ca" 2024-08-06T20:34:30.1274881Z }, 2024-08-06T20:34:30.1275158Z { 2024-08-06T20:34:30.1275571Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1276094Z "size": 378, 2024-08-06T20:34:30.1276614Z "digest": "sha256:202c71391ade27762d5e9cafde793db67b4d5e4d6869debe2643d5092101c461" 2024-08-06T20:34:30.1277179Z }, 2024-08-06T20:34:30.1277450Z { 2024-08-06T20:34:30.1277880Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1278383Z "size": 12893, 2024-08-06T20:34:30.1278965Z "digest": "sha256:5af882ef27a2db3ee5e89c8730de161c0718b951f3f8477e63fe21517c9167f7" 2024-08-06T20:34:30.1279556Z }, 2024-08-06T20:34:30.1279806Z { 2024-08-06T20:34:30.1280240Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1280782Z "size": 802, 2024-08-06T20:34:30.1281289Z "digest": "sha256:bac001ac2fee5df3367ff9332bc1fcd706a8de3ad05a1885903f338197619520" 2024-08-06T20:34:30.1281887Z }, 2024-08-06T20:34:30.1282160Z { 2024-08-06T20:34:30.1282572Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1283100Z "size": 106, 2024-08-06T20:34:30.1283622Z "digest": "sha256:05896f1d52f7a8e2c4a53777e5575da3dc8fca8a786d14075b5bb204bcb4cf04" 2024-08-06T20:34:30.1284187Z }, 2024-08-06T20:34:30.1284459Z { 2024-08-06T20:34:30.1284892Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1285401Z "size": 502, 2024-08-06T20:34:30.1285916Z "digest": "sha256:49a0e418a175ac12f1b4b503e80e22679bc321e6163ed5b203af848204f0606a" 2024-08-06T20:34:30.1286496Z }, 2024-08-06T20:34:30.1286750Z { 2024-08-06T20:34:30.1287182Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1287712Z "size": 121477364, 2024-08-06T20:34:30.1288407Z "digest": "sha256:b5632121ff7a146647683888294bcca31252dd90ca681099dfb81d75c4710d52" 2024-08-06T20:34:30.1288990Z }, 2024-08-06T20:34:30.1289270Z { 2024-08-06T20:34:30.1289690Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1290225Z "size": 109, 2024-08-06T20:34:30.1290721Z "digest": "sha256:deceb46806b535404e025b9b532009eb211e79f85021171997af825253327d1a" 2024-08-06T20:34:30.1291530Z }, 2024-08-06T20:34:30.1291804Z { 2024-08-06T20:34:30.1292215Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1292746Z "size": 487, 2024-08-06T20:34:30.1293257Z "digest": "sha256:a2630e5fe166ae8ff146e4c14db0bdc1299ed2b6120108351c961adcd2375288" 2024-08-06T20:34:30.1293836Z }, 2024-08-06T20:34:30.1294111Z { 2024-08-06T20:34:30.1294539Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1295042Z "size": 297, 2024-08-06T20:34:30.1295578Z "digest": "sha256:603261fb379fff6cf0eb63de1787e7c98932f81e44e6af3f1645c9a977e0ca8c" 2024-08-06T20:34:30.1296163Z }, 2024-08-06T20:34:30.1296421Z { 2024-08-06T20:34:30.1297026Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1297722Z "size": 103, 2024-08-06T20:34:30.1298220Z "digest": "sha256:d8ac71337d62da7c4624f267380da27d834fcf174b873c8c53a963e6cc7193a4" 2024-08-06T20:34:30.1298801Z }, 2024-08-06T20:34:30.1299071Z { 2024-08-06T20:34:30.1299473Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1299995Z "size": 1470, 2024-08-06T20:34:30.1300520Z "digest": "sha256:2c2e35b979f5fc0ecccef6efc157a97f33db01a77958fa33855dc367544a5c46" 2024-08-06T20:34:30.1301090Z }, 2024-08-06T20:34:30.1301363Z { 2024-08-06T20:34:30.1301788Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1302304Z "size": 424146101, 2024-08-06T20:34:30.1302835Z "digest": "sha256:e559222c0ec62d839f78e4215ae553de028be354b7031e7393d1c6be6e91fd70" 2024-08-06T20:34:30.1303425Z }, 2024-08-06T20:34:30.1303677Z { 2024-08-06T20:34:30.1304104Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1304636Z "size": 159, 2024-08-06T20:34:30.1305149Z "digest": "sha256:c4dcb7e9d189eca2e414c757b5cf6d158055b29b0e71dbd082a5fb3585dba665" 2024-08-06T20:34:30.1306156Z }, 2024-08-06T20:34:30.1306433Z { 2024-08-06T20:34:30.1306841Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1307369Z "size": 563, 2024-08-06T20:34:30.1307889Z "digest": "sha256:2b9731d295d60278e361d491fc274e5d558a89ec5adeb9fdf342d7a8cc17bfec" 2024-08-06T20:34:30.1308457Z }, 2024-08-06T20:34:30.1308727Z { 2024-08-06T20:34:30.1309134Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1309666Z "size": 35807525, 2024-08-06T20:34:30.1310198Z "digest": "sha256:19da43312ebac2831028913760327e53d9812e80ab6b806cd7a920c549a3b554" 2024-08-06T20:34:30.1310773Z }, 2024-08-06T20:34:30.1311055Z { 2024-08-06T20:34:30.1311488Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1311999Z "size": 104, 2024-08-06T20:34:30.1312526Z "digest": "sha256:b846263d066405189d21552b7288e3efbc47e6cd2f236aa39f39d88ca576a33a" 2024-08-06T20:34:30.1313110Z }, 2024-08-06T20:34:30.1313364Z { 2024-08-06T20:34:30.1313794Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1314321Z "size": 424, 2024-08-06T20:34:30.1314827Z "digest": "sha256:3886e8f57dc887a87bb710fbda080ef4fd521f1ed2e560a895b000c05f6e9b0a" 2024-08-06T20:34:30.1315424Z }, 2024-08-06T20:34:30.1315700Z { 2024-08-06T20:34:30.1316099Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1316631Z "size": 20262048, 2024-08-06T20:34:30.1317164Z "digest": "sha256:58e13e7b6d7b78400d84805c843f08fb8f1d6583bccdb3373c8122b317e4d15a" 2024-08-06T20:34:30.1317725Z }, 2024-08-06T20:34:30.1318205Z { 2024-08-06T20:34:30.1318639Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1319153Z "size": 438, 2024-08-06T20:34:30.1319686Z "digest": "sha256:0eb80a0ac0006a45aa6b4581263cc1613ddf9545bd4a1d5a544310efe898ad81" 2024-08-06T20:34:30.1320273Z }, 2024-08-06T20:34:30.1320528Z { 2024-08-06T20:34:30.1320954Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1321474Z "size": 699, 2024-08-06T20:34:30.1321977Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:34:30.1322567Z }, 2024-08-06T20:34:30.1322835Z { 2024-08-06T20:34:30.1323237Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1323757Z "size": 141, 2024-08-06T20:34:30.1324282Z "digest": "sha256:a9f11b2b3622f5530c5cfefe973e96e028323fab1ff02e7a6337eb86fadebeb5" 2024-08-06T20:34:30.1324848Z }, 2024-08-06T20:34:30.1325116Z { 2024-08-06T20:34:30.1325530Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1326054Z "size": 135, 2024-08-06T20:34:30.1326695Z "digest": "sha256:d7e94279f8b2fe549e3f23637275d9dca8c831563ce84f1d44117eb297ddc871" 2024-08-06T20:34:30.1327288Z }, 2024-08-06T20:34:30.1327557Z { 2024-08-06T20:34:30.1327983Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1328486Z "size": 32, 2024-08-06T20:34:30.1329009Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:34:30.1329596Z }, 2024-08-06T20:34:30.1329845Z { 2024-08-06T20:34:30.1330274Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1330798Z "size": 190, 2024-08-06T20:34:30.1331300Z "digest": "sha256:0a3560fcd26c387cb3ff0638718fa9da02d3b08ad75b23c434b06d6a6d21cb55" 2024-08-06T20:34:30.1331900Z }, 2024-08-06T20:34:30.1332178Z { 2024-08-06T20:34:30.1332585Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1333127Z "size": 562, 2024-08-06T20:34:30.1333653Z "digest": "sha256:adf9d181cfbca5b0544f23797995d4ea6500cc4a635db9f7e18954c9044e9f4a" 2024-08-06T20:34:30.1334221Z }, 2024-08-06T20:34:30.1334507Z { 2024-08-06T20:34:30.1334940Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1335448Z "size": 43163050, 2024-08-06T20:34:30.1335985Z "digest": "sha256:bc18dd8315979255e24cf91e83757510baeb22f29a64d7579b02aca642215b58" 2024-08-06T20:34:30.1336567Z }, 2024-08-06T20:34:30.1336824Z { 2024-08-06T20:34:30.1337255Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1337784Z "size": 106, 2024-08-06T20:34:30.1338289Z "digest": "sha256:d62f2a6086195c833ef952a43eed18216c6cde79b07f8deeb4fb3aeea648f868" 2024-08-06T20:34:30.1338883Z }, 2024-08-06T20:34:30.1339158Z { 2024-08-06T20:34:30.1339565Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1340102Z "size": 1210, 2024-08-06T20:34:30.1340633Z "digest": "sha256:719a1c2abcfd7e4113aa2f03adeb54350a5e381c720688ea53bc0e18401ad33d" 2024-08-06T20:34:30.1341207Z }, 2024-08-06T20:34:30.1341482Z { 2024-08-06T20:34:30.1341971Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1342482Z "size": 699, 2024-08-06T20:34:30.1343012Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:34:30.1343592Z }, 2024-08-06T20:34:30.1343836Z { 2024-08-06T20:34:30.1344259Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1344758Z "size": 138, 2024-08-06T20:34:30.1345270Z "digest": "sha256:8e1355d516c5e0527c25402c664fc648a8a2b814090e6ee0fdd60f49a8826ac1" 2024-08-06T20:34:30.1346337Z }, 2024-08-06T20:34:30.1346595Z { 2024-08-06T20:34:30.1347023Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1347551Z "size": 117, 2024-08-06T20:34:30.1348232Z "digest": "sha256:701898e66f54fcf3b57d837ab651a876666391d9f9b51b9969ec842a23fd86f0" 2024-08-06T20:34:30.1348820Z }, 2024-08-06T20:34:30.1349095Z { 2024-08-06T20:34:30.1349511Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1350039Z "size": 1916643487, 2024-08-06T20:34:30.1350579Z "digest": "sha256:129b38cfed12360bb7c4d882141b5fb7f7337791383b9f75e4ffed521548f060" 2024-08-06T20:34:30.1351140Z }, 2024-08-06T20:34:30.1351410Z { 2024-08-06T20:34:30.1351838Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1352345Z "size": 172, 2024-08-06T20:34:30.1352853Z "digest": "sha256:1751651a072f518b34289de9dfa631c4e82c7958a5331061379684f9b9b11720" 2024-08-06T20:34:30.1353427Z }, 2024-08-06T20:34:30.1353677Z { 2024-08-06T20:34:30.1354102Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1354622Z "size": 905, 2024-08-06T20:34:30.1355130Z "digest": "sha256:fc956bf4e5b49119cab7cc7bda867c61bed76c76c1e113dfa7c7b184251f8173" 2024-08-06T20:34:30.1355737Z }, 2024-08-06T20:34:30.1356012Z { 2024-08-06T20:34:30.1356414Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1357089Z "size": 699, 2024-08-06T20:34:30.1357660Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:34:30.1358233Z }, 2024-08-06T20:34:30.1358508Z { 2024-08-06T20:34:30.1358937Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1359440Z "size": 134, 2024-08-06T20:34:30.1359946Z "digest": "sha256:874491333fb5b81d9cbbd71520163a6baf07e6f011dc148b1634b94e301972b8" 2024-08-06T20:34:30.1360530Z }, 2024-08-06T20:34:30.1360783Z { 2024-08-06T20:34:30.1361210Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1361736Z "size": 32, 2024-08-06T20:34:30.1362248Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:34:30.1362850Z }, 2024-08-06T20:34:30.1363100Z { 2024-08-06T20:34:30.1363529Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1364055Z "size": 156, 2024-08-06T20:34:30.1364578Z "digest": "sha256:365b4a0eec21fb9dfc58baf7b43a229ae3f8b638b17e9b4db5ee1cfaecb82de4" 2024-08-06T20:34:30.1365189Z }, 2024-08-06T20:34:30.1365459Z { 2024-08-06T20:34:30.1365864Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1366385Z "size": 1839, 2024-08-06T20:34:30.1366908Z "digest": "sha256:c1d4cac78640fc7d364011da807a363d6b248a2c46ccf99a6e1488c4974c17e3" 2024-08-06T20:34:30.1367476Z }, 2024-08-06T20:34:30.1367751Z { 2024-08-06T20:34:30.1368177Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1368680Z "size": 7529788, 2024-08-06T20:34:30.1369211Z "digest": "sha256:29afe2181ca6174db390fd0d9e8c4b8675fb99bb961c93936d4080ae92c8af57" 2024-08-06T20:34:30.1369808Z }, 2024-08-06T20:34:30.1370059Z { 2024-08-06T20:34:30.1370488Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1371015Z "size": 161, 2024-08-06T20:34:30.1371513Z "digest": "sha256:190dccfa687dc1e698f27965d8065a183168d7c66361505f368765456ff9f4bc" 2024-08-06T20:34:30.1372095Z }, 2024-08-06T20:34:30.1372368Z { 2024-08-06T20:34:30.1372775Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1373305Z "size": 7942, 2024-08-06T20:34:30.1373836Z "digest": "sha256:b01bea4b6634f47c51ede2b4a95ceade4e61356ca29551168b493ae6d4492f44" 2024-08-06T20:34:30.1374411Z }, 2024-08-06T20:34:30.1374685Z { 2024-08-06T20:34:30.1375119Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1375625Z "size": 8066, 2024-08-06T20:34:30.1376161Z "digest": "sha256:f13a9cc5968861df73c96e4ad67eef7f58576eeba8cdbee3d39e42ed3d6fe96d" 2024-08-06T20:34:30.1376764Z }, 2024-08-06T20:34:30.1377143Z { 2024-08-06T20:34:30.1377579Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1378107Z "size": 302, 2024-08-06T20:34:30.1378620Z "digest": "sha256:96d6dcd01b9f6c5557ba405d028442d808dd329f4f93e75f2c3ba0554e703a52" 2024-08-06T20:34:30.1379279Z }, 2024-08-06T20:34:30.1379557Z { 2024-08-06T20:34:30.1379970Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1380500Z "size": 7629762, 2024-08-06T20:34:30.1381004Z "digest": "sha256:fe134554f1b8a0509798c1f49e6f973f9b8b7f954eb1fda146f35a37c9c71599" 2024-08-06T20:34:30.1381588Z }, 2024-08-06T20:34:30.1381864Z { 2024-08-06T20:34:30.1382277Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1382804Z "size": 108, 2024-08-06T20:34:30.1383328Z "digest": "sha256:8068d364963fe87857fbb7c3aad42ddee962351a57eddc01b4282be12c69a074" 2024-08-06T20:34:30.1383892Z }, 2024-08-06T20:34:30.1384166Z { 2024-08-06T20:34:30.1384590Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1385112Z "size": 54145777, 2024-08-06T20:34:30.1386193Z "digest": "sha256:f307f545c1a5e086f18e23247c890ddd5b135b1cb7157807aeeaef38ceb3b7dc" 2024-08-06T20:34:30.1387406Z }, 2024-08-06T20:34:30.1387853Z { 2024-08-06T20:34:30.1388340Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1388860Z "size": 471, 2024-08-06T20:34:30.1389351Z "digest": "sha256:51c0b7eab50584e37e08f303a4bf3ba3141998800971452b2024a8b897858877" 2024-08-06T20:34:30.1389925Z }, 2024-08-06T20:34:30.1390374Z { 2024-08-06T20:34:30.1390774Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1391288Z "size": 1373229322, 2024-08-06T20:34:30.1391819Z "digest": "sha256:60aba96f31c695ec1e1433578783bab1257233794a6e8b5cc5c6365fdf68669e" 2024-08-06T20:34:30.1392381Z }, 2024-08-06T20:34:30.1392650Z { 2024-08-06T20:34:30.1393066Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1393594Z "size": 106, 2024-08-06T20:34:30.1394116Z "digest": "sha256:46db894a106ccd6c28198b9236ee84ac2973b37ae608caeb380dfaef91f13961" 2024-08-06T20:34:30.1394698Z }, 2024-08-06T20:34:30.1394958Z { 2024-08-06T20:34:30.1395384Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1395911Z "size": 557, 2024-08-06T20:34:30.1396395Z "digest": "sha256:0052583728e40920c255902250fe4a615b10ff00a5c0e7ed51a0075121d42fad" 2024-08-06T20:34:30.1397083Z }, 2024-08-06T20:34:30.1397353Z { 2024-08-06T20:34:30.1397757Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1398278Z "size": 36532090, 2024-08-06T20:34:30.1398825Z "digest": "sha256:8c6d352a3b51edbb546310cc3d97ded6aa8fd29a7d610e4d13fc54b5828fdd05" 2024-08-06T20:34:30.1399400Z }, 2024-08-06T20:34:30.1399671Z { 2024-08-06T20:34:30.1400075Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1400613Z "size": 111, 2024-08-06T20:34:30.1401142Z "digest": "sha256:380c4e16278f4e3bf0eb6aee3ff4a730e3a1adb82aeb1cfb8e4114f1759a2bcf" 2024-08-06T20:34:30.1401713Z }, 2024-08-06T20:34:30.1401992Z { 2024-08-06T20:34:30.1402426Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1402938Z "size": 32, 2024-08-06T20:34:30.1403461Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:34:30.1404047Z }, 2024-08-06T20:34:30.1404300Z { 2024-08-06T20:34:30.1404729Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1405252Z "size": 32, 2024-08-06T20:34:30.1405751Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:34:30.1406342Z }, 2024-08-06T20:34:30.1406608Z { 2024-08-06T20:34:30.1407015Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1407541Z "size": 32, 2024-08-06T20:34:30.1408237Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:34:30.1408805Z }, 2024-08-06T20:34:30.1409077Z { 2024-08-06T20:34:30.1409509Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:30.1410010Z "size": 32, 2024-08-06T20:34:30.1410528Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:34:30.1411110Z } 2024-08-06T20:34:30.1411361Z ] 2024-08-06T20:34:30.1411631Z } 2024-08-06T20:34:30.1493001Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-06T20:34:30.1493536Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-06T20:34:30.1494057Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-06T20:34:30.1502293Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:30.1502763Z env: 2024-08-06T20:34:30.1503061Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:30.1503993Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:30.1504974Z ##[endgroup] 2024-08-06T20:34:30.1533823Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9-02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:30.1596152Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-06T20:34:30.1596760Z with: 2024-08-06T20:34:30.1597601Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:30.1598664Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:30.1599160Z env: 2024-08-06T20:34:30.1599440Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:30.1599797Z ##[endgroup] 2024-08-06T20:34:30.1629112Z ##[group]Run set -x 2024-08-06T20:34:30.1629500Z set -x 2024-08-06T20:34:30.1629835Z set +e 2024-08-06T20:34:30.1630178Z  2024-08-06T20:34:30.1630507Z login() { 2024-08-06T20:34:30.1631095Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-06T20:34:30.1631755Z } 2024-08-06T20:34:30.1632055Z  2024-08-06T20:34:30.1632421Z retry () { 2024-08-06T20:34:30.1632785Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-06T20:34:30.1633200Z } 2024-08-06T20:34:30.1633500Z  2024-08-06T20:34:30.1633800Z retry login "${DOCKER_REGISTRY}" 2024-08-06T20:34:30.1634208Z  2024-08-06T20:34:30.1634500Z set -e 2024-08-06T20:34:30.1634926Z # ignore output since only exit code is used for conditional 2024-08-06T20:34:30.1635546Z # only pull docker image if it's not available locally 2024-08-06T20:34:30.1636221Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-06T20:34:30.1636850Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-06T20:34:30.1637264Z fi 2024-08-06T20:34:30.1644551Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:30.1645016Z env: 2024-08-06T20:34:30.1645314Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:30.1646243Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:30.1647307Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:30.1647778Z ##[endgroup] 2024-08-06T20:34:30.1670983Z + set +e 2024-08-06T20:34:30.1671415Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:30.1671928Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:30.1674268Z + aws ecr get-login-password --region us-east-1 2024-08-06T20:34:30.1675479Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:30.6286377Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-06T20:34:30.6288036Z Configure a credential helper to remove this warning. See 2024-08-06T20:34:30.6289200Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-06T20:34:30.6289741Z 2024-08-06T20:34:30.6289929Z Login Succeeded 2024-08-06T20:34:30.6299546Z + set -e 2024-08-06T20:34:30.6300559Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:30.6473872Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:30.6475488Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:30.8820024Z 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9: Pulling from pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9 2024-08-06T20:34:30.8820992Z 7a2c55901189: Pulling fs layer 2024-08-06T20:34:30.8821450Z 09d415c238d7: Pulling fs layer 2024-08-06T20:34:30.8821879Z 9fe6e2e61518: Pulling fs layer 2024-08-06T20:34:30.8822357Z 41f16248e682: Pulling fs layer 2024-08-06T20:34:30.8822991Z 95d7b7817039: Pulling fs layer 2024-08-06T20:34:30.8823471Z 8f6c90485347: Pulling fs layer 2024-08-06T20:34:30.8823910Z ab17245097e4: Pulling fs layer 2024-08-06T20:34:30.8824356Z dfecd7e9912b: Pulling fs layer 2024-08-06T20:34:30.8824791Z 464a8f745445: Pulling fs layer 2024-08-06T20:34:30.8825261Z d67c111fa588: Pulling fs layer 2024-08-06T20:34:30.8825945Z 0723abde19c0: Pulling fs layer 2024-08-06T20:34:30.8826399Z ceceb076946e: Pulling fs layer 2024-08-06T20:34:30.8826841Z 889eb8142c51: Pulling fs layer 2024-08-06T20:34:30.8827224Z af8081143f00: Pulling fs layer 2024-08-06T20:34:30.8827762Z cc64a79090ad: Pulling fs layer 2024-08-06T20:34:30.8828169Z ad26542e0d01: Pulling fs layer 2024-08-06T20:34:30.8828650Z 7b42d8ae2367: Pulling fs layer 2024-08-06T20:34:30.8829151Z ab17245097e4: Waiting 2024-08-06T20:34:30.8829501Z 6d46e3b97d9f: Pulling fs layer 2024-08-06T20:34:30.8829943Z 8f6c90485347: Waiting 2024-08-06T20:34:30.8830396Z 91f9dddffbee: Pulling fs layer 2024-08-06T20:34:30.8830823Z af8081143f00: Waiting 2024-08-06T20:34:30.8831219Z 464a8f745445: Waiting 2024-08-06T20:34:30.8831587Z dfecd7e9912b: Waiting 2024-08-06T20:34:30.8831959Z d67c111fa588: Waiting 2024-08-06T20:34:30.8832336Z 889eb8142c51: Waiting 2024-08-06T20:34:30.8832759Z 03fb78f207e3: Pulling fs layer 2024-08-06T20:34:30.8833124Z 0723abde19c0: Waiting 2024-08-06T20:34:30.8833559Z ad26542e0d01: Waiting 2024-08-06T20:34:30.8833932Z eeedca475f8c: Pulling fs layer 2024-08-06T20:34:30.8834526Z 202c71391ade: Pulling fs layer 2024-08-06T20:34:30.8835369Z 5af882ef27a2: Pulling fs layer 2024-08-06T20:34:30.8836245Z bac001ac2fee: Pulling fs layer 2024-08-06T20:34:30.8836685Z 7b42d8ae2367: Waiting 2024-08-06T20:34:30.8837101Z 05896f1d52f7: Pulling fs layer 2024-08-06T20:34:30.8837567Z 6d46e3b97d9f: Waiting 2024-08-06T20:34:30.8837889Z 95d7b7817039: Waiting 2024-08-06T20:34:30.8838312Z ceceb076946e: Waiting 2024-08-06T20:34:30.8838882Z 49a0e418a175: Pulling fs layer 2024-08-06T20:34:30.8839637Z b5632121ff7a: Pulling fs layer 2024-08-06T20:34:30.8840442Z 03fb78f207e3: Waiting 2024-08-06T20:34:30.8841306Z 5af882ef27a2: Waiting 2024-08-06T20:34:30.8841925Z 202c71391ade: Waiting 2024-08-06T20:34:30.8842326Z deceb46806b5: Pulling fs layer 2024-08-06T20:34:30.8842773Z a2630e5fe166: Pulling fs layer 2024-08-06T20:34:30.8843191Z 41f16248e682: Waiting 2024-08-06T20:34:30.8843577Z 49a0e418a175: Waiting 2024-08-06T20:34:30.8843980Z 603261fb379f: Pulling fs layer 2024-08-06T20:34:30.8844395Z d8ac71337d62: Pulling fs layer 2024-08-06T20:34:30.8844840Z 2c2e35b979f5: Pulling fs layer 2024-08-06T20:34:30.8845272Z deceb46806b5: Waiting 2024-08-06T20:34:30.8845660Z e559222c0ec6: Pulling fs layer 2024-08-06T20:34:30.8846087Z a2630e5fe166: Waiting 2024-08-06T20:34:30.8846479Z d8ac71337d62: Waiting 2024-08-06T20:34:30.8847127Z 603261fb379f: Waiting 2024-08-06T20:34:30.8847520Z 2c2e35b979f5: Waiting 2024-08-06T20:34:30.8847934Z c4dcb7e9d189: Pulling fs layer 2024-08-06T20:34:30.8848352Z b5632121ff7a: Waiting 2024-08-06T20:34:30.8848746Z bac001ac2fee: Waiting 2024-08-06T20:34:30.8849165Z 2b9731d295d6: Pulling fs layer 2024-08-06T20:34:30.8849593Z 19da43312eba: Pulling fs layer 2024-08-06T20:34:30.8850069Z b846263d0664: Pulling fs layer 2024-08-06T20:34:30.8850478Z 3886e8f57dc8: Pulling fs layer 2024-08-06T20:34:30.8850902Z 58e13e7b6d7b: Pulling fs layer 2024-08-06T20:34:30.8851329Z 05896f1d52f7: Waiting 2024-08-06T20:34:30.8851770Z 0eb80a0ac000: Pulling fs layer 2024-08-06T20:34:30.8852150Z 2b9731d295d6: Waiting 2024-08-06T20:34:30.8852536Z b846263d0664: Waiting 2024-08-06T20:34:30.8852914Z 58e13e7b6d7b: Waiting 2024-08-06T20:34:30.8853312Z 3886e8f57dc8: Waiting 2024-08-06T20:34:30.8853716Z a9f11b2b3622: Pulling fs layer 2024-08-06T20:34:30.8854141Z d7e94279f8b2: Pulling fs layer 2024-08-06T20:34:30.8854612Z 4f4fb700ef54: Pulling fs layer 2024-08-06T20:34:30.8855040Z 0a3560fcd26c: Pulling fs layer 2024-08-06T20:34:30.8855504Z 19da43312eba: Waiting 2024-08-06T20:34:30.8855893Z adf9d181cfbc: Pulling fs layer 2024-08-06T20:34:30.8856321Z bc18dd831597: Pulling fs layer 2024-08-06T20:34:30.8856952Z d62f2a608619: Pulling fs layer 2024-08-06T20:34:30.8857785Z 719a1c2abcfd: Pulling fs layer 2024-08-06T20:34:30.8858597Z 8e1355d516c5: Pulling fs layer 2024-08-06T20:34:30.8859420Z 701898e66f54: Pulling fs layer 2024-08-06T20:34:30.8860220Z a9f11b2b3622: Waiting 2024-08-06T20:34:30.8860979Z 129b38cfed12: Pulling fs layer 2024-08-06T20:34:30.8861792Z d7e94279f8b2: Waiting 2024-08-06T20:34:30.8862499Z bc18dd831597: Waiting 2024-08-06T20:34:30.8863225Z 1751651a072f: Pulling fs layer 2024-08-06T20:34:30.8864008Z 4f4fb700ef54: Waiting 2024-08-06T20:34:30.8864733Z adf9d181cfbc: Waiting 2024-08-06T20:34:30.8865446Z 719a1c2abcfd: Waiting 2024-08-06T20:34:30.8866419Z fc956bf4e5b4: Pulling fs layer 2024-08-06T20:34:30.8867285Z 874491333fb5: Pulling fs layer 2024-08-06T20:34:30.8868084Z 365b4a0eec21: Pulling fs layer 2024-08-06T20:34:30.8868909Z c1d4cac78640: Pulling fs layer 2024-08-06T20:34:30.8869721Z 874491333fb5: Waiting 2024-08-06T20:34:30.8870446Z 29afe2181ca6: Pulling fs layer 2024-08-06T20:34:30.8871343Z 190dccfa687d: Pulling fs layer 2024-08-06T20:34:30.8872221Z b01bea4b6634: Pulling fs layer 2024-08-06T20:34:30.8872860Z f13a9cc59688: Pulling fs layer 2024-08-06T20:34:30.8873293Z 1751651a072f: Waiting 2024-08-06T20:34:30.8873897Z d62f2a608619: Waiting 2024-08-06T20:34:30.8874354Z 96d6dcd01b9f: Pulling fs layer 2024-08-06T20:34:30.8874810Z fe134554f1b8: Pulling fs layer 2024-08-06T20:34:30.8875241Z 8068d364963f: Pulling fs layer 2024-08-06T20:34:30.8875660Z f307f545c1a5: Pulling fs layer 2024-08-06T20:34:30.8876098Z 51c0b7eab505: Pulling fs layer 2024-08-06T20:34:30.8876507Z 190dccfa687d: Waiting 2024-08-06T20:34:30.8876912Z 60aba96f31c6: Pulling fs layer 2024-08-06T20:34:30.8877350Z 46db894a106c: Pulling fs layer 2024-08-06T20:34:30.8877780Z 0052583728e4: Pulling fs layer 2024-08-06T20:34:30.8878204Z 96d6dcd01b9f: Waiting 2024-08-06T20:34:30.8878613Z 8c6d352a3b51: Pulling fs layer 2024-08-06T20:34:30.8879185Z 380c4e16278f: Pulling fs layer 2024-08-06T20:34:30.8879611Z fe134554f1b8: Waiting 2024-08-06T20:34:30.8880008Z 701898e66f54: Waiting 2024-08-06T20:34:30.8880376Z 8068d364963f: Waiting 2024-08-06T20:34:30.8880762Z 129b38cfed12: Waiting 2024-08-06T20:34:30.8881154Z 8c6d352a3b51: Waiting 2024-08-06T20:34:30.8881525Z 380c4e16278f: Waiting 2024-08-06T20:34:30.8881913Z 8e1355d516c5: Waiting 2024-08-06T20:34:30.8882304Z 60aba96f31c6: Waiting 2024-08-06T20:34:30.8882678Z f13a9cc59688: Waiting 2024-08-06T20:34:30.8883065Z c1d4cac78640: Waiting 2024-08-06T20:34:30.8883458Z 365b4a0eec21: Waiting 2024-08-06T20:34:30.8883834Z 29afe2181ca6: Waiting 2024-08-06T20:34:30.8884227Z f307f545c1a5: Waiting 2024-08-06T20:34:30.8884604Z 51c0b7eab505: Waiting 2024-08-06T20:34:30.8885004Z b01bea4b6634: Waiting 2024-08-06T20:34:30.8885603Z 46db894a106c: Waiting 2024-08-06T20:34:30.8885976Z 0052583728e4: Waiting 2024-08-06T20:34:31.0308706Z 09d415c238d7: Verifying Checksum 2024-08-06T20:34:31.0309215Z 09d415c238d7: Download complete 2024-08-06T20:34:31.1030095Z 41f16248e682: Verifying Checksum 2024-08-06T20:34:31.1030638Z 41f16248e682: Download complete 2024-08-06T20:34:31.1623594Z 95d7b7817039: Download complete 2024-08-06T20:34:31.2212991Z 7a2c55901189: Verifying Checksum 2024-08-06T20:34:31.2213639Z 7a2c55901189: Download complete 2024-08-06T20:34:31.2923598Z ab17245097e4: Download complete 2024-08-06T20:34:31.3677418Z dfecd7e9912b: Download complete 2024-08-06T20:34:31.5009522Z 464a8f745445: Verifying Checksum 2024-08-06T20:34:31.5009995Z 464a8f745445: Download complete 2024-08-06T20:34:31.5925586Z 9fe6e2e61518: Download complete 2024-08-06T20:34:31.6716665Z 0723abde19c0: Download complete 2024-08-06T20:34:32.0363812Z ceceb076946e: Download complete 2024-08-06T20:34:32.5128103Z 7a2c55901189: Pull complete 2024-08-06T20:34:32.8601335Z 09d415c238d7: Pull complete 2024-08-06T20:34:34.0274239Z 9fe6e2e61518: Pull complete 2024-08-06T20:34:34.0942042Z 41f16248e682: Pull complete 2024-08-06T20:34:34.1479775Z 95d7b7817039: Pull complete 2024-08-06T20:34:38.6238683Z 889eb8142c51: Verifying Checksum 2024-08-06T20:34:38.6239510Z 889eb8142c51: Download complete 2024-08-06T20:34:38.6972120Z af8081143f00: Verifying Checksum 2024-08-06T20:34:38.6972660Z af8081143f00: Download complete 2024-08-06T20:34:38.7746814Z cc64a79090ad: Verifying Checksum 2024-08-06T20:34:38.7747819Z cc64a79090ad: Download complete 2024-08-06T20:34:38.8383833Z ad26542e0d01: Verifying Checksum 2024-08-06T20:34:38.8384375Z ad26542e0d01: Download complete 2024-08-06T20:34:39.7965776Z 7b42d8ae2367: Verifying Checksum 2024-08-06T20:34:39.7966275Z 7b42d8ae2367: Download complete 2024-08-06T20:34:39.8780006Z 6d46e3b97d9f: Verifying Checksum 2024-08-06T20:34:39.8780724Z 6d46e3b97d9f: Download complete 2024-08-06T20:34:39.9610079Z 91f9dddffbee: Verifying Checksum 2024-08-06T20:34:39.9610416Z 91f9dddffbee: Download complete 2024-08-06T20:34:40.0467071Z 03fb78f207e3: Verifying Checksum 2024-08-06T20:34:40.0467540Z 03fb78f207e3: Download complete 2024-08-06T20:34:45.0486284Z 8f6c90485347: Verifying Checksum 2024-08-06T20:34:45.0486759Z 8f6c90485347: Download complete 2024-08-06T20:34:45.1258254Z 202c71391ade: Verifying Checksum 2024-08-06T20:34:45.1258793Z 202c71391ade: Download complete 2024-08-06T20:34:45.2848651Z 5af882ef27a2: Verifying Checksum 2024-08-06T20:34:45.2849185Z 5af882ef27a2: Download complete 2024-08-06T20:34:45.3589468Z bac001ac2fee: Verifying Checksum 2024-08-06T20:34:45.3589905Z bac001ac2fee: Download complete 2024-08-06T20:34:45.4385715Z 05896f1d52f7: Verifying Checksum 2024-08-06T20:34:45.4386254Z 05896f1d52f7: Download complete 2024-08-06T20:34:45.5188340Z 49a0e418a175: Download complete 2024-08-06T20:34:46.7996374Z b5632121ff7a: Verifying Checksum 2024-08-06T20:34:46.7996866Z b5632121ff7a: Download complete 2024-08-06T20:34:46.8969532Z deceb46806b5: Verifying Checksum 2024-08-06T20:34:46.8970201Z deceb46806b5: Download complete 2024-08-06T20:34:46.9744085Z a2630e5fe166: Verifying Checksum 2024-08-06T20:34:46.9744529Z a2630e5fe166: Download complete 2024-08-06T20:34:47.0576300Z 603261fb379f: Verifying Checksum 2024-08-06T20:34:47.0576765Z 603261fb379f: Download complete 2024-08-06T20:34:47.1346638Z d8ac71337d62: Verifying Checksum 2024-08-06T20:34:47.1347142Z d8ac71337d62: Download complete 2024-08-06T20:34:47.2473133Z 2c2e35b979f5: Verifying Checksum 2024-08-06T20:34:47.2473549Z 2c2e35b979f5: Download complete 2024-08-06T20:34:51.5405234Z e559222c0ec6: Verifying Checksum 2024-08-06T20:34:51.5405729Z e559222c0ec6: Download complete 2024-08-06T20:34:51.6433449Z c4dcb7e9d189: Verifying Checksum 2024-08-06T20:34:51.6434297Z c4dcb7e9d189: Download complete 2024-08-06T20:34:51.7297602Z 2b9731d295d6: Verifying Checksum 2024-08-06T20:34:51.7298079Z 2b9731d295d6: Download complete 2024-08-06T20:34:52.1394394Z 19da43312eba: Verifying Checksum 2024-08-06T20:34:52.1394842Z 19da43312eba: Download complete 2024-08-06T20:34:52.2118706Z b846263d0664: Verifying Checksum 2024-08-06T20:34:52.2119148Z b846263d0664: Download complete 2024-08-06T20:34:52.2934510Z 3886e8f57dc8: Verifying Checksum 2024-08-06T20:34:52.2934967Z 3886e8f57dc8: Download complete 2024-08-06T20:34:52.6059567Z 58e13e7b6d7b: Verifying Checksum 2024-08-06T20:34:52.6060258Z 58e13e7b6d7b: Download complete 2024-08-06T20:34:52.6864950Z 0eb80a0ac000: Verifying Checksum 2024-08-06T20:34:52.6865396Z 0eb80a0ac000: Download complete 2024-08-06T20:34:52.7640771Z a9f11b2b3622: Verifying Checksum 2024-08-06T20:34:52.7641272Z a9f11b2b3622: Download complete 2024-08-06T20:34:52.8576032Z d7e94279f8b2: Verifying Checksum 2024-08-06T20:34:52.8576779Z d7e94279f8b2: Download complete 2024-08-06T20:34:52.8644176Z 4f4fb700ef54: Verifying Checksum 2024-08-06T20:34:52.8644597Z 4f4fb700ef54: Download complete 2024-08-06T20:34:52.9806103Z 0a3560fcd26c: Verifying Checksum 2024-08-06T20:34:52.9806975Z 0a3560fcd26c: Download complete 2024-08-06T20:34:53.0726502Z adf9d181cfbc: Verifying Checksum 2024-08-06T20:34:53.0727147Z adf9d181cfbc: Download complete 2024-08-06T20:34:53.6300650Z bc18dd831597: Verifying Checksum 2024-08-06T20:34:53.6301121Z bc18dd831597: Download complete 2024-08-06T20:34:53.7199752Z d62f2a608619: Verifying Checksum 2024-08-06T20:34:53.7200486Z d62f2a608619: Download complete 2024-08-06T20:34:53.7939169Z 719a1c2abcfd: Download complete 2024-08-06T20:34:53.8914836Z 8e1355d516c5: Verifying Checksum 2024-08-06T20:34:53.8915514Z 8e1355d516c5: Download complete 2024-08-06T20:34:53.9759589Z 701898e66f54: Download complete 2024-08-06T20:34:56.0120349Z d67c111fa588: Verifying Checksum 2024-08-06T20:34:56.0120815Z d67c111fa588: Download complete 2024-08-06T20:34:56.0841817Z 1751651a072f: Verifying Checksum 2024-08-06T20:34:56.0842258Z 1751651a072f: Download complete 2024-08-06T20:34:56.1559413Z fc956bf4e5b4: Verifying Checksum 2024-08-06T20:34:56.1560206Z fc956bf4e5b4: Download complete 2024-08-06T20:34:56.2350645Z 874491333fb5: Download complete 2024-08-06T20:34:56.3467540Z 365b4a0eec21: Verifying Checksum 2024-08-06T20:34:56.3468016Z 365b4a0eec21: Download complete 2024-08-06T20:34:56.4274612Z c1d4cac78640: Verifying Checksum 2024-08-06T20:34:56.4275049Z c1d4cac78640: Download complete 2024-08-06T20:34:56.6503821Z 29afe2181ca6: Verifying Checksum 2024-08-06T20:34:56.6504606Z 29afe2181ca6: Download complete 2024-08-06T20:34:56.7218666Z 190dccfa687d: Verifying Checksum 2024-08-06T20:34:56.7219427Z 190dccfa687d: Download complete 2024-08-06T20:34:56.7868424Z b01bea4b6634: Verifying Checksum 2024-08-06T20:34:56.7868844Z b01bea4b6634: Download complete 2024-08-06T20:34:56.8654701Z f13a9cc59688: Verifying Checksum 2024-08-06T20:34:56.9495124Z f13a9cc59688: Download complete 2024-08-06T20:34:56.9495548Z 96d6dcd01b9f: Verifying Checksum 2024-08-06T20:34:56.9495904Z 96d6dcd01b9f: Download complete 2024-08-06T20:34:57.1172058Z fe134554f1b8: Verifying Checksum 2024-08-06T20:34:57.1172819Z fe134554f1b8: Download complete 2024-08-06T20:34:57.1863232Z 8068d364963f: Download complete 2024-08-06T20:34:57.7839550Z f307f545c1a5: Verifying Checksum 2024-08-06T20:34:57.7840102Z f307f545c1a5: Download complete 2024-08-06T20:34:57.8512518Z 51c0b7eab505: Verifying Checksum 2024-08-06T20:34:57.8512940Z 51c0b7eab505: Download complete 2024-08-06T20:35:03.7141707Z 8f6c90485347: Pull complete 2024-08-06T20:35:03.7654557Z ab17245097e4: Pull complete 2024-08-06T20:35:03.8199147Z dfecd7e9912b: Pull complete 2024-08-06T20:35:03.8943281Z 464a8f745445: Pull complete 2024-08-06T20:35:08.3109837Z eeedca475f8c: Verifying Checksum 2024-08-06T20:35:08.3110313Z eeedca475f8c: Download complete 2024-08-06T20:35:08.3850674Z 46db894a106c: Verifying Checksum 2024-08-06T20:35:08.3851098Z 46db894a106c: Download complete 2024-08-06T20:35:08.4608722Z 0052583728e4: Verifying Checksum 2024-08-06T20:35:08.4609388Z 0052583728e4: Download complete 2024-08-06T20:35:08.8815782Z 8c6d352a3b51: Verifying Checksum 2024-08-06T20:35:08.8816628Z 8c6d352a3b51: Download complete 2024-08-06T20:35:08.9488959Z 380c4e16278f: Verifying Checksum 2024-08-06T20:35:08.9489762Z 380c4e16278f: Download complete 2024-08-06T20:35:11.6378929Z 60aba96f31c6: Verifying Checksum 2024-08-06T20:35:11.6379567Z 60aba96f31c6: Download complete 2024-08-06T20:35:13.1988363Z 129b38cfed12: Verifying Checksum 2024-08-06T20:35:13.1988859Z 129b38cfed12: Download complete 2024-08-06T20:35:42.0115506Z d67c111fa588: Pull complete 2024-08-06T20:35:44.6125726Z 0723abde19c0: Pull complete 2024-08-06T20:35:47.0560765Z ceceb076946e: Pull complete 2024-08-06T20:36:03.5895880Z 889eb8142c51: Pull complete 2024-08-06T20:36:06.0639250Z af8081143f00: Pull complete 2024-08-06T20:36:08.5061740Z cc64a79090ad: Pull complete 2024-08-06T20:36:10.9782606Z ad26542e0d01: Pull complete 2024-08-06T20:36:16.7806562Z 7b42d8ae2367: Pull complete 2024-08-06T20:36:19.2208496Z 6d46e3b97d9f: Pull complete 2024-08-06T20:36:21.9621746Z 91f9dddffbee: Pull complete 2024-08-06T20:36:24.4061127Z 03fb78f207e3: Pull complete 2024-08-06T20:37:30.6617436Z eeedca475f8c: Pull complete 2024-08-06T20:37:34.2228664Z 202c71391ade: Pull complete 2024-08-06T20:37:38.9207611Z 5af882ef27a2: Pull complete 2024-08-06T20:37:42.3796144Z bac001ac2fee: Pull complete 2024-08-06T20:37:44.4470056Z 05896f1d52f7: Pull complete 2024-08-06T20:37:44.4924875Z 49a0e418a175: Pull complete 2024-08-06T20:37:48.8089528Z b5632121ff7a: Pull complete 2024-08-06T20:37:48.8490183Z deceb46806b5: Pull complete 2024-08-06T20:37:48.8972523Z a2630e5fe166: Pull complete 2024-08-06T20:37:48.9436592Z 603261fb379f: Pull complete 2024-08-06T20:37:48.9881251Z d8ac71337d62: Pull complete 2024-08-06T20:37:49.0289288Z 2c2e35b979f5: Pull complete 2024-08-06T20:38:01.1846219Z e559222c0ec6: Pull complete 2024-08-06T20:38:03.4990650Z c4dcb7e9d189: Pull complete 2024-08-06T20:38:03.5457181Z 2b9731d295d6: Pull complete 2024-08-06T20:38:04.5355811Z 19da43312eba: Pull complete 2024-08-06T20:38:04.5728613Z b846263d0664: Pull complete 2024-08-06T20:38:04.6139717Z 3886e8f57dc8: Pull complete 2024-08-06T20:38:05.0011861Z 58e13e7b6d7b: Pull complete 2024-08-06T20:38:05.0414275Z 0eb80a0ac000: Pull complete 2024-08-06T20:38:05.1494899Z a9f11b2b3622: Pull complete 2024-08-06T20:38:05.1935454Z d7e94279f8b2: Pull complete 2024-08-06T20:38:05.2522369Z 4f4fb700ef54: Pull complete 2024-08-06T20:38:05.3008323Z 0a3560fcd26c: Pull complete 2024-08-06T20:38:05.3408780Z adf9d181cfbc: Pull complete 2024-08-06T20:38:07.8518452Z bc18dd831597: Pull complete 2024-08-06T20:38:07.8983715Z d62f2a608619: Pull complete 2024-08-06T20:38:07.9403394Z 719a1c2abcfd: Pull complete 2024-08-06T20:38:08.0460760Z 8e1355d516c5: Pull complete 2024-08-06T20:38:08.0949948Z 701898e66f54: Pull complete 2024-08-06T20:38:53.3441259Z 129b38cfed12: Pull complete 2024-08-06T20:38:55.7526393Z 1751651a072f: Pull complete 2024-08-06T20:38:58.2182070Z fc956bf4e5b4: Pull complete 2024-08-06T20:39:03.1230560Z 874491333fb5: Pull complete 2024-08-06T20:39:07.6324873Z 365b4a0eec21: Pull complete 2024-08-06T20:39:10.0749288Z c1d4cac78640: Pull complete 2024-08-06T20:39:12.7658422Z 29afe2181ca6: Pull complete 2024-08-06T20:39:15.8250145Z 190dccfa687d: Pull complete 2024-08-06T20:39:18.3592160Z b01bea4b6634: Pull complete 2024-08-06T20:39:21.0101088Z f13a9cc59688: Pull complete 2024-08-06T20:39:23.8835152Z 96d6dcd01b9f: Pull complete 2024-08-06T20:39:26.1896131Z fe134554f1b8: Pull complete 2024-08-06T20:39:26.2328639Z 8068d364963f: Pull complete 2024-08-06T20:39:28.6128583Z f307f545c1a5: Pull complete 2024-08-06T20:39:28.6573197Z 51c0b7eab505: Pull complete 2024-08-06T20:39:45.4788262Z 60aba96f31c6: Pull complete 2024-08-06T20:39:45.5318745Z 46db894a106c: Pull complete 2024-08-06T20:39:45.5811291Z 0052583728e4: Pull complete 2024-08-06T20:39:46.2540779Z 8c6d352a3b51: Pull complete 2024-08-06T20:39:46.3541413Z 380c4e16278f: Pull complete 2024-08-06T20:39:46.6098059Z Digest: sha256:66181a134b3ebc35388dabcc46dc11e767152fd72514dad76711590f50b650f7 2024-08-06T20:39:46.6132563Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:39:46.6164460Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:39:46.6214065Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-06T20:39:46.6214927Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-06T20:39:46.6224478Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:39:46.6224957Z env: 2024-08-06T20:39:46.6225233Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:39:46.6225887Z ##[endgroup] 2024-08-06T20:39:46.6485718Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-06T20:39:46.6486281Z with: 2024-08-06T20:39:46.6486565Z driver-version: 550.54.15 2024-08-06T20:39:46.6486912Z env: 2024-08-06T20:39:46.6487207Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:39:46.6487535Z ##[endgroup] 2024-08-06T20:39:46.6536740Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-06T20:39:46.6537284Z with: 2024-08-06T20:39:46.6537574Z timeout_minutes: 10 2024-08-06T20:39:46.6537886Z max_attempts: 3 2024-08-06T20:39:46.6566931Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-06T20:39:46.6595984Z retry_wait_seconds: 10 2024-08-06T20:39:46.6596360Z polling_interval_seconds: 1 2024-08-06T20:39:46.6596719Z warning_on_retry: true 2024-08-06T20:39:46.6597069Z continue_on_error: false 2024-08-06T20:39:46.6597412Z env: 2024-08-06T20:39:46.6597687Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:39:46.6598040Z DRIVER_VERSION: 550.54.15 2024-08-06T20:39:46.6598380Z ##[endgroup] 2024-08-06T20:39:46.7369002Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-06T20:39:46.7371463Z + pre_install_nvidia_driver_amzn2 2024-08-06T20:39:46.7374277Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-06T20:39:47.2483959Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-08-06T20:39:47.3093125Z No Match for argument: nvidia-driver-latest-dkms 2024-08-06T20:39:47.3475691Z No Packages marked for removal 2024-08-06T20:39:47.3634016Z + install_nvidia_driver_common 2024-08-06T20:39:47.3637425Z + echo 'Before installing NVIDIA driver' 2024-08-06T20:39:47.3638102Z + lspci 2024-08-06T20:39:47.3640545Z Before installing NVIDIA driver 2024-08-06T20:39:47.3815624Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-08-06T20:39:47.3816390Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-06T20:39:47.3817090Z 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 2024-08-06T20:39:47.3818019Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 2024-08-06T20:39:47.3818559Z 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 2024-08-06T20:39:47.3819176Z 00:03.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-06T20:39:47.3819876Z 00:1d.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-08-06T20:39:47.3820657Z 00:1e.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-08-06T20:39:47.3821349Z 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 2024-08-06T20:39:47.3822015Z + lsmod 2024-08-06T20:39:47.3839252Z Module Size Used by 2024-08-06T20:39:47.3840082Z ib_core 266240 0 2024-08-06T20:39:47.3840542Z veth 16384 0 2024-08-06T20:39:47.3840929Z nvidia_modeset 1339392 0 2024-08-06T20:39:47.3841273Z nvidia_uvm 4599808 0 2024-08-06T20:39:47.3841697Z nvidia 53989376 11 nvidia_uvm,nvidia_modeset 2024-08-06T20:39:47.3842189Z drm 421888 1 nvidia 2024-08-06T20:39:47.3842584Z i2c_core 77824 2 nvidia,drm 2024-08-06T20:39:47.3843026Z backlight 16384 1 nvidia_modeset 2024-08-06T20:39:47.3843437Z xt_conntrack 16384 1 2024-08-06T20:39:47.3843812Z ipt_MASQUERADE 16384 1 2024-08-06T20:39:47.3844231Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-08-06T20:39:47.3844667Z nf_conntrack_netlink 49152 0 2024-08-06T20:39:47.3845102Z nfnetlink 16384 2 nf_conntrack_netlink 2024-08-06T20:39:47.3845554Z xfrm_user 45056 1 2024-08-06T20:39:47.3845915Z xfrm_algo 16384 1 xfrm_user 2024-08-06T20:39:47.3846319Z iptable_nat 16384 1 2024-08-06T20:39:47.3846694Z nf_conntrack_ipv4 16384 3 2024-08-06T20:39:47.3847087Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-08-06T20:39:47.3847548Z nf_nat_ipv4 16384 1 iptable_nat 2024-08-06T20:39:47.3848035Z nf_nat 32768 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 2024-08-06T20:39:47.3848865Z nf_conntrack 155648 7 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink 2024-08-06T20:39:47.3849643Z xt_addrtype 16384 2 2024-08-06T20:39:47.3850017Z iptable_filter 16384 1 2024-08-06T20:39:47.3850375Z br_netfilter 24576 0 2024-08-06T20:39:47.3850765Z bridge 172032 1 br_netfilter 2024-08-06T20:39:47.3851186Z stp 16384 1 bridge 2024-08-06T20:39:47.3851578Z llc 16384 2 bridge,stp 2024-08-06T20:39:47.3851981Z overlay 86016 0 2024-08-06T20:39:47.3852341Z sunrpc 393216 1 2024-08-06T20:39:47.3852681Z dm_mirror 28672 0 2024-08-06T20:39:47.3853058Z dm_region_hash 20480 1 dm_mirror 2024-08-06T20:39:47.3853503Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-08-06T20:39:47.3853955Z dm_mod 143360 2 dm_log,dm_mirror 2024-08-06T20:39:47.3854370Z dax 69632 1 dm_mod 2024-08-06T20:39:47.3854753Z sb_edac 24576 0 2024-08-06T20:39:47.3855097Z crc32_pclmul 16384 0 2024-08-06T20:39:47.3855467Z ghash_clmulni_intel 16384 0 2024-08-06T20:39:47.3855838Z pcbc 16384 0 2024-08-06T20:39:47.3856189Z aesni_intel 188416 0 2024-08-06T20:39:47.3856551Z aes_x86_64 20480 1 aesni_intel 2024-08-06T20:39:47.3856997Z crypto_simd 16384 1 aesni_intel 2024-08-06T20:39:47.3857379Z glue_helper 16384 1 aesni_intel 2024-08-06T20:39:47.3857794Z mousedev 24576 0 2024-08-06T20:39:47.3858264Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-08-06T20:39:47.3858757Z pcc_cpufreq 16384 0 2024-08-06T20:39:47.3859124Z evdev 20480 3 2024-08-06T20:39:47.3859517Z psmouse 32768 0 2024-08-06T20:39:47.3860141Z button 16384 0 2024-08-06T20:39:47.3860478Z ena 139264 0 2024-08-06T20:39:47.3860839Z ptp 20480 1 ena 2024-08-06T20:39:47.3861234Z pps_core 20480 1 ptp 2024-08-06T20:39:47.3861600Z ata_piix 36864 0 2024-08-06T20:39:47.3861976Z libata 266240 1 ata_piix 2024-08-06T20:39:47.3862381Z xen_blkfront 49152 2 2024-08-06T20:39:47.3862729Z crc32c_intel 24576 0 2024-08-06T20:39:47.3863102Z scsi_mod 245760 1 libata 2024-08-06T20:39:47.3863475Z autofs4 49152 2 2024-08-06T20:39:47.3863833Z + modinfo nvidia 2024-08-06T20:39:47.3864484Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-08-06T20:39:47.3865107Z alias: char-major-195-* 2024-08-06T20:39:47.3865754Z version: 550.54.15 2024-08-06T20:39:47.3866143Z supported: external 2024-08-06T20:39:47.3866489Z license: NVIDIA 2024-08-06T20:39:47.3866860Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-06T20:39:47.3867326Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-06T20:39:47.3867749Z srcversion: 833721318DA517F0C2FEC97 2024-08-06T20:39:47.3868187Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-06T20:39:47.3868652Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-06T20:39:47.3869097Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-06T20:39:47.3869531Z depends: i2c-core,drm 2024-08-06T20:39:47.3869895Z retpoline: Y 2024-08-06T20:39:47.3870195Z name: nvidia 2024-08-06T20:39:47.3870657Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-08-06T20:39:47.3871395Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-06T20:39:47.3872004Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-06T20:39:47.3872570Z parm: NVreg_ResmanDebugLevel:int 2024-08-06T20:39:47.3873002Z parm: NVreg_RmLogonRC:int 2024-08-06T20:39:47.3873531Z parm: NVreg_ModifyDeviceFiles:int 2024-08-06T20:39:47.3873974Z parm: NVreg_DeviceFileUID:int 2024-08-06T20:39:47.3874397Z parm: NVreg_DeviceFileGID:int 2024-08-06T20:39:47.3874805Z parm: NVreg_DeviceFileMode:int 2024-08-06T20:39:47.3875299Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-06T20:39:47.3875825Z parm: NVreg_UsePageAttributeTable:int 2024-08-06T20:39:47.3876261Z parm: NVreg_EnablePCIeGen3:int 2024-08-06T20:39:47.3876678Z parm: NVreg_EnableMSI:int 2024-08-06T20:39:47.3877078Z parm: NVreg_TCEBypassMode:int 2024-08-06T20:39:47.3877505Z parm: NVreg_EnableStreamMemOPs:int 2024-08-06T20:39:47.3877999Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-06T20:39:47.3878528Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-06T20:39:47.3879022Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-06T20:39:47.3879570Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-06T20:39:47.3880125Z parm: NVreg_DynamicPowerManagement:int 2024-08-06T20:39:47.3880667Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-06T20:39:47.3881219Z parm: NVreg_EnableGpuFirmware:int 2024-08-06T20:39:47.3881683Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-06T20:39:47.3882162Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-06T20:39:47.3882665Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-06T20:39:47.3883131Z parm: NVreg_MemoryPoolSize:int 2024-08-06T20:39:47.3883562Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-06T20:39:47.3884019Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-06T20:39:47.3884465Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-06T20:39:47.3884873Z parm: NVreg_NvLinkDisable:int 2024-08-06T20:39:47.3885351Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-06T20:39:47.3885842Z parm: NVreg_RegisterPCIDriver:int 2024-08-06T20:39:47.3886448Z parm: NVreg_EnableResizableBar:int 2024-08-06T20:39:47.3886925Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-06T20:39:47.3887397Z parm: NVreg_EnableNonblockingOpen:int 2024-08-06T20:39:47.3887840Z parm: NVreg_RegistryDwords:charp 2024-08-06T20:39:47.3888309Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-06T20:39:47.3888762Z parm: NVreg_RmMsg:charp 2024-08-06T20:39:47.3889143Z parm: NVreg_GpuBlacklist:charp 2024-08-06T20:39:47.3889598Z parm: NVreg_TemporaryFilePath:charp 2024-08-06T20:39:47.3890167Z parm: NVreg_ExcludedGpus:charp 2024-08-06T20:39:47.3890611Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-06T20:39:47.3891074Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-06T20:39:47.3891531Z parm: NVreg_ImexChannelCount:int 2024-08-06T20:39:47.3891952Z parm: rm_firmware_active:charp 2024-08-06T20:39:47.3892358Z + HAS_NVIDIA_DRIVER=0 2024-08-06T20:39:47.3892715Z ++ command -v nvidia-smi 2024-08-06T20:39:47.3893063Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-06T20:39:47.3893438Z + set +e 2024-08-06T20:39:47.3893876Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-06T20:39:47.4126382Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-06T20:39:47.4126802Z + NVIDIA_SMI_STATUS=0 2024-08-06T20:39:47.4127134Z + '[' 0 -ne 0 ']' 2024-08-06T20:39:47.4127442Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-06T20:39:47.4127790Z + HAS_NVIDIA_DRIVER=1 2024-08-06T20:39:47.4128387Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-06T20:39:47.4128995Z + set -e 2024-08-06T20:39:47.4129267Z + '[' 1 -eq 0 ']' 2024-08-06T20:39:47.4129782Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-06T20:39:47.4130404Z + post_install_nvidia_driver_common 2024-08-06T20:39:47.4131077Z + sudo modprobe nvidia 2024-08-06T20:39:47.4249223Z + echo 'After installing NVIDIA driver' 2024-08-06T20:39:47.4249649Z + lspci 2024-08-06T20:39:47.4250735Z After installing NVIDIA driver 2024-08-06T20:39:47.4434475Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-08-06T20:39:47.4435203Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-06T20:39:47.4435853Z 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 2024-08-06T20:39:47.4436503Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 2024-08-06T20:39:47.4437067Z 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 2024-08-06T20:39:47.4437679Z 00:03.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-06T20:39:47.4438387Z 00:1d.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-08-06T20:39:47.4439128Z 00:1e.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-08-06T20:39:47.4439843Z 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 2024-08-06T20:39:47.4440369Z + lsmod 2024-08-06T20:39:47.4457692Z Module Size Used by 2024-08-06T20:39:47.4458074Z ib_core 266240 0 2024-08-06T20:39:47.4458427Z veth 16384 0 2024-08-06T20:39:47.4458818Z nvidia_modeset 1339392 0 2024-08-06T20:39:47.4459185Z nvidia_uvm 4599808 0 2024-08-06T20:39:47.4459588Z nvidia 53989376 11 nvidia_uvm,nvidia_modeset 2024-08-06T20:39:47.4460046Z drm 421888 1 nvidia 2024-08-06T20:39:47.4460451Z i2c_core 77824 2 nvidia,drm 2024-08-06T20:39:47.4460888Z backlight 16384 1 nvidia_modeset 2024-08-06T20:39:47.4461321Z xt_conntrack 16384 1 2024-08-06T20:39:47.4461689Z ipt_MASQUERADE 16384 1 2024-08-06T20:39:47.4462087Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-08-06T20:39:47.4462532Z nf_conntrack_netlink 49152 0 2024-08-06T20:39:47.4462957Z nfnetlink 16384 2 nf_conntrack_netlink 2024-08-06T20:39:47.4463622Z xfrm_user 45056 1 2024-08-06T20:39:47.4464002Z xfrm_algo 16384 1 xfrm_user 2024-08-06T20:39:47.4464403Z iptable_nat 16384 1 2024-08-06T20:39:47.4464757Z nf_conntrack_ipv4 16384 3 2024-08-06T20:39:47.4465168Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-08-06T20:39:47.4465891Z nf_nat_ipv4 16384 1 iptable_nat 2024-08-06T20:39:47.4466358Z nf_nat 32768 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 2024-08-06T20:39:47.4467360Z nf_conntrack 155648 7 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink 2024-08-06T20:39:47.4468201Z xt_addrtype 16384 2 2024-08-06T20:39:47.4468534Z iptable_filter 16384 1 2024-08-06T20:39:47.4468905Z br_netfilter 24576 0 2024-08-06T20:39:47.4469287Z bridge 172032 1 br_netfilter 2024-08-06T20:39:47.4469682Z stp 16384 1 bridge 2024-08-06T20:39:47.4470094Z llc 16384 2 bridge,stp 2024-08-06T20:39:47.4470489Z overlay 86016 0 2024-08-06T20:39:47.4470829Z sunrpc 393216 1 2024-08-06T20:39:47.4471186Z dm_mirror 28672 0 2024-08-06T20:39:47.4471559Z dm_region_hash 20480 1 dm_mirror 2024-08-06T20:39:47.4471983Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-08-06T20:39:47.4472438Z dm_mod 143360 2 dm_log,dm_mirror 2024-08-06T20:39:47.4472855Z dax 69632 1 dm_mod 2024-08-06T20:39:47.4473219Z sb_edac 24576 0 2024-08-06T20:39:47.4473580Z crc32_pclmul 16384 0 2024-08-06T20:39:47.4473956Z ghash_clmulni_intel 16384 0 2024-08-06T20:39:47.4474316Z pcbc 16384 0 2024-08-06T20:39:47.4474677Z aesni_intel 188416 0 2024-08-06T20:39:47.4475049Z aes_x86_64 20480 1 aesni_intel 2024-08-06T20:39:47.4475456Z crypto_simd 16384 1 aesni_intel 2024-08-06T20:39:47.4475879Z glue_helper 16384 1 aesni_intel 2024-08-06T20:39:47.4476300Z mousedev 24576 0 2024-08-06T20:39:47.4476747Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-08-06T20:39:47.4477257Z pcc_cpufreq 16384 0 2024-08-06T20:39:47.4477624Z evdev 20480 3 2024-08-06T20:39:47.4477969Z psmouse 32768 0 2024-08-06T20:39:47.4478327Z button 16384 0 2024-08-06T20:39:47.4478680Z ena 139264 0 2024-08-06T20:39:47.4479019Z ptp 20480 1 ena 2024-08-06T20:39:47.4479403Z pps_core 20480 1 ptp 2024-08-06T20:39:47.4479786Z ata_piix 36864 0 2024-08-06T20:39:47.4480146Z libata 266240 1 ata_piix 2024-08-06T20:39:47.4480548Z xen_blkfront 49152 2 2024-08-06T20:39:47.4480916Z crc32c_intel 24576 0 2024-08-06T20:39:47.4481265Z scsi_mod 245760 1 libata 2024-08-06T20:39:47.4481654Z autofs4 49152 2 2024-08-06T20:39:47.4482020Z + modinfo nvidia 2024-08-06T20:39:47.4482710Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-08-06T20:39:47.4483314Z alias: char-major-195-* 2024-08-06T20:39:47.4483691Z version: 550.54.15 2024-08-06T20:39:47.4484135Z supported: external 2024-08-06T20:39:47.4484482Z license: NVIDIA 2024-08-06T20:39:47.4484851Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-06T20:39:47.4485301Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-06T20:39:47.4485742Z srcversion: 833721318DA517F0C2FEC97 2024-08-06T20:39:47.4486185Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-06T20:39:47.4486630Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-06T20:39:47.4487092Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-06T20:39:47.4487522Z depends: i2c-core,drm 2024-08-06T20:39:47.4487866Z retpoline: Y 2024-08-06T20:39:47.4488181Z name: nvidia 2024-08-06T20:39:47.4488825Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-08-06T20:39:47.4489390Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-06T20:39:47.4489984Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-06T20:39:47.4490545Z parm: NVreg_ResmanDebugLevel:int 2024-08-06T20:39:47.4490959Z parm: NVreg_RmLogonRC:int 2024-08-06T20:39:47.4491376Z parm: NVreg_ModifyDeviceFiles:int 2024-08-06T20:39:47.4491810Z parm: NVreg_DeviceFileUID:int 2024-08-06T20:39:47.4492215Z parm: NVreg_DeviceFileGID:int 2024-08-06T20:39:47.4492729Z parm: NVreg_DeviceFileMode:int 2024-08-06T20:39:47.4493240Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-06T20:39:47.4493745Z parm: NVreg_UsePageAttributeTable:int 2024-08-06T20:39:47.4494210Z parm: NVreg_EnablePCIeGen3:int 2024-08-06T20:39:47.4494626Z parm: NVreg_EnableMSI:int 2024-08-06T20:39:47.4495014Z parm: NVreg_TCEBypassMode:int 2024-08-06T20:39:47.4495466Z parm: NVreg_EnableStreamMemOPs:int 2024-08-06T20:39:47.4495960Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-06T20:39:47.4496477Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-06T20:39:47.4496993Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-06T20:39:47.4497548Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-06T20:39:47.4498081Z parm: NVreg_DynamicPowerManagement:int 2024-08-06T20:39:47.4498644Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-06T20:39:47.4499204Z parm: NVreg_EnableGpuFirmware:int 2024-08-06T20:39:47.4499660Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-06T20:39:47.4500166Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-06T20:39:47.4500672Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-06T20:39:47.4501135Z parm: NVreg_MemoryPoolSize:int 2024-08-06T20:39:47.4501592Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-06T20:39:47.4502052Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-06T20:39:47.4502481Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-06T20:39:47.4502915Z parm: NVreg_NvLinkDisable:int 2024-08-06T20:39:47.4503387Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-06T20:39:47.4503861Z parm: NVreg_RegisterPCIDriver:int 2024-08-06T20:39:47.4504316Z parm: NVreg_EnableResizableBar:int 2024-08-06T20:39:47.4504779Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-06T20:39:47.4505234Z parm: NVreg_EnableNonblockingOpen:int 2024-08-06T20:39:47.4505931Z parm: NVreg_RegistryDwords:charp 2024-08-06T20:39:47.4506409Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-06T20:39:47.4506847Z parm: NVreg_RmMsg:charp 2024-08-06T20:39:47.4507250Z parm: NVreg_GpuBlacklist:charp 2024-08-06T20:39:47.4507698Z parm: NVreg_TemporaryFilePath:charp 2024-08-06T20:39:47.4508141Z parm: NVreg_ExcludedGpus:charp 2024-08-06T20:39:47.4508577Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-06T20:39:47.4509033Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-06T20:39:47.4509473Z parm: NVreg_ImexChannelCount:int 2024-08-06T20:39:47.4509905Z parm: rm_firmware_active:charp 2024-08-06T20:39:47.4510297Z + set +e 2024-08-06T20:39:47.4510564Z + nvidia-smi 2024-08-06T20:39:47.4724936Z Tue Aug 6 20:39:47 2024 2024-08-06T20:39:47.4725406Z +-----------------------------------------------------------------------------------------+ 2024-08-06T20:39:47.4726065Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-06T20:39:47.4726721Z |-----------------------------------------+------------------------+----------------------+ 2024-08-06T20:39:47.4727381Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-06T20:39:47.4728066Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-06T20:39:47.4728845Z | | | MIG M. | 2024-08-06T20:39:47.4729319Z |=========================================+========================+======================| 2024-08-06T20:39:47.4814425Z | 0 Tesla M60 On | 00000000:00:1D.0 Off | 0 | 2024-08-06T20:39:47.4815009Z | N/A 32C P8 14W / 150W | 0MiB / 7680MiB | 0% Default | 2024-08-06T20:39:47.4815699Z | | | N/A | 2024-08-06T20:39:47.4816271Z +-----------------------------------------+------------------------+----------------------+ 2024-08-06T20:39:47.4882465Z | 1 Tesla M60 On | 00000000:00:1E.0 Off | 0 | 2024-08-06T20:39:47.4883058Z | N/A 33C P8 14W / 150W | 0MiB / 7680MiB | 0% Default | 2024-08-06T20:39:47.4883598Z | | | N/A | 2024-08-06T20:39:47.4884138Z +-----------------------------------------+------------------------+----------------------+ 2024-08-06T20:39:47.4884656Z 2024-08-06T20:39:47.4885186Z +-----------------------------------------------------------------------------------------+ 2024-08-06T20:39:47.4885770Z | Processes: | 2024-08-06T20:39:47.4886366Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-06T20:39:47.4886943Z | ID ID Usage | 2024-08-06T20:39:47.4887427Z |=========================================================================================| 2024-08-06T20:39:47.4888462Z | No running processes found | 2024-08-06T20:39:47.4889100Z +-----------------------------------------------------------------------------------------+ 2024-08-06T20:39:47.6025397Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-06T20:39:47.6213325Z Tesla M60 2024-08-06T20:39:47.6277886Z + NVIDIA_SMI_STATUS=0 2024-08-06T20:39:47.6278214Z + '[' 0 -eq 0 ']' 2024-08-06T20:39:47.6278549Z + echo 'INFO: Ignoring allowed status 0' 2024-08-06T20:39:47.6278938Z + set -e 2024-08-06T20:39:47.6279659Z INFO: Ignoring allowed status 0 2024-08-06T20:39:47.6284570Z == Installing nvidia container toolkit for amzn2 == 2024-08-06T20:39:47.6288057Z + sudo yum install -y yum-utils 2024-08-06T20:39:48.1187547Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-08-06T20:39:49.8402872Z Package yum-utils-1.1.31-46.amzn2.0.1.noarch already installed and latest version 2024-08-06T20:39:49.8403512Z Nothing to do 2024-08-06T20:39:49.9873125Z + [[ amzn2 == \a\m\z\n\2\0\2\3 ]] 2024-08-06T20:39:49.9873749Z + YUM_REPO_URL=https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-08-06T20:39:49.9874629Z + sudo yum-config-manager --add-repo https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-08-06T20:39:50.4976743Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-08-06T20:39:50.5328959Z adding repo from: https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-08-06T20:39:50.5329913Z grabbing file https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo to /etc/yum.repos.d/nvidia-docker.repo 2024-08-06T20:39:50.5330759Z repo saved to /etc/yum.repos.d/nvidia-docker.repo 2024-08-06T20:39:50.5475926Z + sudo yum install -y nvidia-docker2 2024-08-06T20:39:51.0418312Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-08-06T20:39:52.7150785Z Package nvidia-docker2-2.14.0-1.noarch already installed and latest version 2024-08-06T20:39:52.7151355Z Nothing to do 2024-08-06T20:39:52.8564161Z + sudo systemctl restart docker 2024-08-06T20:40:11.9191244Z nvidia-persistenced failed to initialize. Check syslog for more details. 2024-08-06T20:40:11.9639986Z Tue Aug 6 20:40:11 2024 2024-08-06T20:40:11.9641008Z +-----------------------------------------------------------------------------------------+ 2024-08-06T20:40:11.9642276Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-06T20:40:11.9643582Z |-----------------------------------------+------------------------+----------------------+ 2024-08-06T20:40:11.9645200Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-06T20:40:11.9646655Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-06T20:40:11.9647794Z | | | MIG M. | 2024-08-06T20:40:11.9648687Z |=========================================+========================+======================| 2024-08-06T20:40:11.9730244Z | 0 Tesla M60 On | 00000000:00:1D.0 Off | 0 | 2024-08-06T20:40:11.9731375Z | N/A 32C P8 13W / 150W | 0MiB / 7680MiB | 0% Default | 2024-08-06T20:40:11.9732291Z | | | N/A | 2024-08-06T20:40:11.9733285Z +-----------------------------------------+------------------------+----------------------+ 2024-08-06T20:40:11.9797655Z | 1 Tesla M60 On | 00000000:00:1E.0 Off | 0 | 2024-08-06T20:40:11.9798875Z | N/A 33C P8 14W / 150W | 0MiB / 7680MiB | 0% Default | 2024-08-06T20:40:11.9799803Z | | | N/A | 2024-08-06T20:40:11.9800829Z +-----------------------------------------+------------------------+----------------------+ 2024-08-06T20:40:11.9801889Z 2024-08-06T20:40:11.9802841Z +-----------------------------------------------------------------------------------------+ 2024-08-06T20:40:11.9803914Z | Processes: | 2024-08-06T20:40:11.9805111Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-06T20:40:11.9806158Z | ID ID Usage | 2024-08-06T20:40:11.9806910Z |=========================================================================================| 2024-08-06T20:40:11.9807823Z | No running processes found | 2024-08-06T20:40:11.9808864Z +-----------------------------------------------------------------------------------------+ 2024-08-06T20:40:12.7637419Z Command completed after 1 attempt(s). 2024-08-06T20:40:12.7720181Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-06T20:40:12.7721065Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-06T20:40:12.7721795Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-06T20:40:12.7722489Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:40:12.7731416Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:12.7732000Z env: 2024-08-06T20:40:12.7732399Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:12.7732903Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:12.7733466Z ##[endgroup] 2024-08-06T20:40:13.0770794Z Defaulting to user installation because normal site-packages is not writeable 2024-08-06T20:40:13.0979593Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.7/site-packages (5.9.1) 2024-08-06T20:40:13.1105322Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.7/site-packages (11.525.84) 2024-08-06T20:40:13.2244391Z Prepare all required actions 2024-08-06T20:40:13.2245090Z Getting action download info 2024-08-06T20:40:13.3444811Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-06T20:40:13.5081544Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-06T20:40:13.6357648Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-06T20:40:13.6358111Z with: 2024-08-06T20:40:13.6358431Z name: linux-focal-cuda11.8-py3.10-gcc9 2024-08-06T20:40:13.6358836Z s3-bucket: gha-artifacts 2024-08-06T20:40:13.6359182Z env: 2024-08-06T20:40:13.6359476Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:13.6359896Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:13.6360350Z ##[endgroup] 2024-08-06T20:40:13.6407993Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-06T20:40:13.6408454Z with: 2024-08-06T20:40:13.6408828Z name: linux-focal-cuda11.8-py3.10-gcc9 2024-08-06T20:40:13.6409235Z s3-bucket: gha-artifacts 2024-08-06T20:40:13.6409590Z region: us-east-1 2024-08-06T20:40:13.6409901Z env: 2024-08-06T20:40:13.6410172Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:13.6410602Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:13.6411053Z ##[endgroup] 2024-08-06T20:40:14.2482749Z (node:10147) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-06T20:40:14.2483354Z 2024-08-06T20:40:14.2483604Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-06T20:40:14.2484210Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-06T20:40:14.2484972Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-06T20:40:14.3379114Z Found 1 objects with prefix pytorch/pytorch/10273124344/linux-focal-cuda11.8-py3.10-gcc9/ 2024-08-06T20:40:14.3380010Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-06T20:40:36.8585301Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-06T20:40:36.8597186Z Artifact download has finished successfully 2024-08-06T20:40:36.8809118Z ##[group]Run unzip -o artifacts.zip 2024-08-06T20:40:36.8809546Z unzip -o artifacts.zip 2024-08-06T20:40:36.8817642Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:36.8818105Z env: 2024-08-06T20:40:36.8818404Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:36.8818835Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:36.8819276Z ##[endgroup] 2024-08-06T20:40:36.8856188Z Archive: artifacts.zip 2024-08-06T20:40:36.8858250Z creating: dist/ 2024-08-06T20:40:39.3444644Z inflating: dist/torch-2.5.0a0+gitb9d86fa-cp310-cp310-linux_x86_64.whl 2024-08-06T20:40:39.3445288Z creating: build/custom_test_artifacts/ 2024-08-06T20:40:39.3445660Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-06T20:40:39.3446325Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-06T20:40:39.3447049Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-06T20:40:39.3454903Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-06T20:40:39.3455686Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-06T20:40:39.3456487Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-06T20:40:39.3457228Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-06T20:40:39.3458044Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-06T20:40:39.3459536Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-06T20:40:39.3461955Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-06T20:40:39.3462910Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-06T20:40:39.3463750Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-06T20:40:39.3465183Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-06T20:40:39.3467856Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-06T20:40:39.3469589Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-06T20:40:39.3470566Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-06T20:40:39.3472567Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-06T20:40:39.3473624Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-06T20:40:39.3474503Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-06T20:40:39.3475243Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-06T20:40:39.3532703Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-06T20:40:39.3533919Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-06T20:40:39.3535134Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-06T20:40:39.3536359Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-06T20:40:39.3537653Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-06T20:40:39.3538872Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-06T20:40:39.3539931Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-06T20:40:39.3541132Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-06T20:40:39.3542324Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-06T20:40:39.3589145Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-06T20:40:39.3639025Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-06T20:40:39.3640264Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-06T20:40:39.3641395Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-06T20:40:39.3642471Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-06T20:40:39.3643518Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-06T20:40:39.3644584Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-06T20:40:39.3645531Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-06T20:40:39.3646674Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-06T20:40:39.3723855Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-06T20:40:39.3800378Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-06T20:40:39.3801418Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-06T20:40:39.3802272Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-06T20:40:39.3802999Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-06T20:40:39.3803793Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-06T20:40:39.3804606Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-06T20:40:39.3805474Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-06T20:40:39.3806469Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-06T20:40:39.3807460Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-06T20:40:39.3808317Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-06T20:40:39.3809252Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-06T20:40:39.3810135Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-06T20:40:39.3811033Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-06T20:40:39.3811954Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-06T20:40:39.3812880Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-06T20:40:39.3835732Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-06T20:40:39.4014848Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-06T20:40:39.4015700Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-06T20:40:39.4016671Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-06T20:40:39.4017729Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-06T20:40:39.4018711Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-06T20:40:39.4019666Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-06T20:40:39.4020591Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-06T20:40:39.4021629Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-06T20:40:39.4022483Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-06T20:40:39.4023466Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-06T20:40:39.4024436Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-06T20:40:39.4048042Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-06T20:40:39.4152584Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-06T20:40:39.4153629Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-06T20:40:39.4154466Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-06T20:40:39.4155620Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-06T20:40:39.4156535Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-06T20:40:39.4157200Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-06T20:40:39.4157987Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-06T20:40:39.4160395Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-06T20:40:39.4161181Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-06T20:40:39.4162065Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-06T20:40:39.4311900Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-06T20:40:39.4390614Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-06T20:40:39.4391267Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-06T20:40:39.4391766Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-06T20:40:39.4392457Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-06T20:40:39.4400657Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-06T20:40:39.4401472Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-06T20:40:39.4402257Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-06T20:40:39.4402967Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-06T20:40:39.4403792Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-06T20:40:39.4404874Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-06T20:40:39.4407084Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-06T20:40:39.4407839Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-06T20:40:39.4408677Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-06T20:40:39.4410411Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-06T20:40:39.4412269Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-06T20:40:39.4414491Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-06T20:40:39.4415772Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-06T20:40:39.4418080Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-06T20:40:39.4419060Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-06T20:40:39.4419882Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-06T20:40:39.4420689Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-06T20:40:39.4478144Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-06T20:40:39.4479382Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-06T20:40:39.4480550Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-06T20:40:39.4481840Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-06T20:40:39.4483083Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-06T20:40:39.4484593Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-06T20:40:39.4485673Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-06T20:40:39.4486873Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-06T20:40:39.4488031Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-06T20:40:39.4534979Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-06T20:40:39.4584896Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-06T20:40:39.4586416Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-06T20:40:39.4587607Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-06T20:40:39.4588677Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-06T20:40:39.4589715Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-06T20:40:39.4590771Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-06T20:40:39.4591783Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-06T20:40:39.4592697Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-06T20:40:39.4668104Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-06T20:40:39.4743698Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-06T20:40:39.4744716Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-06T20:40:39.4745553Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-06T20:40:39.4746523Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-06T20:40:39.4747296Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-06T20:40:39.4748091Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-06T20:40:39.4749016Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-06T20:40:39.4750001Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-06T20:40:39.4751032Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-06T20:40:39.4751961Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-06T20:40:39.4752825Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-06T20:40:39.4753881Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-06T20:40:39.4754700Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-06T20:40:39.4755656Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-06T20:40:39.4756605Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-06T20:40:39.4779444Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-06T20:40:39.4860182Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-06T20:40:39.4861233Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-06T20:40:39.4862094Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-06T20:40:39.4862891Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-06T20:40:39.4863759Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-06T20:40:39.4864542Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-06T20:40:39.4865182Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-06T20:40:39.4868153Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-06T20:40:39.4869114Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-06T20:40:39.4869956Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-06T20:40:39.4931827Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-06T20:40:39.4932351Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-06T20:40:39.4933075Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-06T20:40:39.4933831Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-06T20:40:39.4942006Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-06T20:40:39.4942916Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-06T20:40:39.4943628Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-06T20:40:39.4944540Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-06T20:40:39.4945876Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-06T20:40:39.4946807Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-06T20:40:39.4948173Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-06T20:40:39.4949118Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-06T20:40:39.4950031Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-06T20:40:39.4951597Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-06T20:40:39.4953430Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-06T20:40:39.4955699Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-06T20:40:39.4956773Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-06T20:40:39.4958286Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-06T20:40:39.4959396Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-06T20:40:39.4960356Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-06T20:40:39.4961262Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-06T20:40:39.5018453Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-06T20:40:39.5019753Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-06T20:40:39.5021385Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-06T20:40:39.5022751Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-06T20:40:39.5024088Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-06T20:40:39.5025352Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-06T20:40:39.5026910Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-06T20:40:39.5028193Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-06T20:40:39.5029462Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-06T20:40:39.5075006Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-06T20:40:39.5124845Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-06T20:40:39.5126292Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-06T20:40:39.5127511Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-06T20:40:39.5128606Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-06T20:40:39.5129768Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-06T20:40:39.5130874Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-06T20:40:39.5132022Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-06T20:40:39.5133010Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-06T20:40:39.5208138Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-06T20:40:39.5284123Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-06T20:40:39.5285234Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-06T20:40:39.5286169Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-06T20:40:39.5286961Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-06T20:40:39.5287815Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-06T20:40:39.5288698Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-06T20:40:39.5289681Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-06T20:40:39.5290718Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-06T20:40:39.5291859Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-06T20:40:39.5292860Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-06T20:40:39.5293745Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-06T20:40:39.5294992Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-06T20:40:39.5296195Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-06T20:40:39.5297229Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-06T20:40:39.5298360Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-06T20:40:39.5299342Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-06T20:40:39.5456256Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-06T20:40:39.5457284Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-06T20:40:39.5458297Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-06T20:40:39.5459369Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-06T20:40:39.5460488Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-06T20:40:39.5461618Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-06T20:40:39.5462686Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-06T20:40:39.5463674Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-06T20:40:39.5464760Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-06T20:40:39.5466128Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-06T20:40:39.5467224Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-06T20:40:39.5489377Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-06T20:40:39.5558997Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-06T20:40:39.5560124Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-06T20:40:39.5560999Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-06T20:40:39.5562028Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-06T20:40:39.5562817Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-06T20:40:39.5563686Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-06T20:40:39.5564442Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-06T20:40:39.5566934Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-06T20:40:39.5567866Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-06T20:40:39.5568798Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-06T20:40:39.5699488Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-06T20:40:39.5754031Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-06T20:40:39.5754615Z creating: build/lib/ 2024-08-06T20:40:39.5766905Z inflating: build/lib/libpthreadpool.a 2024-08-06T20:40:39.5777092Z inflating: build/lib/libcpuinfo.a 2024-08-06T20:40:39.5787023Z inflating: build/lib/libcpuinfo_internals.a 2024-08-06T20:40:39.5788035Z inflating: build/lib/libclog.a 2024-08-06T20:40:39.5898852Z inflating: build/lib/libprotobuf-lite.a 2024-08-06T20:40:39.5901452Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-06T20:40:39.6468177Z inflating: build/lib/libprotobuf.a 2024-08-06T20:40:39.6549119Z inflating: build/lib/libgtest.a 2024-08-06T20:40:39.6642247Z inflating: build/lib/libbenchmark.a 2024-08-06T20:40:39.6651622Z inflating: build/lib/libittnotify.a 2024-08-06T20:40:39.6686180Z inflating: build/lib/libtensorpipe_uv.a 2024-08-06T20:40:39.6708852Z inflating: build/lib/libnnpack.a 2024-08-06T20:40:39.6787905Z inflating: build/lib/libasmjit.a 2024-08-06T20:40:39.6945981Z inflating: build/lib/libgloo.a 2024-08-06T20:40:39.6972080Z inflating: build/lib/libfmt.a 2024-08-06T20:40:39.7089964Z inflating: build/lib/libc10.so 2024-08-06T20:40:39.7092061Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-06T20:40:39.7092716Z inflating: build/lib/libfoxi_loader.a 2024-08-06T20:40:39.7094877Z inflating: build/lib/libtorch_global_deps.so 2024-08-06T20:40:39.7119043Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-06T20:40:39.7141398Z inflating: build/lib/libgmock.a 2024-08-06T20:40:39.7142112Z inflating: build/lib/libgtest_main.a 2024-08-06T20:40:39.7143168Z inflating: build/lib/libbenchmark_main.a 2024-08-06T20:40:39.7771149Z inflating: build/lib/libprotoc.a 2024-08-06T20:40:41.0411562Z inflating: build/lib/libdnnl.a 2024-08-06T20:40:41.1124331Z inflating: build/lib/libtensorpipe.a 2024-08-06T20:40:41.1195893Z inflating: build/lib/libc10_cuda.so 2024-08-06T20:40:41.1196741Z inflating: build/lib/libgmock_main.a 2024-08-06T20:40:41.2758302Z inflating: build/lib/libfbgemm.a 2024-08-06T20:40:41.3387104Z inflating: build/lib/libkineto.a 2024-08-06T20:40:41.3706707Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-06T20:40:41.3936005Z inflating: build/lib/libXNNPACK.a 2024-08-06T20:40:41.3988991Z inflating: build/lib/libonnx_proto.a 2024-08-06T20:40:41.4473234Z inflating: build/lib/libgloo_cuda.a 2024-08-06T20:40:41.5334229Z inflating: build/lib/libonnx.a 2024-08-06T20:40:44.5497756Z inflating: build/lib/libtorch_cpu.so 2024-08-06T20:40:44.5502778Z inflating: build/lib/libshm.so 2024-08-06T20:40:44.5509089Z inflating: build/lib/libunbox_lib.a 2024-08-06T20:40:47.0235876Z inflating: build/lib/libtorch_cuda.so 2024-08-06T20:40:47.0237092Z inflating: build/lib/libtorch.so 2024-08-06T20:40:47.0240286Z inflating: build/lib/libc10d_cuda_test.so 2024-08-06T20:40:48.1626612Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-06T20:40:48.1652247Z inflating: build/lib/libjitbackend_test.so 2024-08-06T20:40:48.1743270Z inflating: build/lib/libtorchbind_test.so 2024-08-06T20:40:48.1775960Z inflating: build/lib/libaoti_custom_ops.so 2024-08-06T20:40:48.1807619Z inflating: build/lib/libbackend_with_compiler.so 2024-08-06T20:40:48.4289052Z inflating: build/lib/libtorch_python.so 2024-08-06T20:40:48.4332380Z inflating: build/lib/libnnapi_backend.so 2024-08-06T20:40:48.4332827Z creating: build/bin/ 2024-08-06T20:40:48.4395752Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-06T20:40:48.4460250Z inflating: build/bin/c10_DeviceGuard_test 2024-08-06T20:40:48.4523490Z inflating: build/bin/c10_Device_test 2024-08-06T20:40:48.4597191Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-06T20:40:48.4663879Z inflating: build/bin/c10_Scalar_test 2024-08-06T20:40:48.4725051Z inflating: build/bin/c10_StreamGuard_test 2024-08-06T20:40:48.4788930Z inflating: build/bin/c10_SymInt_test 2024-08-06T20:40:48.4857046Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-06T20:40:48.4926727Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-06T20:40:48.4997130Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-06T20:40:48.5085777Z inflating: build/bin/c10_cow_test 2024-08-06T20:40:48.5152506Z inflating: build/bin/c10_Bitset_test 2024-08-06T20:40:48.5213782Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-06T20:40:48.5275904Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-06T20:40:48.5339155Z inflating: build/bin/c10_Half_test 2024-08-06T20:40:48.5409574Z inflating: build/bin/c10_LeftRight_test 2024-08-06T20:40:48.5478434Z inflating: build/bin/c10_Metaprogramming_test 2024-08-06T20:40:48.5540795Z inflating: build/bin/c10_Synchronized_test 2024-08-06T20:40:48.5605150Z inflating: build/bin/c10_TypeIndex_test 2024-08-06T20:40:48.5674818Z inflating: build/bin/c10_ThreadLocal_test 2024-08-06T20:40:48.5740061Z inflating: build/bin/c10_TypeList_test 2024-08-06T20:40:48.5801475Z inflating: build/bin/c10_TypeTraits_test 2024-08-06T20:40:48.5866082Z inflating: build/bin/c10_accumulate_test 2024-08-06T20:40:48.5935666Z inflating: build/bin/c10_bfloat16_test 2024-08-06T20:40:48.5998961Z inflating: build/bin/c10_bit_cast_test 2024-08-06T20:40:48.6069382Z inflating: build/bin/c10_complex_math_test 2024-08-06T20:40:48.6138822Z inflating: build/bin/c10_complex_test 2024-08-06T20:40:48.6204740Z inflating: build/bin/c10_exception_test 2024-08-06T20:40:48.6267840Z inflating: build/bin/c10_flags_test 2024-08-06T20:40:48.6330224Z inflating: build/bin/c10_generic_math_test 2024-08-06T20:40:48.6536098Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-06T20:40:48.6599618Z inflating: build/bin/c10_irange_test 2024-08-06T20:40:48.6666775Z inflating: build/bin/c10_lazy_test 2024-08-06T20:40:48.6738032Z inflating: build/bin/c10_logging_test 2024-08-06T20:40:48.6831901Z inflating: build/bin/c10_optional_test 2024-08-06T20:40:48.6909816Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-06T20:40:48.6977175Z inflating: build/bin/c10_registry_test 2024-08-06T20:40:48.7166424Z inflating: build/bin/c10_small_vector_test 2024-08-06T20:40:48.7230913Z inflating: build/bin/c10_ssize_test 2024-08-06T20:40:48.7295853Z inflating: build/bin/c10_string_util_test 2024-08-06T20:40:48.7369326Z inflating: build/bin/c10_string_view_test 2024-08-06T20:40:48.7432272Z inflating: build/bin/c10_tempfile_test 2024-08-06T20:40:48.7502523Z inflating: build/bin/c10_typeid_test 2024-08-06T20:40:48.7563258Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-06T20:40:48.8122650Z inflating: build/bin/protoc-3.13.0.0 2024-08-06T20:40:48.8680662Z inflating: build/bin/protoc 2024-08-06T20:40:48.8747013Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-06T20:40:48.8812496Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-06T20:40:48.8878754Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-06T20:40:48.8944530Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-06T20:40:48.9009719Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-06T20:40:48.9076970Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-06T20:40:48.9138595Z inflating: build/bin/c10_cuda_CUDATest 2024-08-06T20:40:48.9204844Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-06T20:40:48.9619801Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-06T20:40:49.0051128Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-06T20:40:49.0500386Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-06T20:40:49.0567164Z inflating: build/bin/HashStoreTest 2024-08-06T20:40:49.0633506Z inflating: build/bin/FileStoreTest 2024-08-06T20:40:49.0699117Z inflating: build/bin/BackoffTest 2024-08-06T20:40:49.0769485Z inflating: build/bin/TCPStoreTest 2024-08-06T20:40:49.0786593Z inflating: build/bin/ProcessGroupMPITest 2024-08-06T20:40:49.0791850Z inflating: build/bin/torch_shm_manager 2024-08-06T20:40:49.0859100Z inflating: build/bin/test_edge_op_registration 2024-08-06T20:40:49.0862871Z inflating: build/bin/example_allreduce 2024-08-06T20:40:49.0927814Z inflating: build/bin/Dimname_test 2024-08-06T20:40:49.1019856Z inflating: build/bin/Dict_test 2024-08-06T20:40:49.1100538Z inflating: build/bin/MaybeOwned_test 2024-08-06T20:40:49.1172625Z inflating: build/bin/NamedTensor_test 2024-08-06T20:40:49.1246613Z inflating: build/bin/apply_utils_test 2024-08-06T20:40:49.1320414Z inflating: build/bin/atest 2024-08-06T20:40:49.1399170Z inflating: build/bin/basic 2024-08-06T20:40:49.1462474Z inflating: build/bin/cpu_allocator_test 2024-08-06T20:40:49.1531278Z inflating: build/bin/broadcast_test 2024-08-06T20:40:49.1604099Z inflating: build/bin/cpu_generator_test 2024-08-06T20:40:49.1670906Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-06T20:40:49.1734952Z inflating: build/bin/dispatch_key_set_test 2024-08-06T20:40:49.1797955Z inflating: build/bin/dlconvertor_test 2024-08-06T20:40:49.1914595Z inflating: build/bin/cpu_rng_test 2024-08-06T20:40:49.1987499Z inflating: build/bin/extension_backend_test 2024-08-06T20:40:49.2056576Z inflating: build/bin/half_test 2024-08-06T20:40:49.2176298Z inflating: build/bin/ivalue_test 2024-08-06T20:40:49.2238675Z inflating: build/bin/lazy_tensor_test 2024-08-06T20:40:49.2306153Z inflating: build/bin/memory_format_test 2024-08-06T20:40:49.2372579Z inflating: build/bin/memory_overlapping_test 2024-08-06T20:40:49.2440768Z inflating: build/bin/math_kernel_test 2024-08-06T20:40:49.2507236Z inflating: build/bin/mobile_memory_cleanup 2024-08-06T20:40:49.2577024Z inflating: build/bin/native_test 2024-08-06T20:40:49.2640468Z inflating: build/bin/operator_name_test 2024-08-06T20:40:49.2704787Z inflating: build/bin/operators_test 2024-08-06T20:40:49.2769905Z inflating: build/bin/packedtensoraccessor_test 2024-08-06T20:40:49.2853986Z inflating: build/bin/pow_test 2024-08-06T20:40:49.2925505Z inflating: build/bin/quantized_test 2024-08-06T20:40:49.2989680Z inflating: build/bin/reportMemoryUsage_test 2024-08-06T20:40:49.3052089Z inflating: build/bin/reduce_ops_test 2024-08-06T20:40:49.3122870Z inflating: build/bin/scalar_tensor_test 2024-08-06T20:40:49.3187808Z inflating: build/bin/StorageUtils_test 2024-08-06T20:40:49.3260891Z inflating: build/bin/scalar_test 2024-08-06T20:40:49.3326777Z inflating: build/bin/stride_properties_test 2024-08-06T20:40:49.3425162Z inflating: build/bin/tensor_iterator_test 2024-08-06T20:40:49.3492983Z inflating: build/bin/test_parallel 2024-08-06T20:40:49.3496716Z inflating: build/bin/thread_init_test 2024-08-06T20:40:49.3566659Z inflating: build/bin/type_ptr_test 2024-08-06T20:40:49.3641445Z inflating: build/bin/type_test 2024-08-06T20:40:49.3708860Z inflating: build/bin/undefined_tensor_test 2024-08-06T20:40:49.3711032Z inflating: build/bin/verify_api_visibility 2024-08-06T20:40:49.3775516Z inflating: build/bin/weakref_test 2024-08-06T20:40:49.3862115Z inflating: build/bin/legacy_vmap_test 2024-08-06T20:40:49.3926554Z inflating: build/bin/wrapdim_test 2024-08-06T20:40:49.4059549Z inflating: build/bin/List_test 2024-08-06T20:40:49.4134898Z inflating: build/bin/IListRef_test 2024-08-06T20:40:49.4199634Z inflating: build/bin/xla_tensor_test 2024-08-06T20:40:49.4352234Z inflating: build/bin/kernel_function_legacy_test 2024-08-06T20:40:49.4473199Z inflating: build/bin/kernel_function_test 2024-08-06T20:40:49.4556468Z inflating: build/bin/KernelFunction_test 2024-08-06T20:40:49.4717255Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-06T20:40:49.4846768Z inflating: build/bin/kernel_lambda_test 2024-08-06T20:40:49.4923402Z inflating: build/bin/kernel_stackbased_test 2024-08-06T20:40:49.4987291Z inflating: build/bin/CppSignature_test 2024-08-06T20:40:49.5107594Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-06T20:40:49.5168801Z inflating: build/bin/op_allowlist_test 2024-08-06T20:40:49.5238778Z inflating: build/bin/backend_fallback_test 2024-08-06T20:40:49.5318243Z inflating: build/bin/inline_container_test 2024-08-06T20:40:49.5697438Z inflating: build/bin/op_registration_test 2024-08-06T20:40:49.5764758Z inflating: build/bin/cuda_apply_test 2024-08-06T20:40:49.5829729Z inflating: build/bin/cuda_allocator_test 2024-08-06T20:40:49.5897870Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-06T20:40:49.5985810Z inflating: build/bin/cuda_complex_math_test 2024-08-06T20:40:49.6059852Z inflating: build/bin/cuda_atomic_ops_test 2024-08-06T20:40:49.6133712Z inflating: build/bin/cuda_complex_test 2024-08-06T20:40:49.6195917Z inflating: build/bin/cuda_device_test 2024-08-06T20:40:49.6268340Z inflating: build/bin/cuda_cub_test 2024-08-06T20:40:49.6331634Z inflating: build/bin/cuda_dlconvertor_test 2024-08-06T20:40:49.6395708Z inflating: build/bin/cuda_integer_divider_test 2024-08-06T20:40:49.6477432Z inflating: build/bin/cuda_distributions_test 2024-08-06T20:40:49.6548348Z inflating: build/bin/cuda_generator_test 2024-08-06T20:40:49.6610556Z inflating: build/bin/cuda_half_test 2024-08-06T20:40:49.6672472Z inflating: build/bin/cuda_optional_test 2024-08-06T20:40:49.6734701Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-06T20:40:49.6810676Z inflating: build/bin/cuda_stream_test 2024-08-06T20:40:49.6876505Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-06T20:40:49.6940669Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-06T20:40:49.7002559Z inflating: build/bin/cuda_cudnn_test 2024-08-06T20:40:49.7067873Z inflating: build/bin/cuda_vectorized_test 2024-08-06T20:40:49.7085775Z inflating: build/bin/tutorial_tensorexpr 2024-08-06T20:40:49.7167700Z inflating: build/bin/ProcessGroupGlooTest 2024-08-06T20:40:49.7239584Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-06T20:40:49.7305089Z inflating: build/bin/ProcessGroupUCCTest 2024-08-06T20:40:49.7382734Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-06T20:40:49.7462272Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-06T20:40:49.7530985Z inflating: build/bin/test_dist_autograd 2024-08-06T20:40:49.7534169Z inflating: build/bin/parallel_benchmark 2024-08-06T20:40:49.7618045Z inflating: build/bin/test_mobile_nnc 2024-08-06T20:40:49.7705515Z inflating: build/bin/test_cpp_rpc 2024-08-06T20:40:49.7716966Z inflating: build/bin/aot_model_compiler_test 2024-08-06T20:40:49.8749933Z inflating: build/bin/test_tensorexpr 2024-08-06T20:40:49.9182612Z inflating: build/bin/test_lazy 2024-08-06T20:40:50.0658950Z inflating: build/bin/test_api 2024-08-06T20:40:50.1383047Z inflating: build/bin/test_jit 2024-08-06T20:40:50.1383504Z creating: .additional_ci_files/ 2024-08-06T20:40:50.1455629Z inflating: .additional_ci_files/test-times.json 2024-08-06T20:40:50.1744047Z inflating: .additional_ci_files/test-class-times.json 2024-08-06T20:40:50.3089840Z ##[group]Run rm artifacts.zip 2024-08-06T20:40:50.3090357Z rm artifacts.zip 2024-08-06T20:40:50.3098573Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:50.3099128Z env: 2024-08-06T20:40:50.3099473Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:50.3099978Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:50.3100511Z ##[endgroup] 2024-08-06T20:40:51.0074385Z ##[group]Run df -H 2024-08-06T20:40:51.0075049Z df -H 2024-08-06T20:40:51.0087523Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:51.0088473Z env: 2024-08-06T20:40:51.0088997Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:51.0089785Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:51.0090687Z ##[endgroup] 2024-08-06T20:40:51.0134429Z Filesystem Size Used Avail Use% Mounted on 2024-08-06T20:40:51.0135550Z devtmpfs 129G 0 129G 0% /dev 2024-08-06T20:40:51.0136439Z tmpfs 129G 25k 129G 1% /dev/shm 2024-08-06T20:40:51.0137829Z tmpfs 129G 594k 129G 1% /run 2024-08-06T20:40:51.0138682Z tmpfs 129G 0 129G 0% /sys/fs/cgroup 2024-08-06T20:40:51.0139521Z /dev/xvda1 162G 38G 125G 23% / 2024-08-06T20:40:51.0140319Z tmpfs 26G 0 26G 0% /run/user/0 2024-08-06T20:40:51.0211471Z Prepare all required actions 2024-08-06T20:40:51.0212349Z Getting action download info 2024-08-06T20:40:51.1376489Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-06T20:40:51.1376958Z with: 2024-08-06T20:40:51.1377256Z env: 2024-08-06T20:40:51.1377542Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:51.1378000Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:51.1378474Z ##[endgroup] 2024-08-06T20:40:51.1437936Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-06T20:40:51.1438390Z with: 2024-08-06T20:40:51.1438690Z name: td_results 2024-08-06T20:40:51.1439005Z s3-bucket: gha-artifacts 2024-08-06T20:40:51.1439386Z region: us-east-1 2024-08-06T20:40:51.1439701Z env: 2024-08-06T20:40:51.1439977Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:51.1440420Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:51.1440877Z ##[endgroup] 2024-08-06T20:40:51.7662023Z (node:10169) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-06T20:40:51.7662502Z 2024-08-06T20:40:51.7662800Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-06T20:40:51.7663450Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-06T20:40:51.7664188Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-06T20:40:51.8507180Z Found 1 objects with prefix pytorch/pytorch/10273124344/td_results/ 2024-08-06T20:40:51.8507950Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-06T20:40:51.9206075Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-06T20:40:51.9216553Z Artifact download has finished successfully 2024-08-06T20:40:51.9411862Z ##[group]Run mkdir -p .additional_ci_files 2024-08-06T20:40:51.9412404Z mkdir -p .additional_ci_files 2024-08-06T20:40:51.9412985Z mv td_results.json .additional_ci_files/td_results.json 2024-08-06T20:40:51.9421581Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:51.9422132Z env: 2024-08-06T20:40:51.9422472Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:51.9422973Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:51.9423499Z ##[endgroup] 2024-08-06T20:40:51.9507384Z ##[group]Run .github/scripts/parse_ref.py 2024-08-06T20:40:51.9507892Z .github/scripts/parse_ref.py 2024-08-06T20:40:51.9515239Z shell: /usr/bin/bash -e {0} 2024-08-06T20:40:51.9515587Z env: 2024-08-06T20:40:51.9515882Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:51.9516323Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:51.9516783Z ##[endgroup] 2024-08-06T20:40:51.9822008Z Prepare all required actions 2024-08-06T20:40:51.9867066Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-06T20:40:51.9867501Z with: 2024-08-06T20:40:51.9868024Z github-token: *** 2024-08-06T20:40:51.9868351Z env: 2024-08-06T20:40:51.9868643Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:51.9869060Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:51.9869519Z ##[endgroup] 2024-08-06T20:40:51.9898621Z ##[group]Run set -eux 2024-08-06T20:40:51.9898989Z set -eux 2024-08-06T20:40:51.9899529Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-06T20:40:51.9907404Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:51.9907893Z env: 2024-08-06T20:40:51.9908176Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:51.9908608Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:51.9909268Z GITHUB_TOKEN: *** 2024-08-06T20:40:51.9909799Z ##[endgroup] 2024-08-06T20:40:51.9933440Z + python3 .github/scripts/get_workflow_job_id.py 10273124344 i-0b13d1134efc770c9 2024-08-06T20:40:54.8827849Z setting job-id=28428107361 2024-08-06T20:40:54.8828596Z setting job-name=linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 1, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:40:54.9058288Z Prepare all required actions 2024-08-06T20:40:54.9058869Z Getting action download info 2024-08-06T20:40:55.0110242Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-06T20:40:55.0110680Z with: 2024-08-06T20:40:55.0111203Z github-token: *** 2024-08-06T20:40:55.0112479Z test-matrix: {"include": [{"config": "distributed", "shard": 1, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 2, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 3, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}]} 2024-08-06T20:40:55.0114058Z job-name: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 1, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:40:55.0114703Z env: 2024-08-06T20:40:55.0115006Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:55.0115442Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:55.0115879Z ##[endgroup] 2024-08-06T20:40:55.0169553Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-06T20:40:55.0170099Z with: 2024-08-06T20:40:55.0170389Z shell: bash 2024-08-06T20:40:55.0170683Z timeout_minutes: 10 2024-08-06T20:40:55.0171021Z max_attempts: 5 2024-08-06T20:40:55.0171340Z retry_wait_seconds: 30 2024-08-06T20:40:55.0172268Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-06T20:40:55.0173248Z polling_interval_seconds: 1 2024-08-06T20:40:55.0173622Z warning_on_retry: true 2024-08-06T20:40:55.0173973Z continue_on_error: false 2024-08-06T20:40:55.0174296Z env: 2024-08-06T20:40:55.0174594Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:55.0175027Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:55.0175656Z GITHUB_TOKEN: *** 2024-08-06T20:40:55.0175979Z ##[endgroup] 2024-08-06T20:40:55.0891937Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-06T20:40:55.3772401Z Defaulting to user installation because normal site-packages is not writeable 2024-08-06T20:40:55.3979967Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.7/site-packages (2.27.1) 2024-08-06T20:40:55.4164043Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.7/site-packages (6.0.1) 2024-08-06T20:40:55.4174982Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-06T20:40:55.4185914Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (1.26.19) 2024-08-06T20:40:55.4436768Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (3.7) 2024-08-06T20:40:55.4452439Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2.0.12) 2024-08-06T20:40:56.0881725Z Command completed after 1 attempt(s). 2024-08-06T20:40:56.0976459Z ##[group]Run set -x 2024-08-06T20:40:56.0976814Z set -x 2024-08-06T20:40:56.0977103Z  2024-08-06T20:40:56.0977650Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-06T20:40:56.0978244Z # in runner workspace 2024-08-06T20:40:56.0978720Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-06T20:40:56.0987255Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:56.0987741Z env: 2024-08-06T20:40:56.0988019Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:56.0988673Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:56.0989133Z ##[endgroup] 2024-08-06T20:40:56.1014565Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-06T20:40:56.1306138Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-06T20:40:56.1306760Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-06T20:40:56.1307266Z echo "Job name: ${JOB_NAME}" 2024-08-06T20:40:56.1307721Z  2024-08-06T20:40:56.1308254Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-06T20:40:56.1308919Z # in runner workspace 2024-08-06T20:40:56.1309527Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-06T20:40:56.1310202Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-06T20:40:56.1310687Z  --job-name "${JOB_NAME}" \ 2024-08-06T20:40:56.1312152Z  --test-matrix "{"include": [{"config": "distributed", "shard": 1, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 2, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 3, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}]}" \ 2024-08-06T20:40:56.1313620Z  --selected-test-configs "" \ 2024-08-06T20:40:56.1314126Z  --pr-number "${PR_NUMBER}" \ 2024-08-06T20:40:56.1314583Z  --tag "${TAG}" \ 2024-08-06T20:40:56.1315040Z  --event-name "${EVENT_NAME}" \ 2024-08-06T20:40:56.1315529Z  --schedule "${SCHEDULE}" \ 2024-08-06T20:40:56.1315985Z  --branch "${HEAD_BRANCH}" 2024-08-06T20:40:56.1339573Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:56.1340126Z env: 2024-08-06T20:40:56.1340472Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:56.1340976Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:56.1341760Z GITHUB_TOKEN: *** 2024-08-06T20:40:56.1342449Z JOB_NAME: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 1, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:40:56.1343166Z PR_NUMBER: 132710 2024-08-06T20:40:56.1343554Z TAG: 2024-08-06T20:40:56.1343898Z EVENT_NAME: pull_request 2024-08-06T20:40:56.1344328Z SCHEDULE: 2024-08-06T20:40:56.1344776Z HEAD_BRANCH: 2024-08-06T20:40:56.1345130Z ##[endgroup] 2024-08-06T20:40:56.1370412Z Workflow: pull 2024-08-06T20:40:56.1371127Z Job name: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 1, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:40:56.3927506Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-06T20:40:56.5260489Z ##[group]Run echo "Filtered matrix:" 2024-08-06T20:40:56.5260938Z echo "Filtered matrix:" 2024-08-06T20:40:56.5262245Z echo "{"include": [{"config": "distributed", "shard": 1, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 2, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 3, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}]}" 2024-08-06T20:40:56.5263589Z  2024-08-06T20:40:56.5263877Z echo 2024-08-06T20:40:56.5264244Z echo "Is the current job unstable? False" 2024-08-06T20:40:56.5264679Z  2024-08-06T20:40:56.5264963Z echo 2024-08-06T20:40:56.5265291Z echo "Is keep-going label set? False" 2024-08-06T20:40:56.5266019Z  2024-08-06T20:40:56.5266306Z echo 2024-08-06T20:40:56.5266603Z echo "Renabled issues? " 2024-08-06T20:40:56.5274654Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:56.5275139Z env: 2024-08-06T20:40:56.5275416Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:56.5275852Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:56.5276309Z ##[endgroup] 2024-08-06T20:40:56.5300464Z Filtered matrix: 2024-08-06T20:40:56.5301884Z {include: [{config: distributed, shard: 1, num_shards: 3, runner: linux.8xlarge.nvidia.gpu}, {config: distributed, shard: 2, num_shards: 3, runner: linux.8xlarge.nvidia.gpu}, {config: distributed, shard: 3, num_shards: 3, runner: linux.8xlarge.nvidia.gpu}]} 2024-08-06T20:40:56.5303336Z 2024-08-06T20:40:56.5303680Z Is the current job unstable? False 2024-08-06T20:40:56.5303944Z 2024-08-06T20:40:56.5304109Z Is keep-going label set? False 2024-08-06T20:40:56.5304352Z 2024-08-06T20:40:56.5304488Z Renabled issues? 2024-08-06T20:40:56.5359638Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-06T20:40:56.5360327Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-06T20:40:56.5367623Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:56.5368100Z env: 2024-08-06T20:40:56.5368396Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:56.5368838Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:56.5369287Z JOB_TIMEOUT: 360 2024-08-06T20:40:56.5369615Z ##[endgroup] 2024-08-06T20:40:56.5451481Z ##[group]Run set -x 2024-08-06T20:40:56.5451938Z set -x 2024-08-06T20:40:56.5452232Z  2024-08-06T20:40:56.5452571Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-06T20:40:56.5453073Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-06T20:40:56.5453563Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-06T20:40:56.5454029Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-06T20:40:56.5454427Z else 2024-08-06T20:40:56.5454749Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-06T20:40:56.5455151Z fi 2024-08-06T20:40:56.5455440Z  2024-08-06T20:40:56.5455860Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-06T20:40:56.5456521Z # TODO: Stop building test binaries as part of the build phase 2024-08-06T20:40:56.5457127Z # Used for GPU_FLAG since that doesn't play nice 2024-08-06T20:40:56.5457675Z # shellcheck disable=SC2086,SC2090 2024-08-06T20:40:56.5458099Z container_name=$(docker run \ 2024-08-06T20:40:56.5458503Z  ${GPU_FLAG:-} \ 2024-08-06T20:40:56.5458877Z  -e BUILD_ENVIRONMENT \ 2024-08-06T20:40:56.5459251Z  -e PR_NUMBER \ 2024-08-06T20:40:56.5459618Z  -e GITHUB_ACTIONS \ 2024-08-06T20:40:56.5459997Z  -e GITHUB_REPOSITORY \ 2024-08-06T20:40:56.5460372Z  -e GITHUB_WORKFLOW \ 2024-08-06T20:40:56.5460752Z  -e GITHUB_JOB \ 2024-08-06T20:40:56.5461115Z  -e GITHUB_RUN_ID \ 2024-08-06T20:40:56.5461465Z  -e GITHUB_RUN_NUMBER \ 2024-08-06T20:40:56.5461856Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-06T20:40:56.5462241Z  -e JOB_ID \ 2024-08-06T20:40:56.5462567Z  -e JOB_NAME \ 2024-08-06T20:40:56.5462920Z  -e BASE_SHA \ 2024-08-06T20:40:56.5463265Z  -e BRANCH \ 2024-08-06T20:40:56.5463584Z  -e SHA1 \ 2024-08-06T20:40:56.5463951Z  -e AWS_DEFAULT_REGION \ 2024-08-06T20:40:56.5464343Z  -e IN_WHEEL_TEST \ 2024-08-06T20:40:56.5464689Z  -e SHARD_NUMBER \ 2024-08-06T20:40:56.5465057Z  -e TEST_CONFIG \ 2024-08-06T20:40:56.5465422Z  -e NUM_TEST_SHARDS \ 2024-08-06T20:40:56.5466202Z  -e REENABLED_ISSUES \ 2024-08-06T20:40:56.5466613Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-06T20:40:56.5467023Z  -e VERBOSE_TEST_LOGS \ 2024-08-06T20:40:56.5467396Z  -e TEST_SHOWLOCALS \ 2024-08-06T20:40:56.5467779Z  -e NO_TEST_TIMEOUT \ 2024-08-06T20:40:56.5468148Z  -e NO_TD \ 2024-08-06T20:40:56.5468475Z  -e TD_DISTRIBUTED \ 2024-08-06T20:40:56.5468850Z  -e PR_LABELS \ 2024-08-06T20:40:56.5469237Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-06T20:40:56.5469649Z  -e SCCACHE_BUCKET \ 2024-08-06T20:40:56.5470036Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-06T20:40:56.5470622Z  -e XLA_CUDA \ 2024-08-06T20:40:56.5470993Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-06T20:40:56.5471460Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-06T20:40:56.5471935Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-06T20:40:56.5472392Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-06T20:40:56.5472836Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-06T20:40:56.5473239Z  -e DASHBOARD_TAG \ 2024-08-06T20:40:56.5473659Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-06T20:40:56.5474167Z  --security-opt seccomp=unconfined \ 2024-08-06T20:40:56.5474610Z  --cap-add=SYS_PTRACE \ 2024-08-06T20:40:56.5474979Z  --ipc=host \ 2024-08-06T20:40:56.5475336Z  --shm-size="${SHM_SIZE}" \ 2024-08-06T20:40:56.5475724Z  --tty \ 2024-08-06T20:40:56.5476024Z  --detach \ 2024-08-06T20:40:56.5476380Z  --name="${container_name}" \ 2024-08-06T20:40:56.5476916Z  --user jenkins \ 2024-08-06T20:40:56.5477369Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-06T20:40:56.5477875Z  -w /var/lib/jenkins/workspace \ 2024-08-06T20:40:56.5478287Z  "${DOCKER_IMAGE}" 2024-08-06T20:40:56.5478619Z ) 2024-08-06T20:40:56.5479007Z # Propagate download.pytorch.org IP to container 2024-08-06T20:40:56.5479820Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-06T20:40:56.5480659Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-06T20:40:56.5481457Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-06T20:40:56.5489036Z shell: /usr/bin/bash -e {0} 2024-08-06T20:40:56.5489383Z env: 2024-08-06T20:40:56.5489676Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:56.5490111Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:56.5490675Z BUILD_ENVIRONMENT: linux-focal-cuda11.8-py3.10-gcc9 2024-08-06T20:40:56.5491122Z PR_NUMBER: 132710 2024-08-06T20:40:56.5491473Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-06T20:40:56.5491861Z GITHUB_WORKFLOW: pull 2024-08-06T20:40:56.5492198Z GITHUB_JOB: test 2024-08-06T20:40:56.5492524Z GITHUB_RUN_ID: 10273124344 2024-08-06T20:40:56.5492876Z GITHUB_RUN_NUMBER: 233985 2024-08-06T20:40:56.5493234Z GITHUB_RUN_ATTEMPT: 1 2024-08-06T20:40:56.5493571Z JOB_ID: 28428107361 2024-08-06T20:40:56.5494140Z JOB_NAME: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 1, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:40:56.5494783Z BRANCH: pull/132710 2024-08-06T20:40:56.5495158Z SHA1: b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:40:56.5495630Z BASE_SHA: 1736af7cf736184c356be1bb00f59fb2feea6d7d 2024-08-06T20:40:56.5496074Z TEST_CONFIG: distributed 2024-08-06T20:40:56.5496427Z SHARD_NUMBER: 1 2024-08-06T20:40:56.5496732Z NUM_TEST_SHARDS: 3 2024-08-06T20:40:56.5497076Z REENABLED_ISSUES: 2024-08-06T20:40:56.5497418Z CONTINUE_THROUGH_ERROR: False 2024-08-06T20:40:56.5497780Z VERBOSE_TEST_LOGS: False 2024-08-06T20:40:56.5498135Z TEST_SHOWLOCALS: False 2024-08-06T20:40:56.5498484Z NO_TEST_TIMEOUT: False 2024-08-06T20:40:56.5498830Z NO_TD: False 2024-08-06T20:40:56.5499140Z TD_DISTRIBUTED: False 2024-08-06T20:40:56.5499526Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-06T20:40:56.5500007Z SCCACHE_S3_KEY_PREFIX: pull 2024-08-06T20:40:56.5500369Z SHM_SIZE: 2g 2024-08-06T20:40:56.5501214Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:40:56.5502156Z XLA_CUDA: 2024-08-06T20:40:56.5502621Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-06T20:40:56.5503196Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-06T20:40:56.5503605Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-06T20:40:56.5504136Z DASHBOARD_TAG: 2024-08-06T20:40:56.5504466Z HUGGING_FACE_HUB_TOKEN: 2024-08-06T20:40:56.5504803Z ##[endgroup] 2024-08-06T20:40:56.5528862Z + [[ distributed == \m\u\l\t\i\g\p\u ]] 2024-08-06T20:40:56.5529332Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *onnx* ]] 2024-08-06T20:40:56.5529721Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-06T20:40:56.5539319Z +++ nproc --ignore=2 2024-08-06T20:40:56.5555037Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=30 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10273124344 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:41:07.6753528Z + container_name=ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:41:07.6756473Z + grep download.pytorch.org /etc/hosts 2024-08-06T20:41:07.6758198Z + docker exec -i ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-06T20:41:07.7623133Z + echo DOCKER_CONTAINER_ID=ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:41:07.7627479Z ++ echo dist/torch-2.5.0a0+gitb9d86fa-cp310-cp310-linux_x86_64.whl 2024-08-06T20:41:07.7629401Z + docker exec -t ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e sh -c 'pip install dist/torch-2.5.0a0+gitb9d86fa-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-06T20:41:08.3063882Z Processing ./dist/torch-2.5.0a0+gitb9d86fa-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+gitb9d86fa) 2024-08-06T20:41:09.4419359Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (3.13.1) 2024-08-06T20:41:09.4426777Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (4.12.2) 2024-08-06T20:41:09.4432457Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (2.8.8) 2024-08-06T20:41:09.4437423Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (3.1.4) 2024-08-06T20:41:09.4442586Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (2024.6.1) 2024-08-06T20:41:09.4456871Z Requirement already satisfied: sympy>=1.13.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (1.13.1) 2024-08-06T20:41:09.4499370Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (3.3.0) 2024-08-06T20:41:09.4582268Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (1.21.2) 2024-08-06T20:41:09.4632663Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.0->torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (1.3.0) 2024-08-06T20:41:09.5945971Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (2.1.5) 2024-08-06T20:41:10.7740891Z Installing collected packages: torch 2024-08-06T20:41:23.7141007Z Successfully installed torch-2.5.0a0+gitb9d86fa 2024-08-06T20:41:23.7843665Z ++ dirname .ci/pytorch/test.sh 2024-08-06T20:41:23.7850869Z + source .ci/pytorch/common.sh 2024-08-06T20:41:23.7854579Z +++ dirname .ci/pytorch/common.sh 2024-08-06T20:41:23.7862689Z ++ source .ci/pytorch/common_utils.sh 2024-08-06T20:41:23.7865377Z +++ declare -f -t trap_add 2024-08-06T20:41:23.7872652Z ++ set -ex 2024-08-06T20:41:23.7873182Z ++ [[ linux-focal-cuda11.8-py3.10-gcc9 == *rocm* ]] 2024-08-06T20:41:23.7874247Z ++ BUILD_TEST_LIBTORCH=0 2024-08-06T20:41:23.7875216Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *rocm* ]] 2024-08-06T20:41:23.7878107Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-06T20:41:23.7889400Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-06T20:41:23.7889899Z + trap_add cleanup_workspace EXIT 2024-08-06T20:41:23.7890374Z + trap_add_cmd=cleanup_workspace 2024-08-06T20:41:23.7890793Z + shift 2024-08-06T20:41:23.7891165Z + for trap_add_name in "$@" 2024-08-06T20:41:23.7899716Z +++ trap -p EXIT 2024-08-06T20:41:23.7900605Z ++ eval 'extract_trap_cmd ' 2024-08-06T20:41:23.7901040Z +++ extract_trap_cmd 2024-08-06T20:41:23.7901327Z +++ printf '%s\n' '' 2024-08-06T20:41:23.7901719Z ++ printf '%s\n' cleanup_workspace 2024-08-06T20:41:23.7904843Z + trap -- ' 2024-08-06T20:41:23.7905088Z cleanup_workspace' EXIT 2024-08-06T20:41:23.7905790Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-06T20:41:24.3325977Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-06T20:41:24.3344830Z + echo 'Environment variables:' 2024-08-06T20:41:24.3345233Z Environment variables: 2024-08-06T20:41:24.3345780Z + env 2024-08-06T20:41:24.3355250Z INSTALLED_DB=yes 2024-08-06T20:41:24.3355865Z NV_LIBCUBLAS_VERSION=11.11.3.6-1 2024-08-06T20:41:24.3356422Z NVIDIA_VISIBLE_DEVICES=all 2024-08-06T20:41:24.3356793Z NV_NVML_DEV_VERSION=11.8.86-1 2024-08-06T20:41:24.3357230Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:41:24.3357857Z CONTINUE_THROUGH_ERROR=False 2024-08-06T20:41:24.3358411Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.16.2-1+cuda11.8 2024-08-06T20:41:24.3358837Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.16.2-1 2024-08-06T20:41:24.3359353Z BUILD_ENVIRONMENT=linux-focal-cuda11.8-py3.10-gcc9 2024-08-06T20:41:24.3359797Z HOSTNAME=ed694363f944 2024-08-06T20:41:24.3360509Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_9ec21232-fd46-4a04-a696-b627680afb3a 2024-08-06T20:41:24.3361289Z GITHUB_ACTION=__self 2024-08-06T20:41:24.3361660Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-06T20:41:24.3363499Z NVIDIA_REQUIRE_CUDA=cuda>=11.8 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 2024-08-06T20:41:24.3365483Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-11-8=11.11.3.6-1 2024-08-06T20:41:24.3365930Z NV_NVTX_VERSION=11.8.86-1 2024-08-06T20:41:24.3366282Z GITHUB_RUN_NUMBER=233985 2024-08-06T20:41:24.3366629Z TEST_CONFIG=distributed 2024-08-06T20:41:24.3366969Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-06T20:41:24.3367388Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-06T20:41:24.3367809Z NV_CUDA_CUDART_DEV_VERSION=11.8.89-1 2024-08-06T20:41:24.3368199Z NV_LIBCUSPARSE_VERSION=11.7.5.86-1 2024-08-06T20:41:24.3368587Z NV_LIBNPP_VERSION=11.8.0.86-1 2024-08-06T20:41:24.3369255Z GITHUB_TRIGGERING_ACTOR=drisspg 2024-08-06T20:41:24.3369692Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-06T20:41:24.3370142Z GITHUB_REF_TYPE=branch 2024-08-06T20:41:24.3370548Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-06T20:41:24.3370891Z NCCL_VERSION=2.16.2-1 2024-08-06T20:41:24.3371275Z BASE_SHA=1736af7cf736184c356be1bb00f59fb2feea6d7d 2024-08-06T20:41:24.3371684Z XLA_CUDA= 2024-08-06T20:41:24.3371984Z HUGGING_FACE_HUB_TOKEN= 2024-08-06T20:41:24.3372508Z *** 2024-08-06T20:41:24.3372790Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-06T20:41:24.3373187Z GITHUB_REPOSITORY_ID=65600975 2024-08-06T20:41:24.3373548Z GITHUB_ACTIONS=true 2024-08-06T20:41:24.3373868Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:41:24.3374325Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-11-8=11.8.87-1 2024-08-06T20:41:24.3374776Z NV_LIBNPP_PACKAGE=libnpp-11-8=11.8.0.86-1 2024-08-06T20:41:24.3375222Z SHA1=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:41:24.3375668Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-06T20:41:24.3376276Z GITHUB_SHA=bf5bb5a1585a03379137fab341e87c02c77e76cd 2024-08-06T20:41:24.3376962Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/132710/merge 2024-08-06T20:41:24.3377570Z UCC_HOME=/usr 2024-08-06T20:41:24.3377875Z NV_LIBCUBLAS_DEV_VERSION=11.11.3.6-1 2024-08-06T20:41:24.3378276Z VERBOSE_TEST_LOGS=False 2024-08-06T20:41:24.3378666Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-06T20:41:24.3379053Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-11-8 2024-08-06T20:41:24.3379509Z GITHUB_REF=refs/pull/132710/merge 2024-08-06T20:41:24.3379905Z NV_CUDA_CUDART_VERSION=11.8.89-1 2024-08-06T20:41:24.3380270Z SHARD_NUMBER=1 2024-08-06T20:41:24.3380578Z GITHUB_REF_PROTECTED=false 2024-08-06T20:41:24.3380938Z HOME=/var/lib/jenkins 2024-08-06T20:41:24.3381307Z GITHUB_API_URL=https://api.github.com 2024-08-06T20:41:24.3381748Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-06T20:41:24.3382199Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-06T20:41:24.3382643Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-06T20:41:24.3383006Z CUDA_VERSION=11.8.0 2024-08-06T20:41:24.3383378Z NV_LIBCUBLAS_PACKAGE=libcublas-11-8=11.11.3.6-1 2024-08-06T20:41:24.3383789Z NUM_TEST_SHARDS=3 2024-08-06T20:41:24.3384100Z UCX_HOME=/usr 2024-08-06T20:41:24.3384543Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-11-8=11.8.0-1 2024-08-06T20:41:24.3385458Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_9ec21232-fd46-4a04-a696-b627680afb3a 2024-08-06T20:41:24.3386888Z JOB_NAME=linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 1, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:41:24.3387973Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_9ec21232-fd46-4a04-a696-b627680afb3a 2024-08-06T20:41:24.3388907Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-06T20:41:24.3389531Z GITHUB_EVENT_NAME=pull_request 2024-08-06T20:41:24.3389897Z DASHBOARD_TAG= 2024-08-06T20:41:24.3390204Z GITHUB_RUN_ID=10273124344 2024-08-06T20:41:24.3390608Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-11-8=11.8.0.86-1 2024-08-06T20:41:24.3391141Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-11-8 2024-08-06T20:41:24.3392252Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_9ec21232-fd46-4a04-a696-b627680afb3a 2024-08-06T20:41:24.3393056Z GITHUB_ACTOR=drisspg 2024-08-06T20:41:24.3393396Z NV_LIBNPP_DEV_VERSION=11.8.0.86-1 2024-08-06T20:41:24.3393754Z PR_NUMBER=132710 2024-08-06T20:41:24.3394070Z GITHUB_RUN_ATTEMPT=1 2024-08-06T20:41:24.3394415Z ANACONDA_PYTHON_VERSION=3.10 2024-08-06T20:41:24.3394832Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-06T20:41:24.3395276Z TERM=xterm 2024-08-06T20:41:24.3395726Z NV_LIBCUSPARSE_DEV_VERSION=11.7.5.86-1 2024-08-06T20:41:24.3396111Z INSTALLED_VISION=yes 2024-08-06T20:41:24.3396433Z BRANCH=pull/132710 2024-08-06T20:41:24.3396768Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-06T20:41:24.3397143Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-06T20:41:24.3397772Z CUDA_PATH=/usr/local/cuda 2024-08-06T20:41:24.3398409Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-06T20:41:24.3399098Z GITHUB_SERVER_URL=https://github.com 2024-08-06T20:41:24.3399588Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-06T20:41:24.3400021Z REENABLED_ISSUES= 2024-08-06T20:41:24.3400320Z SHLVL=1 2024-08-06T20:41:24.3400594Z MAX_JOBS=30 2024-08-06T20:41:24.3400899Z NV_CUDA_LIB_VERSION=11.8.0-1 2024-08-06T20:41:24.3401243Z NVARCH=x86_64 2024-08-06T20:41:24.3401553Z GITHUB_ACTOR_ID=32754868 2024-08-06T20:41:24.3401996Z GITHUB_WORKFLOW_SHA=bf5bb5a1585a03379137fab341e87c02c77e76cd 2024-08-06T20:41:24.3402871Z GITHUB_REF_NAME=132710/merge 2024-08-06T20:41:24.3403337Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-11-8 2024-08-06T20:41:24.3403913Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-06T20:41:24.3404455Z GITHUB_JOB=test 2024-08-06T20:41:24.3404790Z NV_LIBNCCL_PACKAGE=libnccl2=2.16.2-1+cuda11.8 2024-08-06T20:41:24.3405455Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-06T20:41:24.3405965Z NO_TEST_TIMEOUT=False 2024-08-06T20:41:24.3406301Z TD_DISTRIBUTED=False 2024-08-06T20:41:24.3406641Z NV_CUDA_NSIGHT_COMPUTE_VERSION=11.8.0-1 2024-08-06T20:41:24.3407067Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-06T20:41:24.3407461Z NV_NVPROF_VERSION=11.8.87-1 2024-08-06T20:41:24.3407807Z GITHUB_RETENTION_DAYS=90 2024-08-06T20:41:24.3408162Z OPENSSL_DIR=/opt/openssl 2024-08-06T20:41:24.3408519Z GITHUB_ACTION_REPOSITORY= 2024-08-06T20:41:24.3409422Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:41:24.3410382Z GITHUB_BASE_REF=gh/drisspg/23/base 2024-08-06T20:41:24.3410784Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-06T20:41:24.3411130Z CI=true 2024-08-06T20:41:24.3411432Z NV_LIBNCCL_PACKAGE_VERSION=2.16.2-1 2024-08-06T20:41:24.3411845Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-06T20:41:24.3412201Z JOB_ID=28428107361 2024-08-06T20:41:24.3412528Z INSTALLED_PROTOBUF=yes 2024-08-06T20:41:24.3412878Z GITHUB_HEAD_REF=gh/drisspg/23/head 2024-08-06T20:41:24.3413239Z GITHUB_ACTION_REF= 2024-08-06T20:41:24.3413621Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-06T20:41:24.3414071Z TEST_SHOWLOCALS=False 2024-08-06T20:41:24.3414387Z GITHUB_WORKFLOW=pull 2024-08-06T20:41:24.3414729Z DEBIAN_FRONTEND=noninteractive 2024-08-06T20:41:24.3415487Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_9ec21232-fd46-4a04-a696-b627680afb3a 2024-08-06T20:41:24.3416235Z NO_TD=False 2024-08-06T20:41:24.3416548Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-06T20:41:24.3416912Z _=/usr/bin/env 2024-08-06T20:41:24.3417305Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-06T20:41:24.3583616Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-06T20:41:24.3584379Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-06T20:41:24.3585128Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-06T20:41:24.3586190Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-06T20:41:24.3586780Z + BUILD_DIR=build 2024-08-06T20:41:24.3587094Z + BUILD_RENAMED_DIR=build_renamed 2024-08-06T20:41:24.3587456Z + BUILD_BIN_DIR=build/bin 2024-08-06T20:41:24.3587845Z + SHARD_NUMBER=1 2024-08-06T20:41:24.3588106Z + NUM_TEST_SHARDS=3 2024-08-06T20:41:24.3588455Z + export VALGRIND=ON 2024-08-06T20:41:24.3588762Z + VALGRIND=ON 2024-08-06T20:41:24.3589175Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *clang9* ]] 2024-08-06T20:41:24.3589523Z + [[ 0 == \1 ]] 2024-08-06T20:41:24.3589808Z + [[ False == \1 ]] 2024-08-06T20:41:24.3590181Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *bazel* ]] 2024-08-06T20:41:24.3592556Z ++ realpath build/custom_test_artifacts 2024-08-06T20:41:24.3601510Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-06T20:41:24.3602274Z + [[ -n '' ]] 2024-08-06T20:41:24.3602571Z + echo 'Environment variables' 2024-08-06T20:41:24.3602931Z Environment variables 2024-08-06T20:41:24.3603262Z + env 2024-08-06T20:41:24.3609962Z INSTALLED_DB=yes 2024-08-06T20:41:24.3610302Z NV_LIBCUBLAS_VERSION=11.11.3.6-1 2024-08-06T20:41:24.3610717Z NVIDIA_VISIBLE_DEVICES=all 2024-08-06T20:41:24.3611066Z NV_NVML_DEV_VERSION=11.8.86-1 2024-08-06T20:41:24.3611601Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:41:24.3612120Z CONTINUE_THROUGH_ERROR=False 2024-08-06T20:41:24.3612531Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.16.2-1+cuda11.8 2024-08-06T20:41:24.3613010Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.16.2-1 2024-08-06T20:41:24.3613432Z BUILD_ENVIRONMENT=linux-focal-cuda11.8-py3.10-gcc9 2024-08-06T20:41:24.3613877Z HOSTNAME=ed694363f944 2024-08-06T20:41:24.3614517Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_9ec21232-fd46-4a04-a696-b627680afb3a 2024-08-06T20:41:24.3615526Z GITHUB_ACTION=__self 2024-08-06T20:41:24.3615859Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-06T20:41:24.3617729Z NVIDIA_REQUIRE_CUDA=cuda>=11.8 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 2024-08-06T20:41:24.3619628Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-11-8=11.11.3.6-1 2024-08-06T20:41:24.3620092Z NV_NVTX_VERSION=11.8.86-1 2024-08-06T20:41:24.3620458Z GITHUB_RUN_NUMBER=233985 2024-08-06T20:41:24.3620762Z TEST_CONFIG=distributed 2024-08-06T20:41:24.3621128Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-06T20:41:24.3621592Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-06T20:41:24.3621924Z NV_CUDA_CUDART_DEV_VERSION=11.8.89-1 2024-08-06T20:41:24.3622348Z NV_LIBCUSPARSE_VERSION=11.7.5.86-1 2024-08-06T20:41:24.3622743Z NV_LIBNPP_VERSION=11.8.0.86-1 2024-08-06T20:41:24.3623107Z GITHUB_TRIGGERING_ACTOR=drisspg 2024-08-06T20:41:24.3623535Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-06T20:41:24.3623983Z GITHUB_REF_TYPE=branch 2024-08-06T20:41:24.3624311Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-06T20:41:24.3624669Z NCCL_VERSION=2.16.2-1 2024-08-06T20:41:24.3625050Z BASE_SHA=1736af7cf736184c356be1bb00f59fb2feea6d7d 2024-08-06T20:41:24.3625826Z XLA_CUDA= 2024-08-06T20:41:24.3626136Z HUGGING_FACE_HUB_TOKEN= 2024-08-06T20:41:24.3626562Z *** 2024-08-06T20:41:24.3626845Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-06T20:41:24.3627237Z GITHUB_REPOSITORY_ID=65600975 2024-08-06T20:41:24.3627603Z GITHUB_ACTIONS=true 2024-08-06T20:41:24.3627923Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:41:24.3628347Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-11-8=11.8.87-1 2024-08-06T20:41:24.3628825Z NV_LIBNPP_PACKAGE=libnpp-11-8=11.8.0.86-1 2024-08-06T20:41:24.3629275Z SHA1=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:41:24.3629726Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-06T20:41:24.3630187Z GITHUB_SHA=bf5bb5a1585a03379137fab341e87c02c77e76cd 2024-08-06T20:41:24.3630830Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/132710/merge 2024-08-06T20:41:24.3631434Z UCC_HOME=/usr 2024-08-06T20:41:24.3631762Z NV_LIBCUBLAS_DEV_VERSION=11.11.3.6-1 2024-08-06T20:41:24.3632134Z VERBOSE_TEST_LOGS=False 2024-08-06T20:41:24.3632480Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-06T20:41:24.3632868Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-11-8 2024-08-06T20:41:24.3633322Z GITHUB_REF=refs/pull/132710/merge 2024-08-06T20:41:24.3633715Z NV_CUDA_CUDART_VERSION=11.8.89-1 2024-08-06T20:41:24.3634067Z SHARD_NUMBER=1 2024-08-06T20:41:24.3634390Z GITHUB_REF_PROTECTED=false 2024-08-06T20:41:24.3634745Z HOME=/var/lib/jenkins 2024-08-06T20:41:24.3635094Z GITHUB_API_URL=https://api.github.com 2024-08-06T20:41:24.3635519Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-06T20:41:24.3636180Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-06T20:41:24.3636618Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-06T20:41:24.3636972Z CUDA_VERSION=11.8.0 2024-08-06T20:41:24.3637334Z NV_LIBCUBLAS_PACKAGE=libcublas-11-8=11.11.3.6-1 2024-08-06T20:41:24.3637741Z NUM_TEST_SHARDS=3 2024-08-06T20:41:24.3638048Z UCX_HOME=/usr 2024-08-06T20:41:24.3638487Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-11-8=11.8.0-1 2024-08-06T20:41:24.3639388Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_9ec21232-fd46-4a04-a696-b627680afb3a 2024-08-06T20:41:24.3640405Z JOB_NAME=linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 1, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:41:24.3641392Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_9ec21232-fd46-4a04-a696-b627680afb3a 2024-08-06T20:41:24.3642330Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-06T20:41:24.3642962Z GITHUB_EVENT_NAME=pull_request 2024-08-06T20:41:24.3643448Z DASHBOARD_TAG= 2024-08-06T20:41:24.3643765Z GITHUB_RUN_ID=10273124344 2024-08-06T20:41:24.3644169Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-11-8=11.8.0.86-1 2024-08-06T20:41:24.3644637Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-11-8 2024-08-06T20:41:24.3645449Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_9ec21232-fd46-4a04-a696-b627680afb3a 2024-08-06T20:41:24.3646268Z GITHUB_ACTOR=drisspg 2024-08-06T20:41:24.3646610Z NV_LIBNPP_DEV_VERSION=11.8.0.86-1 2024-08-06T20:41:24.3646969Z PR_NUMBER=132710 2024-08-06T20:41:24.3647285Z GITHUB_RUN_ATTEMPT=1 2024-08-06T20:41:24.3647607Z VALGRIND=ON 2024-08-06T20:41:24.3647966Z ANACONDA_PYTHON_VERSION=3.10 2024-08-06T20:41:24.3648388Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-06T20:41:24.3648839Z TERM=xterm 2024-08-06T20:41:24.3649161Z NV_LIBCUSPARSE_DEV_VERSION=11.7.5.86-1 2024-08-06T20:41:24.3649546Z INSTALLED_VISION=yes 2024-08-06T20:41:24.3649890Z BRANCH=pull/132710 2024-08-06T20:41:24.3650228Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-06T20:41:24.3650609Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-06T20:41:24.3651028Z CUDA_PATH=/usr/local/cuda 2024-08-06T20:41:24.3651674Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-06T20:41:24.3652364Z GITHUB_SERVER_URL=https://github.com 2024-08-06T20:41:24.3652829Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-06T20:41:24.3653280Z REENABLED_ISSUES= 2024-08-06T20:41:24.3653572Z SHLVL=1 2024-08-06T20:41:24.3653850Z MAX_JOBS=30 2024-08-06T20:41:24.3654154Z NV_CUDA_LIB_VERSION=11.8.0-1 2024-08-06T20:41:24.3654492Z NVARCH=x86_64 2024-08-06T20:41:24.3654801Z GITHUB_ACTOR_ID=32754868 2024-08-06T20:41:24.3655228Z GITHUB_WORKFLOW_SHA=bf5bb5a1585a03379137fab341e87c02c77e76cd 2024-08-06T20:41:24.3655723Z GITHUB_REF_NAME=132710/merge 2024-08-06T20:41:24.3656117Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-11-8 2024-08-06T20:41:24.3656676Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-06T20:41:24.3657228Z GITHUB_JOB=test 2024-08-06T20:41:24.3657578Z NV_LIBNCCL_PACKAGE=libnccl2=2.16.2-1+cuda11.8 2024-08-06T20:41:24.3658076Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-06T20:41:24.3658562Z NO_TEST_TIMEOUT=False 2024-08-06T20:41:24.3658894Z TD_DISTRIBUTED=False 2024-08-06T20:41:24.3659224Z NV_CUDA_NSIGHT_COMPUTE_VERSION=11.8.0-1 2024-08-06T20:41:24.3659641Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-06T20:41:24.3660030Z NV_NVPROF_VERSION=11.8.87-1 2024-08-06T20:41:24.3660371Z GITHUB_RETENTION_DAYS=90 2024-08-06T20:41:24.3660719Z OPENSSL_DIR=/opt/openssl 2024-08-06T20:41:24.3661072Z GITHUB_ACTION_REPOSITORY= 2024-08-06T20:41:24.3661976Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:41:24.3662932Z GITHUB_BASE_REF=gh/drisspg/23/base 2024-08-06T20:41:24.3663335Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-06T20:41:24.3663796Z CI=true 2024-08-06T20:41:24.3664107Z NV_LIBNCCL_PACKAGE_VERSION=2.16.2-1 2024-08-06T20:41:24.3664511Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-06T20:41:24.3664857Z JOB_ID=28428107361 2024-08-06T20:41:24.3665171Z INSTALLED_PROTOBUF=yes 2024-08-06T20:41:24.3665885Z GITHUB_HEAD_REF=gh/drisspg/23/head 2024-08-06T20:41:24.3666276Z GITHUB_ACTION_REF= 2024-08-06T20:41:24.3666647Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-06T20:41:24.3667101Z TEST_SHOWLOCALS=False 2024-08-06T20:41:24.3667413Z GITHUB_WORKFLOW=pull 2024-08-06T20:41:24.3667757Z DEBIAN_FRONTEND=noninteractive 2024-08-06T20:41:24.3668518Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_9ec21232-fd46-4a04-a696-b627680afb3a 2024-08-06T20:41:24.3669266Z NO_TD=False 2024-08-06T20:41:24.3669580Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-06T20:41:24.3669945Z _=/usr/bin/env 2024-08-06T20:41:24.3670241Z + echo 'Testing pytorch' 2024-08-06T20:41:24.3670583Z Testing pytorch 2024-08-06T20:41:24.3670923Z + export LANG=C.UTF-8 2024-08-06T20:41:24.3671372Z + LANG=C.UTF-8 2024-08-06T20:41:24.3671700Z + PR_NUMBER=132710 2024-08-06T20:41:24.3672018Z + [[ distributed == \d\e\f\a\u\l\t ]] 2024-08-06T20:41:24.3672441Z + [[ distributed == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-06T20:41:24.3672910Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *rocm* ]] 2024-08-06T20:41:24.3673350Z + [[ distributed == \s\l\o\w ]] 2024-08-06T20:41:24.3673817Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *slow-gradcheck* ]] 2024-08-06T20:41:24.3674353Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *cuda* ]] 2024-08-06T20:41:24.3674821Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-06T20:41:24.3675276Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-06T20:41:24.3675688Z + [[ distributed == *crossref* ]] 2024-08-06T20:41:24.3676101Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *rocm* ]] 2024-08-06T20:41:24.3676590Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *xpu* ]] 2024-08-06T20:41:24.3677097Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *-bazel-* ]] 2024-08-06T20:41:24.3677568Z + pip_install --user ninja==1.10.2 2024-08-06T20:41:24.3678032Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-06T20:41:24.9035081Z Collecting ninja==1.10.2 2024-08-06T20:41:24.9225867Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-06T20:41:24.9393838Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-06T20:41:26.1419239Z Installing collected packages: ninja 2024-08-06T20:41:26.1516135Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-06T20:41:26.1517184Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-06T20:41:26.1568612Z Successfully installed ninja-1.10.2 2024-08-06T20:41:26.2271990Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:41:26.2273779Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:41:26.2274925Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *aarch64* ]] 2024-08-06T20:41:26.2275373Z + install_tlparse 2024-08-06T20:41:26.2275724Z + pip_install --user tlparse==0.3.7 2024-08-06T20:41:26.2276180Z + pip install --progress-bar off --user tlparse==0.3.7 2024-08-06T20:41:26.7516152Z Collecting tlparse==0.3.7 2024-08-06T20:41:26.7702711Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-08-06T20:41:26.7832623Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-06T20:41:28.0152824Z Installing collected packages: tlparse 2024-08-06T20:41:28.0641586Z Successfully installed tlparse-0.3.7 2024-08-06T20:41:28.1362890Z ++ python -m site --user-base 2024-08-06T20:41:28.1596815Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:41:28.1599059Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *asan* ]] 2024-08-06T20:41:28.1599596Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *-debug* ]] 2024-08-06T20:41:28.1600127Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *-bazel-* ]] 2024-08-06T20:41:28.1600773Z + echo 'We are not in debug mode: linux-focal-cuda11.8-py3.10-gcc9. Expect the assertion to pass' 2024-08-06T20:41:28.1601785Z We are not in debug mode: linux-focal-cuda11.8-py3.10-gcc9. Expect the assertion to pass 2024-08-06T20:41:28.1602905Z + cd test 2024-08-06T20:41:28.1603366Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-06T20:41:30.2987863Z + [[ distributed == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-06T20:41:30.2988760Z + [[ distributed == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-06T20:41:30.2993456Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-06T20:41:30.2993909Z + [[ distributed == *dynamo_eager* ]] 2024-08-06T20:41:30.2994311Z + [[ distributed == *aot_eager* ]] 2024-08-06T20:41:30.2994719Z + [[ distributed == *aot_inductor* ]] 2024-08-06T20:41:30.2995084Z + [[ distributed == *inductor* ]] 2024-08-06T20:41:30.2995454Z + [[ distributed == *dynamic* ]] 2024-08-06T20:41:30.2995840Z + [[ distributed == *cpu* ]] 2024-08-06T20:41:30.2996517Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-06T20:41:30.3036352Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *libtorch* ]] 2024-08-06T20:41:30.3036869Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *-bazel-* ]] 2024-08-06T20:41:30.3039368Z + cd test 2024-08-06T20:41:30.3039726Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-06T20:41:32.2482270Z PyTorch built with: 2024-08-06T20:41:32.2482683Z - GCC 9.4 2024-08-06T20:41:32.2482973Z - C++ Version: 201703 2024-08-06T20:41:32.2483739Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-06T20:41:32.2484613Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-06T20:41:32.2485149Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-06T20:41:32.2485590Z - LAPACK is enabled (usually provided by MKL) 2024-08-06T20:41:32.2486021Z - NNPACK is enabled 2024-08-06T20:41:32.2486357Z - CPU capability usage: AVX2 2024-08-06T20:41:32.2486725Z - CUDA Runtime 11.8 2024-08-06T20:41:32.2487171Z - NVCC architecture flags: -gencode;arch=compute_52,code=sm_52 2024-08-06T20:41:32.2487649Z - CuDNN 90.1 2024-08-06T20:41:32.2487951Z - Magma 2.6.1 2024-08-06T20:41:32.2493447Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-06T20:41:32.2499527Z 2024-08-06T20:41:32.5179251Z + cd test 2024-08-06T20:41:32.5179762Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-06T20:41:34.4078506Z ATen/Parallel: 2024-08-06T20:41:34.4078924Z at::get_num_threads() : 16 2024-08-06T20:41:34.4079314Z at::get_num_interop_threads() : 16 2024-08-06T20:41:34.4079674Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-06T20:41:34.4080049Z omp_get_max_threads() : 16 2024-08-06T20:41:34.4080752Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-06T20:41:34.4081486Z mkl_get_max_threads() : 16 2024-08-06T20:41:34.4081969Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-06T20:41:34.4082526Z std::thread::hardware_concurrency() : 32 2024-08-06T20:41:34.4082946Z Environment variables: 2024-08-06T20:41:34.4083285Z OMP_NUM_THREADS : [not set] 2024-08-06T20:41:34.4083664Z MKL_NUM_THREADS : [not set] 2024-08-06T20:41:34.4084044Z ATen parallel backend: OpenMP 2024-08-06T20:41:34.4084644Z 2024-08-06T20:41:34.6621276Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *aarch64* ]] 2024-08-06T20:41:34.6621814Z + [[ distributed == *backward* ]] 2024-08-06T20:41:34.6622200Z + [[ distributed == *xla* ]] 2024-08-06T20:41:34.6622660Z + [[ distributed == *executorch* ]] 2024-08-06T20:41:34.6622997Z + [[ distributed == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-06T20:41:34.6623461Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *libtorch* ]] 2024-08-06T20:41:34.6623988Z + [[ distributed == distributed ]] 2024-08-06T20:41:34.6624309Z + test_distributed 2024-08-06T20:41:34.6624644Z + echo 'Testing distributed python tests' 2024-08-06T20:41:34.6625144Z Testing distributed python tests 2024-08-06T20:41:34.6626040Z + python test/run_test.py --distributed-tests --shard 1 3 --verbose 2024-08-06T20:41:34.7915452Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-06T20:41:34.7916464Z import pkg_resources 2024-08-06T20:41:39.2956473Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-06T20:41:39.3594591Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-06T20:41:39.4354275Z Ignoring disabled issues: [''] 2024-08-06T20:41:39.4492904Z Found test times from artifacts 2024-08-06T20:41:39.5018185Z Found test times from artifacts 2024-08-06T20:41:39.5035385Z Running 25% of tests based on TD 2024-08-06T20:41:39.5161727Z Running parallel tests on 2 processes 2024-08-06T20:41:39.5162613Z Name: tests to run (est. time: 5.68min) 2024-08-06T20:41:39.5163016Z Serial tests (14): 2024-08-06T20:41:39.5163435Z distributed/test_c10d_functional_native 1/1 2024-08-06T20:41:39.5163860Z distributed/_tensor/test_dtensor_ops 1/1 2024-08-06T20:41:39.5164342Z distributed/fsdp/test_fsdp_fx 1/1 2024-08-06T20:41:39.5164866Z distributed/_tensor/experimental/test_register_sharding 1/1 2024-08-06T20:41:39.5165781Z distributed/_tensor/test_api 1/1 2024-08-06T20:41:39.5166122Z distributed/_tensor/test_random_ops 1/1 2024-08-06T20:41:39.5166604Z distributed/_tensor/test_init 1/1 2024-08-06T20:41:39.5167098Z distributed/_tensor/debug/test_comm_mode_features 1/1 2024-08-06T20:41:39.5167567Z distributed/_tensor/test_math_ops 1/1 2024-08-06T20:41:39.5167976Z distributed/checkpoint/e2e/test_fsdp_ep 1/1 2024-08-06T20:41:39.5168457Z distributed/test_compute_comm_reordering 1/1 2024-08-06T20:41:39.5168903Z distributed/_tensor/test_matrix_ops 1/1 2024-08-06T20:41:39.5169390Z distributed/_tensor/test_redistribute 1/1 2024-08-06T20:41:39.5169825Z distributed/checkpoint/test_format_utils 1/1 2024-08-06T20:41:39.5170242Z Parallel tests (0): 2024-08-06T20:41:39.5170684Z Name: excluded (est. time: 81.36min) 2024-08-06T20:41:39.5171014Z Serial tests (55): 2024-08-06T20:41:39.5171630Z distributed/test_distributed_spawn 1/10 2024-08-06T20:41:39.5172077Z distributed/test_distributed_spawn 4/10 2024-08-06T20:41:39.5172524Z distributed/test_distributed_spawn 7/10 2024-08-06T20:41:39.5173015Z distributed/test_distributed_spawn 10/10 2024-08-06T20:41:39.5173391Z distributed/fsdp/test_fsdp_core 2/2 2024-08-06T20:41:39.5173850Z distributed/rpc/cuda/test_tensorpipe_agent 1/2 2024-08-06T20:41:39.5174398Z distributed/fsdp/test_fsdp_optim_state 1/1 2024-08-06T20:41:39.5174848Z distributed/_shard/sharded_tensor/test_sharded_tensor 1/1 2024-08-06T20:41:39.5175502Z distributed/optim/test_zero_redundancy_optimizer 1/1 2024-08-06T20:41:39.5175952Z distributed/_composable/fsdp/test_fully_shard_comm 1/1 2024-08-06T20:41:39.5176492Z distributed/fsdp/test_fsdp_sharded_grad_scaler 1/1 2024-08-06T20:41:39.5176977Z distributed/pipelining/test_schedule 1/1 2024-08-06T20:41:39.5177398Z distributed/test_pg_wrapper 1/1 2024-08-06T20:41:39.5177901Z distributed/fsdp/test_fsdp_misc 1/1 2024-08-06T20:41:39.5178404Z distributed/_tensor/test_optimizers 1/1 2024-08-06T20:41:39.5178902Z distributed/checkpoint/e2e/test_e2e_save_and_load 1/1 2024-08-06T20:41:39.5179376Z distributed/test_store 1/1 2024-08-06T20:41:39.5179761Z distributed/test_c10d_spawn_nccl 1/1 2024-08-06T20:41:39.5180225Z distributed/tensor/parallel/test_fsdp_2d_parallel 1/1 2024-08-06T20:41:39.5180772Z distributed/tensor/parallel/test_parallelize_api 1/1 2024-08-06T20:41:39.5181340Z distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 2024-08-06T20:41:39.5181858Z distributed/fsdp/test_fsdp_exec_order 1/1 2024-08-06T20:41:39.5182303Z distributed/fsdp/test_fsdp_fine_tune 1/1 2024-08-06T20:41:39.5182900Z distributed/algorithms/ddp_comm_hooks/test_ddp_hooks 1/1 2024-08-06T20:41:39.5183304Z distributed/_tensor/test_view_ops 1/1 2024-08-06T20:41:39.5183795Z distributed/checkpoint/test_file_system_checkpoint 1/1 2024-08-06T20:41:39.5184287Z distributed/_tensor/test_utils 1/1 2024-08-06T20:41:39.5184776Z distributed/_composable/fsdp/test_fully_shard_frozen 1/1 2024-08-06T20:41:39.5185355Z distributed/test_c10d_logger 1/1 2024-08-06T20:41:39.5186045Z distributed/fsdp/test_fsdp_apply 1/1 2024-08-06T20:41:39.5186586Z distributed/elastic/timer/local_timer_example 1/1 2024-08-06T20:41:39.5187019Z distributed/_shard/sharded_tensor/ops/test_init 1/1 2024-08-06T20:41:39.5187691Z distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_ 1/1 2024-08-06T20:41:39.5188209Z distributed/_composable/fsdp/test_fully_shard_compile 1/1 2024-08-06T20:41:39.5188723Z distributed/fsdp/test_fsdp_pure_fp16 1/1 2024-08-06T20:41:39.5189164Z distributed/fsdp/test_fsdp_input 1/1 2024-08-06T20:41:39.5189612Z distributed/checkpoint/test_fsdp_model_state 1/1 2024-08-06T20:41:39.5190141Z distributed/_shard/test_sharder 1/1 2024-08-06T20:41:39.5190561Z distributed/_composable/fsdp/test_fully_shard_init 1/1 2024-08-06T20:41:39.5191065Z distributed/elastic/test_control_plane 1/1 2024-08-06T20:41:39.5191623Z distributed/checkpoint/test_fsdp_tp_checkpoint_conversion 1/1 2024-08-06T20:41:39.5192308Z distributed/_composable/fully_shard/test_fully_shard_mixed_precision 1/1 2024-08-06T20:41:39.5192898Z distributed/checkpoint/e2e/test_pipeline 1/1 2024-08-06T20:41:39.5193453Z distributed/_composable/fully_shard/test_fully_shard_util 1/1 2024-08-06T20:41:39.5193984Z distributed/pipelining/test_stage 1/1 2024-08-06T20:41:39.5194394Z distributed/test_fake_pg 1/1 2024-08-06T20:41:39.5194802Z distributed/test_multi_threaded_pg 1/1 2024-08-06T20:41:39.5195225Z distributed/test_nccl 1/1 2024-08-06T20:41:39.5195616Z distributed/_tensor/debug/test_comm_mode 1/1 2024-08-06T20:41:39.5196110Z distributed/checkpoint/test_compatibility 1/1 2024-08-06T20:41:39.5196590Z distributed/elastic/utils/logging_test 1/1 2024-08-06T20:41:39.5197029Z distributed/nn/jit/test_instantiator 1/1 2024-08-06T20:41:39.5197691Z distributed/fsdp/test_distributed_checkpoint 1/1 2024-08-06T20:41:39.5198245Z distributed/_composable/fsdp/test_fully_shard_logging 1/1 2024-08-06T20:41:39.5198821Z distributed/rpc/test_tensorpipe_agent 1/1 2024-08-06T20:41:39.5199180Z Parallel tests (0): 2024-08-06T20:41:39.5199657Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-06T20:41:39.5233325Z Running distributed/test_c10d_functional_native 1/1 ... [2024-08-06 20:41:39.522946] 2024-08-06T20:41:39.5237053Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/test_c10d_functional_native.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:41:39.523331] 2024-08-06T20:42:43.8224673Z 2024-08-06T20:42:43.8226650Z distributed/test_c10d_functional_native 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_functional_native_1.1_2ebe2b2be1095a10_.log 2024-08-06T20:42:43.8239568Z Running 25 items in this shard: test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_gather_into_tensor_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_gather_into_tensor_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_coalesced_, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_single_, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_to_all_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_broadcast, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_inductor_dtypeview_memory_leak, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_py_work, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_reduce_scatter_tensor_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_threading, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_unwaited, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_to_all_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_broadcast, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_inplace_op_on_view, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reduce_scatter_tensor_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reuse_buffer_after_inplace_collective, test/distributed/test_c10d_functional_native.py::CompileTest::test_ranks_and_tag 2024-08-06T20:42:43.8250496Z 2024-08-06T20:42:43.8250849Z Running distributed/_tensor/test_dtensor_ops 1/1 ... [2024-08-06 20:42:43.822985] 2024-08-06T20:42:43.8252310Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_dtensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:42:43.823422] 2024-08-06T20:42:48.6854032Z 2024-08-06T20:42:48.6855559Z distributed/_tensor/test_dtensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_dtensor_ops_1.1_a728983e33e9c5ea_.log 2024-08-06T20:42:48.6857062Z Running 0 items in this shard: 2024-08-06T20:42:48.6857227Z 2024-08-06T20:42:48.6860395Z Running distributed/fsdp/test_fsdp_fx 1/1 ... [2024-08-06 20:42:48.685751] 2024-08-06T20:42:48.6865245Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/fsdp/test_fsdp_fx.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:42:48.686187] 2024-08-06T20:42:52.6084493Z 2024-08-06T20:42:52.6085667Z distributed/fsdp/test_fsdp_fx 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_fx_1.1_6daa2efcf1b810b5_.log 2024-08-06T20:42:52.6087072Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_fx.py::TestSymbolicTracing::test_symbolic_tracing_outputs 2024-08-06T20:42:52.6087719Z 2024-08-06T20:42:52.6090944Z Running distributed/_tensor/experimental/test_register_sharding 1/1 ... [2024-08-06 20:42:52.608771] 2024-08-06T20:42:52.6107380Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/experimental/test_register_sharding.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:42:52.610343] 2024-08-06T20:43:05.4512061Z 2024-08-06T20:43:05.4513661Z distributed/_tensor/experimental/test_register_sharding 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_register_sharding_1.1_de3ed1c0bf0aacba_.log 2024-08-06T20:43:05.4515952Z Running 2 items in this shard: test/distributed/_tensor/experimental/test_register_sharding.py::TestRegisterSharding::test_argmax, test/distributed/_tensor/experimental/test_register_sharding.py::TestRegisterSharding::test_softmax_fwd 2024-08-06T20:43:05.4517169Z 2024-08-06T20:43:05.4517472Z Running distributed/_tensor/test_api 1/1 ... [2024-08-06 20:43:05.451438] 2024-08-06T20:43:05.4521914Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:43:05.451846] 2024-08-06T20:43:40.0419752Z 2024-08-06T20:43:40.0423621Z distributed/_tensor/test_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_api_1.1_5676bad0d47cc01f_.log 2024-08-06T20:43:40.0428070Z Running 8 items in this shard: test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module_casting, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module_input_fn_output_fn, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module_input_fn_output_fn_warning, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module_meta, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_tensor, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_tensor_errors, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_tensor_uneven_sharding 2024-08-06T20:43:40.0431451Z 2024-08-06T20:43:40.0431785Z Running distributed/_tensor/test_random_ops 1/1 ... [2024-08-06 20:43:40.042256] 2024-08-06T20:43:40.0433243Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_random_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:43:40.042716] 2024-08-06T20:44:13.7766850Z 2024-08-06T20:44:13.7768642Z distributed/_tensor/test_random_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_random_ops_1.1_6b7f634b2f4db62f_.log 2024-08-06T20:44:13.7773023Z Running 7 items in this shard: test/distributed/_tensor/test_random_ops.py::DistTensorRandomInitTest::test_init_ops, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_dropout_1d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_rand_1d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_uniform_2d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_manual_seed, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_meta_tensor_init, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_rng_tracker_init 2024-08-06T20:44:13.7789644Z 2024-08-06T20:44:13.7790042Z Running distributed/_tensor/test_init 1/1 ... [2024-08-06 20:44:13.776913] 2024-08-06T20:44:13.7791734Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:44:13.777363] 2024-08-06T20:44:44.8104678Z 2024-08-06T20:44:44.8108093Z distributed/_tensor/test_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_init_1.1_82fde18823aad1ff_.log 2024-08-06T20:44:44.8112072Z Running 7 items in this shard: test/distributed/_tensor/test_init.py::DTensorInitOpsTest::test_init_ops, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_empty, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_full, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_ones, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_zeros, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_zeros_full_mesh, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_zeros_submesh 2024-08-06T20:44:44.8115442Z 2024-08-06T20:44:44.8115831Z Running distributed/_tensor/debug/test_comm_mode_features 1/1 ... [2024-08-06 20:44:44.810812] 2024-08-06T20:44:44.8117440Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/debug/test_comm_mode_features.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:44:44.811220] 2024-08-06T20:45:07.5221535Z 2024-08-06T20:45:07.5222942Z distributed/_tensor/debug/test_comm_mode_features 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.debug.test_comm_mode_features_1.1_bd05722d18b5c1b9_.log 2024-08-06T20:45:07.5226520Z Running 4 items in this shard: test/distributed/_tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_MLPStacked_distributed_sharding_display, test/distributed/_tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_MLP_distributed_sharding_display, test/distributed/_tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_MLP_module_tracing, test/distributed/_tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_transformer_module_tracing 2024-08-06T20:45:07.5228933Z 2024-08-06T20:45:07.5229245Z Running distributed/_tensor/test_math_ops 1/1 ... [2024-08-06 20:45:07.522387] 2024-08-06T20:45:07.5231828Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_math_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:45:07.522830] 2024-08-06T20:46:06.0573973Z 2024-08-06T20:46:06.0575972Z distributed/_tensor/test_math_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_math_ops_1.1_07bce20189e09810_.log 2024-08-06T20:46:06.0582462Z Running 11 items in this shard: test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_foreach_norm, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_layer_norm_bwd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_layer_norm_fwd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_linear_op_reductions, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_mean, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_nll_loss_and_cross_entropy, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_shard0_svd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_shard_math_ops, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_softmax_fwd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_softmax_with_bwd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_topk 2024-08-06T20:46:06.0590016Z 2024-08-06T20:46:06.0590406Z Running distributed/checkpoint/e2e/test_fsdp_ep 1/1 ... [2024-08-06 20:46:06.057688] 2024-08-06T20:46:06.0592159Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/checkpoint/e2e/test_fsdp_ep.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:46:06.058140] 2024-08-06T20:46:13.5418562Z 2024-08-06T20:46:13.5419787Z distributed/checkpoint/e2e/test_fsdp_ep 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_fsdp_ep_1.1_14130551e8d85e3e_.log 2024-08-06T20:46:13.5421237Z Running 1 items in this shard: test/distributed/checkpoint/e2e/test_fsdp_ep.py::TestFSDPWithEP::test_e2e 2024-08-06T20:46:13.5421827Z 2024-08-06T20:46:13.5424154Z Running distributed/test_compute_comm_reordering 1/1 ... [2024-08-06 20:46:13.542082] 2024-08-06T20:46:13.5428986Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/test_compute_comm_reordering.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:46:13.542500] 2024-08-06T20:46:21.6274822Z 2024-08-06T20:46:21.6276042Z distributed/test_compute_comm_reordering 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_compute_comm_reordering_1.1_e9f5996b7610929d_.log 2024-08-06T20:46:21.6281137Z Running 7 items in this shard: test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_grouped_scheduler_node, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_nccl_heuristics, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_raise_comms, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_reorder_compute_for_overlap, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_reorder_compute_for_overlap_custom_runtime_estimation, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_sink_waits, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_sink_waits_raise_comms 2024-08-06T20:46:21.6285239Z 2024-08-06T20:46:21.6285565Z Running distributed/_tensor/test_matrix_ops 1/1 ... [2024-08-06 20:46:21.627733] 2024-08-06T20:46:21.6287010Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_matrix_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:46:21.628173] 2024-08-06T20:47:06.7416060Z 2024-08-06T20:47:06.7419431Z distributed/_tensor/test_matrix_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_matrix_ops_1.1_fdb63b4266d6568a_.log 2024-08-06T20:47:06.7423798Z Running 9 items in this shard: test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_addmm, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_addmm_auto_redistribute, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_addmm_empty_operand, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_baddbmm, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_bmm, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_mm, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_scaled_dot_product_attention, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_t, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_t_partial 2024-08-06T20:47:06.7427928Z 2024-08-06T20:47:06.7428294Z Running distributed/_tensor/test_redistribute 1/1 ... [2024-08-06 20:47:06.741904] 2024-08-06T20:47:06.7429956Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_redistribute.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:47:06.742347] 2024-08-06T20:47:58.4256823Z 2024-08-06T20:47:58.4257993Z distributed/_tensor/test_redistribute 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_redistribute_1.1_cf6070e796ea39d7_.log 2024-08-06T20:47:58.4264907Z Running 12 items in this shard: test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_partial_to_replicate_forward_backward, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_partial_to_shard, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_redistribute_negative_shard_dim, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_redistribute_shard_dim_change, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_redistribute_uneven_sharding, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_replicate_to_local_partial_grad, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_replicate_to_partial, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_replicate_to_replicate_forward_backward, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_replicate_to_shard_forward_backward, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_shard_dim_alltoall, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_shard_to_replicate_forward_backward, test/distributed/_tensor/test_redistribute.py::MultiDimRedistributeTest::test_multi_dim_mesh 2024-08-06T20:47:58.4271653Z 2024-08-06T20:47:58.4272023Z Running distributed/checkpoint/test_format_utils 1/1 ... [2024-08-06 20:47:58.426164] 2024-08-06T20:47:58.4273572Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/checkpoint/test_format_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:47:58.426677] 2024-08-06T20:48:14.2739913Z 2024-08-06T20:48:14.2741306Z distributed/checkpoint/test_format_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_format_utils_1.1_1137081102ea5d7c_.log 2024-08-06T20:48:14.2743698Z Running 3 items in this shard: test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_dcp_to_torch_save, test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_online_torch_save_to_dcp, test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_torch_save_to_dcp 2024-08-06T20:48:15.2069763Z 2024-08-06T20:48:15.2069772Z 2024-08-06T20:48:15.2070234Z real 6m40.545s 2024-08-06T20:48:15.2070610Z user 15m26.466s 2024-08-06T20:48:15.2070896Z sys 5m28.785s 2024-08-06T20:48:15.2071208Z + assert_git_not_dirty 2024-08-06T20:48:15.2071629Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *rocm* ]] 2024-08-06T20:48:15.2072598Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *xla* ]] 2024-08-06T20:48:15.2076389Z ++ git status --porcelain 2024-08-06T20:48:15.2076685Z ++ grep -v '?? third_party' 2024-08-06T20:48:18.0958135Z ++ true 2024-08-06T20:48:18.0959275Z + git_status= 2024-08-06T20:48:18.0959592Z + [[ -n '' ]] 2024-08-06T20:48:18.0960207Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *cuda* ]] 2024-08-06T20:48:18.0960693Z + [[ 1 == 1 ]] 2024-08-06T20:48:18.0961001Z + echo 'Testing distributed C++ tests' 2024-08-06T20:48:18.0961418Z Testing distributed C++ tests 2024-08-06T20:48:18.0965022Z + ln -sf /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_cuda_linalg.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_global_deps.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_python.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorchbind_test.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-06T20:48:18.0976770Z + ln -sf /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libc10.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libc10_cuda.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libc10d_cuda_test.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-06T20:48:18.0986817Z + export CPP_TESTS_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-06T20:48:18.0987588Z + CPP_TESTS_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-06T20:48:18.0988247Z + python test/run_test.py --cpp --verbose -i cpp/FileStoreTest 2024-08-06T20:48:18.2284848Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-06T20:48:18.2285884Z import pkg_resources 2024-08-06T20:48:22.7752111Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-06T20:48:22.7753518Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-06T20:48:22.7889249Z Found test times from artifacts 2024-08-06T20:48:22.8423489Z Found test times from artifacts 2024-08-06T20:48:22.8440863Z Running 25% of tests based on TD 2024-08-06T20:48:22.8444248Z Running parallel tests on 2 processes 2024-08-06T20:48:22.8444628Z Name: tests to run (est. time: 0.0min) 2024-08-06T20:48:22.8445021Z Serial tests (1): 2024-08-06T20:48:22.8445338Z cpp/FileStoreTest 1/1 2024-08-06T20:48:22.8445696Z Parallel tests (0): 2024-08-06T20:48:22.8446023Z Name: excluded (est. time: 0.0min) 2024-08-06T20:48:22.8446405Z Serial tests (0): 2024-08-06T20:48:22.8446720Z Parallel tests (0): 2024-08-06T20:48:22.8447224Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-06T20:48:22.8552684Z Running cpp/FileStoreTest 1/1 ... [2024-08-06 20:48:22.854940] 2024-08-06T20:48:22.8560169Z Executing ['pytest', '/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin/FileStoreTest', '-v', '-vv', '-rfEX', '-n', '2', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-20db868e5ab574d7.xml', '-x', '--reruns=2'] ... [2024-08-06 20:48:22.855633] 2024-08-06T20:48:26.6786181Z 2024-08-06T20:48:26.6788277Z cpp/FileStoreTest 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.FileStoreTest_1.1_02dca750a74550ea_.log 2024-08-06T20:48:26.6789239Z 2024-08-06T20:48:27.6103568Z + python test/run_test.py --cpp --verbose -i cpp/HashStoreTest 2024-08-06T20:48:27.7401428Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-06T20:48:27.7402847Z import pkg_resources 2024-08-06T20:48:32.2562875Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-06T20:48:32.2564255Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-06T20:48:32.2699801Z Found test times from artifacts 2024-08-06T20:48:32.3228823Z Found test times from artifacts 2024-08-06T20:48:32.3245727Z Running 25% of tests based on TD 2024-08-06T20:48:32.3249595Z Running parallel tests on 2 processes 2024-08-06T20:48:32.3250098Z Name: tests to run (est. time: 0.0min) 2024-08-06T20:48:32.3250475Z Serial tests (1): 2024-08-06T20:48:32.3250805Z cpp/HashStoreTest 1/1 2024-08-06T20:48:32.3251170Z Parallel tests (0): 2024-08-06T20:48:32.3251504Z Name: excluded (est. time: 0.0min) 2024-08-06T20:48:32.3251888Z Serial tests (0): 2024-08-06T20:48:32.3252209Z Parallel tests (0): 2024-08-06T20:48:32.3253017Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-06T20:48:32.3347791Z Running cpp/HashStoreTest 1/1 ... [2024-08-06 20:48:32.334436] 2024-08-06T20:48:32.3354702Z Executing ['pytest', '/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin/HashStoreTest', '-v', '-vv', '-rfEX', '-n', '2', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-a0778e6ba22228b8.xml', '-x', '--reruns=2'] ... [2024-08-06 20:48:32.335065] 2024-08-06T20:48:35.7574761Z 2024-08-06T20:48:35.7576322Z cpp/HashStoreTest 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.HashStoreTest_1.1_8232ff1aa92ec82a_.log 2024-08-06T20:48:35.7577207Z 2024-08-06T20:48:36.7179866Z + python test/run_test.py --cpp --verbose -i cpp/TCPStoreTest 2024-08-06T20:48:36.8511613Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-06T20:48:36.8512695Z import pkg_resources 2024-08-06T20:48:41.3702229Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-06T20:48:41.3703637Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-06T20:48:41.3839574Z Found test times from artifacts 2024-08-06T20:48:41.4375075Z Found test times from artifacts 2024-08-06T20:48:41.4392129Z Running 25% of tests based on TD 2024-08-06T20:48:41.4395623Z Running parallel tests on 2 processes 2024-08-06T20:48:41.4396005Z Name: tests to run (est. time: 0.0min) 2024-08-06T20:48:41.4396393Z Serial tests (1): 2024-08-06T20:48:41.4396754Z cpp/TCPStoreTest 1/1 2024-08-06T20:48:41.4397110Z Parallel tests (0): 2024-08-06T20:48:41.4397447Z Name: excluded (est. time: 0.0min) 2024-08-06T20:48:41.4397836Z Serial tests (0): 2024-08-06T20:48:41.4398163Z Parallel tests (0): 2024-08-06T20:48:41.4398684Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-06T20:48:41.4501571Z Running cpp/TCPStoreTest 1/1 ... [2024-08-06 20:48:41.449840] 2024-08-06T20:48:41.4508352Z Executing ['pytest', '/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin/TCPStoreTest', '-v', '-vv', '-rfEX', '-n', '2', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-aa41b9db8fd8df99.xml', '-x', '--reruns=2'] ... [2024-08-06 20:48:41.450457] 2024-08-06T20:48:57.5003930Z 2024-08-06T20:48:57.5005314Z cpp/TCPStoreTest 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.TCPStoreTest_1.1_ae2d119ee9ea84db_.log 2024-08-06T20:48:57.5006071Z 2024-08-06T20:48:58.4291611Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *cuda* ]] 2024-08-06T20:48:58.4294261Z ++ command -v mpiexec 2024-08-06T20:48:58.4298319Z + MPIEXEC=/usr/bin/mpiexec 2024-08-06T20:48:58.4298613Z + [[ -n /usr/bin/mpiexec ]] 2024-08-06T20:48:58.4299781Z + MPICMD='/usr/bin/mpiexec -np 2 /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin/ProcessGroupMPITest' 2024-08-06T20:48:58.4300846Z + eval '/usr/bin/mpiexec -np 2 /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin/ProcessGroupMPITest' 2024-08-06T20:48:58.4301855Z ++ /usr/bin/mpiexec -np 2 /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin/ProcessGroupMPITest 2024-08-06T20:49:01.4846168Z Test successful 2024-08-06T20:49:01.4895289Z Test successful 2024-08-06T20:49:01.6491627Z + python test/run_test.py --cpp --verbose -i cpp/ProcessGroupGlooTest 2024-08-06T20:49:01.7829075Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-06T20:49:01.7830094Z import pkg_resources 2024-08-06T20:49:06.3440708Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-06T20:49:06.3442477Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-06T20:49:06.3576648Z Found test times from artifacts 2024-08-06T20:49:06.4106276Z Found test times from artifacts 2024-08-06T20:49:06.4123029Z Running 25% of tests based on TD 2024-08-06T20:49:06.4126323Z Running parallel tests on 2 processes 2024-08-06T20:49:06.4126688Z Name: tests to run (est. time: 0.0min) 2024-08-06T20:49:06.4127075Z Serial tests (1): 2024-08-06T20:49:06.4127462Z cpp/ProcessGroupGlooTest 1/1 2024-08-06T20:49:06.4127832Z Parallel tests (0): 2024-08-06T20:49:06.4128199Z Name: excluded (est. time: 0.0min) 2024-08-06T20:49:06.4128551Z Serial tests (0): 2024-08-06T20:49:06.4128877Z Parallel tests (0): 2024-08-06T20:49:06.4129341Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-06T20:49:06.4228814Z Running cpp/ProcessGroupGlooTest 1/1 ... [2024-08-06 20:49:06.422466] 2024-08-06T20:49:06.4235086Z Executing ['pytest', '/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin/ProcessGroupGlooTest', '-v', '-vv', '-rfEX', '-n', '2', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-e8e4e32c532ea93c.xml', '-x', '--reruns=2'] ... [2024-08-06 20:49:06.423067] 2024-08-06T20:49:15.6080289Z 2024-08-06T20:49:15.6081624Z cpp/ProcessGroupGlooTest 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.ProcessGroupGlooTest_1.1_a144a89a7ac8cf70_.log 2024-08-06T20:49:15.6082450Z 2024-08-06T20:49:16.5812036Z + python test/run_test.py --cpp --verbose -i cpp/ProcessGroupNCCLTest 2024-08-06T20:49:16.7139903Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-06T20:49:16.7140946Z import pkg_resources 2024-08-06T20:49:21.2903916Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-06T20:49:21.2905365Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-06T20:49:21.3041095Z Found test times from artifacts 2024-08-06T20:49:21.3576564Z Found test times from artifacts 2024-08-06T20:49:21.3593769Z Running 25% of tests based on TD 2024-08-06T20:49:21.3597228Z Running parallel tests on 2 processes 2024-08-06T20:49:21.3597686Z Name: tests to run (est. time: 0.0min) 2024-08-06T20:49:21.3598058Z Serial tests (1): 2024-08-06T20:49:21.3598413Z cpp/ProcessGroupNCCLTest 1/1 2024-08-06T20:49:21.3598815Z Parallel tests (0): 2024-08-06T20:49:21.3599149Z Name: excluded (est. time: 0.0min) 2024-08-06T20:49:21.3599528Z Serial tests (0): 2024-08-06T20:49:21.3599849Z Parallel tests (0): 2024-08-06T20:49:21.3600310Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-06T20:49:21.3701272Z Running cpp/ProcessGroupNCCLTest 1/1 ... [2024-08-06 20:49:21.369791] 2024-08-06T20:49:21.3708532Z Executing ['pytest', '/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin/ProcessGroupNCCLTest', '-v', '-vv', '-rfEX', '-n', '2', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-75741edf2ca83522.xml', '-x', '--reruns=2'] ... [2024-08-06 20:49:21.370442] 2024-08-06T20:49:47.9410356Z 2024-08-06T20:49:47.9411367Z cpp/ProcessGroupNCCLTest 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.ProcessGroupNCCLTest_1.1_068d1fb6a9c25506_.log 2024-08-06T20:49:47.9412235Z 2024-08-06T20:49:48.9033716Z + python test/run_test.py --cpp --verbose -i cpp/ProcessGroupNCCLErrorsTest 2024-08-06T20:49:49.0362641Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-06T20:49:49.0363660Z import pkg_resources 2024-08-06T20:49:53.5722355Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-06T20:49:53.5725234Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-06T20:49:53.5861335Z Found test times from artifacts 2024-08-06T20:49:53.6396288Z Found test times from artifacts 2024-08-06T20:49:53.6414354Z Running 25% of tests based on TD 2024-08-06T20:49:53.6417566Z Running parallel tests on 2 processes 2024-08-06T20:49:53.6418351Z Name: tests to run (est. time: 0.0min) 2024-08-06T20:49:53.6419070Z Serial tests (1): 2024-08-06T20:49:53.6419750Z cpp/ProcessGroupNCCLErrorsTest 1/1 2024-08-06T20:49:53.6420628Z Parallel tests (0): 2024-08-06T20:49:53.6421252Z Name: excluded (est. time: 0.0min) 2024-08-06T20:49:53.6421903Z Serial tests (0): 2024-08-06T20:49:53.6422480Z Parallel tests (0): 2024-08-06T20:49:53.6423345Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-06T20:49:53.6527753Z Running cpp/ProcessGroupNCCLErrorsTest 1/1 ... [2024-08-06 20:49:53.652319] 2024-08-06T20:49:53.6536051Z Executing ['pytest', '/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin/ProcessGroupNCCLErrorsTest', '-v', '-vv', '-rfEX', '-n', '2', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-a9311b904878955f.xml', '-x', '--reruns=2'] ... [2024-08-06 20:49:53.653041] 2024-08-06T20:50:05.9948883Z 2024-08-06T20:50:05.9951248Z cpp/ProcessGroupNCCLErrorsTest 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.ProcessGroupNCCLErrorsTest_1.1_619e5d07f8a75c4a_.log 2024-08-06T20:50:05.9952179Z 2024-08-06T20:50:06.9405723Z + [[ 1 == 1 ]] 2024-08-06T20:50:06.9406379Z + test_rpc 2024-08-06T20:50:06.9406837Z + echo 'Testing RPC C++ tests' 2024-08-06T20:50:06.9407241Z Testing RPC C++ tests 2024-08-06T20:50:06.9410457Z + ln -sf /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_cuda_linalg.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_global_deps.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_python.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorchbind_test.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-06T20:50:06.9423975Z + ln -sf /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libc10.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libc10_cuda.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libc10d_cuda_test.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-06T20:50:06.9436733Z + CPP_TESTS_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-06T20:50:06.9437791Z + python test/run_test.py --cpp --verbose -i cpp/test_cpp_rpc 2024-08-06T20:50:07.0731760Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-06T20:50:07.0732804Z import pkg_resources 2024-08-06T20:50:11.4779884Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-06T20:50:11.4781272Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-06T20:50:11.4915883Z Found test times from artifacts 2024-08-06T20:50:11.5444425Z Found test times from artifacts 2024-08-06T20:50:11.5460459Z Running 25% of tests based on TD 2024-08-06T20:50:11.5463876Z Running parallel tests on 2 processes 2024-08-06T20:50:11.5464344Z Name: tests to run (est. time: 0.0min) 2024-08-06T20:50:11.5464728Z Serial tests (1): 2024-08-06T20:50:11.5465056Z cpp/test_cpp_rpc 1/1 2024-08-06T20:50:11.5465924Z Parallel tests (0): 2024-08-06T20:50:11.5466308Z Name: excluded (est. time: 0.0min) 2024-08-06T20:50:11.5466690Z Serial tests (0): 2024-08-06T20:50:11.5467011Z Parallel tests (0): 2024-08-06T20:50:11.5467465Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-06T20:50:11.5566162Z Running cpp/test_cpp_rpc 1/1 ... [2024-08-06 20:50:11.556276] 2024-08-06T20:50:11.5572934Z Executing ['pytest', '/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin/test_cpp_rpc', '-v', '-vv', '-rfEX', '-n', '2', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-fc284a177e47c83a.xml', '-x', '--reruns=2'] ... [2024-08-06 20:50:11.556855] 2024-08-06T20:50:17.5345895Z 2024-08-06T20:50:17.5347109Z cpp/test_cpp_rpc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.test_cpp_rpc_1.1_cb81b5479c8e4136_.log 2024-08-06T20:50:17.5347833Z 2024-08-06T20:50:18.4850927Z + cleanup_workspace 2024-08-06T20:50:18.4851694Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-06T20:50:18.4852633Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-06T20:50:18.4853389Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-06T20:50:18.4853951Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-06T20:50:18.4854610Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-06T20:50:18.4855357Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-06T20:50:18.4855955Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-06T20:50:19.0170471Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-06T20:50:19.0170998Z cat test/**/*_toprint.log || true 2024-08-06T20:50:19.0179066Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:50:19.0179547Z env: 2024-08-06T20:50:19.0179848Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:19.0180285Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:19.0180971Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:19.0181574Z ##[endgroup] 2024-08-06T20:50:19.0252556Z cat: test/**/*_toprint.log: No such file or directory 2024-08-06T20:50:19.0297340Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-06T20:50:19.0297816Z kill "$MONITOR_SCRIPT_PID" 2024-08-06T20:50:19.0305156Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:50:19.0306138Z env: 2024-08-06T20:50:19.0306427Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:19.0306861Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:19.0307554Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:19.0308161Z MONITOR_SCRIPT_PID: 10130 2024-08-06T20:50:19.0308516Z ##[endgroup] 2024-08-06T20:50:19.0474540Z Prepare all required actions 2024-08-06T20:50:19.0475286Z Getting action download info 2024-08-06T20:50:19.2100049Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-06T20:50:19.3700957Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-06T20:50:19.3701422Z with: 2024-08-06T20:50:19.3701878Z file-suffix: test-distributed-1-3-linux.8xlarge.nvidia.gpu_28428107361 2024-08-06T20:50:19.3702456Z s3-bucket: gha-artifacts 2024-08-06T20:50:19.3702808Z env: 2024-08-06T20:50:19.3703087Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:19.3703522Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:19.3704214Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:19.3704815Z ##[endgroup] 2024-08-06T20:50:19.3767363Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-06T20:50:19.3768010Z # Remove any previous test jsons if they exist 2024-08-06T20:50:19.3768479Z rm -f test-jsons-*.zip 2024-08-06T20:50:19.3768986Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-06T20:50:19.3776601Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:50:19.3777067Z env: 2024-08-06T20:50:19.3777369Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:19.3777803Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:19.3778472Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:19.3779261Z FILE_SUFFIX: test-distributed-1-3-linux.8xlarge.nvidia.gpu_28428107361 2024-08-06T20:50:19.3779813Z ##[endgroup] 2024-08-06T20:50:19.4021923Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-06T20:50:19.4057342Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-06T20:50:19.4057996Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-06T20:50:19.4065419Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-06T20:50:19.4068110Z adding: test/test-reports/td_exclusions-fb546569775c1690d137.json (deflated 86%) 2024-08-06T20:50:19.4068908Z adding: test/test-reports/td_exclusions-a8bc85af3d03da563887.json (deflated 15%) 2024-08-06T20:50:19.4069688Z adding: test/test-reports/td_exclusions-90140957e31abb073f01.json (deflated 14%) 2024-08-06T20:50:19.4070454Z adding: test/test-reports/td_exclusions-e0461a735f4f02300fc9.json (deflated 14%) 2024-08-06T20:50:19.4071223Z adding: test/test-reports/td_exclusions-2cddd4bda4832f6b6b96.json (deflated 12%) 2024-08-06T20:50:19.4072084Z adding: test/test-reports/td_exclusions-2ea9ef5c4659f8284246.json (deflated 12%) 2024-08-06T20:50:19.4072852Z adding: test/test-reports/td_exclusions-33c279e89f465841813b.json (deflated 11%) 2024-08-06T20:50:19.4073598Z adding: test/test-reports/td_exclusions-84cdbf8825a95d21b1c0.json (deflated 18%) 2024-08-06T20:50:19.4074268Z adding: test/.pytorch-slow-tests.json (deflated 66%) 2024-08-06T20:50:19.4087174Z adding: test/.pytorch-disabled-tests.json (deflated 89%) 2024-08-06T20:50:19.4124623Z ##[group]Run # Remove any previous test reports if they exist 2024-08-06T20:50:19.4125218Z # Remove any previous test reports if they exist 2024-08-06T20:50:19.4125690Z rm -f test-reports-*.zip 2024-08-06T20:50:19.4126228Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-06T20:50:19.4133737Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:50:19.4134197Z env: 2024-08-06T20:50:19.4134493Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:19.4134918Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:19.4135587Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:19.4136373Z FILE_SUFFIX: test-distributed-1-3-linux.8xlarge.nvidia.gpu_28428107361 2024-08-06T20:50:19.4136924Z ##[endgroup] 2024-08-06T20:50:19.4348000Z adding: test/test-reports/python-pytest/distributed.test_c10d_functional_native/distributed.test_c10d_functional_native-09c40d04a5b24a98.xml (deflated 89%) 2024-08-06T20:50:19.4349708Z adding: test/test-reports/python-pytest/distributed._tensor.test_dtensor_ops/distributed._tensor.test_dtensor_ops-d108d13777d8e8e6.xml (deflated 28%) 2024-08-06T20:50:19.4351070Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_fx/distributed.fsdp.test_fsdp_fx-5e807ee139308b12.xml (deflated 37%) 2024-08-06T20:50:19.4352672Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_register_sharding/distributed._tensor.experimental.test_register_sharding-edfc7ea8cb075092.xml (deflated 49%) 2024-08-06T20:50:19.4354245Z adding: test/test-reports/python-pytest/distributed._tensor.test_api/distributed._tensor.test_api-79e924c9de248ca5.xml (deflated 75%) 2024-08-06T20:50:19.4355620Z adding: test/test-reports/python-pytest/distributed._tensor.test_random_ops/distributed._tensor.test_random_ops-6d91abf48896e8a9.xml (deflated 72%) 2024-08-06T20:50:19.4356974Z adding: test/test-reports/python-pytest/distributed._tensor.test_init/distributed._tensor.test_init-5eaec111aebbc5db.xml (deflated 71%) 2024-08-06T20:50:19.4358469Z adding: test/test-reports/python-pytest/distributed._tensor.debug.test_comm_mode_features/distributed._tensor.debug.test_comm_mode_features-0c8bd0341e5ea851.xml (deflated 63%) 2024-08-06T20:50:19.4359988Z adding: test/test-reports/python-pytest/distributed._tensor.test_math_ops/distributed._tensor.test_math_ops-a19f66a48f7e0764.xml (deflated 76%) 2024-08-06T20:50:19.4361430Z adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_fsdp_ep/distributed.checkpoint.e2e.test_fsdp_ep-5b68ba10ad778e20.xml (deflated 50%) 2024-08-06T20:50:19.4363217Z adding: test/test-reports/python-pytest/distributed.test_compute_comm_reordering/distributed.test_compute_comm_reordering-76d8db2d0c0e27b3.xml (deflated 82%) 2024-08-06T20:50:19.4364718Z adding: test/test-reports/python-pytest/distributed._tensor.test_matrix_ops/distributed._tensor.test_matrix_ops-d31dc2a251469383.xml (deflated 75%) 2024-08-06T20:50:19.4366171Z adding: test/test-reports/python-pytest/distributed._tensor.test_redistribute/distributed._tensor.test_redistribute-cfa165f0852071ce.xml (deflated 79%) 2024-08-06T20:50:19.4367691Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_format_utils/distributed.checkpoint.test_format_utils-092651a8b625c558.xml (deflated 56%) 2024-08-06T20:50:19.4368960Z adding: test/test-reports/python-pytest/test.run_test/test.run_test-20db868e5ab574d7.xml (deflated 60%) 2024-08-06T20:50:19.4369954Z adding: test/test-reports/python-pytest/test.run_test/test.run_test-a0778e6ba22228b8.xml (deflated 60%) 2024-08-06T20:50:19.4370937Z adding: test/test-reports/python-pytest/test.run_test/test.run_test-aa41b9db8fd8df99.xml (deflated 69%) 2024-08-06T20:50:19.4371902Z adding: test/test-reports/python-pytest/test.run_test/test.run_test-e8e4e32c532ea93c.xml (deflated 78%) 2024-08-06T20:50:19.4372883Z adding: test/test-reports/python-pytest/test.run_test/test.run_test-75741edf2ca83522.xml (deflated 74%) 2024-08-06T20:50:19.4373862Z adding: test/test-reports/python-pytest/test.run_test/test.run_test-a9311b904878955f.xml (deflated 76%) 2024-08-06T20:50:19.4374832Z adding: test/test-reports/python-pytest/test.run_test/test.run_test-fc284a177e47c83a.xml (deflated 64%) 2024-08-06T20:50:19.4411472Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-06T20:50:19.4412036Z # Remove any previous usage logs if they exist 2024-08-06T20:50:19.4412497Z rm -f logs-*.zip 2024-08-06T20:50:19.4413048Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-06T20:50:19.4413699Z # so check to see if the file exists first 2024-08-06T20:50:19.4414158Z if [ -f 'usage_log.txt' ]; then 2024-08-06T20:50:19.4414614Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-06T20:50:19.4415207Z fi 2024-08-06T20:50:19.4415685Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-06T20:50:19.4416171Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-06T20:50:19.4416614Z fi 2024-08-06T20:50:19.4423593Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:50:19.4424052Z env: 2024-08-06T20:50:19.4424340Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:19.4424776Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:19.4425436Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:19.4426522Z FILE_SUFFIX: test-distributed-1-3-linux.8xlarge.nvidia.gpu_28428107361 2024-08-06T20:50:19.4427060Z ##[endgroup] 2024-08-06T20:50:19.4472244Z adding: usage_log.txt (deflated 92%) 2024-08-06T20:50:19.4689780Z adding: test/test-reports/distributed.test_c10d_functional_native_1.1_2ebe2b2be1095a10_.log (deflated 88%) 2024-08-06T20:50:19.4690810Z adding: test/test-reports/distributed._tensor.test_dtensor_ops_1.1_a728983e33e9c5ea_.log (deflated 52%) 2024-08-06T20:50:19.4691773Z adding: test/test-reports/distributed.fsdp.test_fsdp_fx_1.1_6daa2efcf1b810b5_.log (deflated 53%) 2024-08-06T20:50:19.4692822Z adding: test/test-reports/distributed._tensor.experimental.test_register_sharding_1.1_de3ed1c0bf0aacba_.log (deflated 60%) 2024-08-06T20:50:19.4693864Z adding: test/test-reports/distributed._tensor.test_api_1.1_5676bad0d47cc01f_.log (deflated 89%) 2024-08-06T20:50:19.4694812Z adding: test/test-reports/distributed._tensor.test_random_ops_1.1_6b7f634b2f4db62f_.log (deflated 70%) 2024-08-06T20:50:19.4695725Z adding: test/test-reports/distributed._tensor.test_init_1.1_82fde18823aad1ff_.log (deflated 69%) 2024-08-06T20:50:19.4696718Z adding: test/test-reports/distributed._tensor.debug.test_comm_mode_features_1.1_bd05722d18b5c1b9_.log (deflated 73%) 2024-08-06T20:50:19.4697737Z adding: test/test-reports/distributed._tensor.test_math_ops_1.1_07bce20189e09810_.log (deflated 74%) 2024-08-06T20:50:19.4698729Z adding: test/test-reports/distributed.checkpoint.e2e.test_fsdp_ep_1.1_14130551e8d85e3e_.log (deflated 62%) 2024-08-06T20:50:19.4699738Z adding: test/test-reports/distributed.test_compute_comm_reordering_1.1_e9f5996b7610929d_.log (deflated 75%) 2024-08-06T20:50:19.4700735Z adding: test/test-reports/distributed._tensor.test_matrix_ops_1.1_fdb63b4266d6568a_.log (deflated 73%) 2024-08-06T20:50:19.4702359Z adding: test/test-reports/distributed._tensor.test_redistribute_1.1_cf6070e796ea39d7_.log (deflated 93%) 2024-08-06T20:50:19.4703710Z adding: test/test-reports/distributed.checkpoint.test_format_utils_1.1_1137081102ea5d7c_.log (deflated 72%) 2024-08-06T20:50:19.4704617Z adding: test/test-reports/cpp.FileStoreTest_1.1_02dca750a74550ea_.log (deflated 71%) 2024-08-06T20:50:19.4705434Z adding: test/test-reports/cpp.HashStoreTest_1.1_8232ff1aa92ec82a_.log (deflated 71%) 2024-08-06T20:50:19.4706524Z adding: test/test-reports/cpp.TCPStoreTest_1.1_ae2d119ee9ea84db_.log (deflated 78%) 2024-08-06T20:50:19.4707609Z adding: test/test-reports/cpp.ProcessGroupGlooTest_1.1_a144a89a7ac8cf70_.log (deflated 85%) 2024-08-06T20:50:19.4708849Z adding: test/test-reports/cpp.ProcessGroupNCCLTest_1.1_068d1fb6a9c25506_.log (deflated 82%) 2024-08-06T20:50:19.4709852Z adding: test/test-reports/cpp.ProcessGroupNCCLErrorsTest_1.1_619e5d07f8a75c4a_.log (deflated 78%) 2024-08-06T20:50:19.4710699Z adding: test/test-reports/cpp.test_cpp_rpc_1.1_cb81b5479c8e4136_.log (deflated 79%) 2024-08-06T20:50:19.4751116Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-06T20:50:19.4751860Z # Remove any previous debugging artifacts if they exist 2024-08-06T20:50:19.4752341Z rm -f debug-*.zip 2024-08-06T20:50:19.4752718Z if [ -d 'test/debug' ]; then 2024-08-06T20:50:19.4753177Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-06T20:50:19.4753595Z fi 2024-08-06T20:50:19.4761211Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:50:19.4761825Z env: 2024-08-06T20:50:19.4762102Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:19.4762533Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:19.4763218Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:19.4763988Z FILE_SUFFIX: test-distributed-1-3-linux.8xlarge.nvidia.gpu_28428107361 2024-08-06T20:50:19.4764539Z ##[endgroup] 2024-08-06T20:50:19.4852757Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-06T20:50:19.4853173Z with: 2024-08-06T20:50:19.4853474Z s3-bucket: gha-artifacts 2024-08-06T20:50:19.4853895Z s3-prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:50:19.4854327Z retention-days: 14 2024-08-06T20:50:19.4854668Z if-no-files-found: warn 2024-08-06T20:50:19.4855031Z path: test-jsons-*.zip 2024-08-06T20:50:19.4855356Z name: artifact 2024-08-06T20:50:19.4855669Z region: us-east-1 2024-08-06T20:50:19.4855982Z env: 2024-08-06T20:50:19.4856272Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:19.4856718Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:19.4857404Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:19.4858002Z ##[endgroup] 2024-08-06T20:50:19.9278604Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-06T20:50:19.9279354Z With the provided path, there will be 1 file uploaded 2024-08-06T20:50:19.9279940Z Uploading to s3 prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:50:19.9326662Z Starting upload of test-jsons-test-distributed-1-3-linux.8xlarge.nvidia.gpu_28428107361.zip 2024-08-06T20:50:20.0893431Z Finished upload of test-jsons-test-distributed-1-3-linux.8xlarge.nvidia.gpu_28428107361.zip 2024-08-06T20:50:20.1061740Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-06T20:50:20.1062181Z with: 2024-08-06T20:50:20.1062467Z s3-bucket: gha-artifacts 2024-08-06T20:50:20.1062896Z s3-prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:50:20.1063369Z retention-days: 14 2024-08-06T20:50:20.1063713Z if-no-files-found: error 2024-08-06T20:50:20.1064083Z path: test-reports-*.zip 2024-08-06T20:50:20.1064438Z name: artifact 2024-08-06T20:50:20.1064732Z region: us-east-1 2024-08-06T20:50:20.1065044Z env: 2024-08-06T20:50:20.1065351Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:20.1066052Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:20.1066751Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:20.1067366Z ##[endgroup] 2024-08-06T20:50:20.5573112Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-06T20:50:20.5574002Z With the provided path, there will be 1 file uploaded 2024-08-06T20:50:20.5574601Z Uploading to s3 prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:50:20.5622392Z Starting upload of test-reports-test-distributed-1-3-linux.8xlarge.nvidia.gpu_28428107361.zip 2024-08-06T20:50:20.7808391Z Finished upload of test-reports-test-distributed-1-3-linux.8xlarge.nvidia.gpu_28428107361.zip 2024-08-06T20:50:20.7971577Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-06T20:50:20.7972014Z with: 2024-08-06T20:50:20.7972303Z s3-bucket: gha-artifacts 2024-08-06T20:50:20.7972730Z s3-prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:50:20.7973180Z retention-days: 14 2024-08-06T20:50:20.7973509Z if-no-files-found: ignore 2024-08-06T20:50:20.7973870Z path: logs-*.zip 2024-08-06T20:50:20.7974186Z name: artifact 2024-08-06T20:50:20.7974475Z region: us-east-1 2024-08-06T20:50:20.7974785Z env: 2024-08-06T20:50:20.7975074Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:20.7975487Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:20.7976241Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:20.7976854Z ##[endgroup] 2024-08-06T20:50:21.2385190Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-06T20:50:21.2386691Z With the provided path, there will be 1 file uploaded 2024-08-06T20:50:21.2387490Z Uploading to s3 prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:50:21.2434130Z Starting upload of logs-test-distributed-1-3-linux.8xlarge.nvidia.gpu_28428107361.zip 2024-08-06T20:50:21.4099356Z Finished upload of logs-test-distributed-1-3-linux.8xlarge.nvidia.gpu_28428107361.zip 2024-08-06T20:50:21.4277789Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-06T20:50:21.4278225Z with: 2024-08-06T20:50:21.4278508Z s3-bucket: gha-artifacts 2024-08-06T20:50:21.4278930Z s3-prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:50:21.4279381Z retention-days: 14 2024-08-06T20:50:21.4279708Z if-no-files-found: ignore 2024-08-06T20:50:21.4280069Z path: debug-*.zip 2024-08-06T20:50:21.4280385Z name: artifact 2024-08-06T20:50:21.4280677Z region: us-east-1 2024-08-06T20:50:21.4280986Z env: 2024-08-06T20:50:21.4281271Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:21.4281699Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:21.4282385Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:21.4283006Z ##[endgroup] 2024-08-06T20:50:21.8645096Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-06T20:50:21.8809470Z ##[group]Run # shellcheck disable=SC2156 2024-08-06T20:50:21.8809982Z # shellcheck disable=SC2156 2024-08-06T20:50:21.8810696Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-06T20:50:21.8819591Z shell: /usr/bin/bash -e {0} 2024-08-06T20:50:21.8819949Z env: 2024-08-06T20:50:21.8820240Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:21.8820673Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:21.8821347Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:21.8821959Z ##[endgroup] 2024-08-06T20:50:22.1825980Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-06T20:50:22.1826542Z with: 2024-08-06T20:50:22.1826829Z env: 2024-08-06T20:50:22.1827120Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:22.1827542Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:22.1828226Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:22.1828840Z ##[endgroup] 2024-08-06T20:50:22.1870890Z ##[group]Run set -eou pipefail 2024-08-06T20:50:22.1871319Z set -eou pipefail 2024-08-06T20:50:22.1871657Z  2024-08-06T20:50:22.1872124Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-06T20:50:22.1872700Z for _ in $(seq 1440); do 2024-08-06T20:50:22.1873106Z  # Break if no ssh session exists anymore 2024-08-06T20:50:22.1873547Z  if [ "$(who)" = "" ]; then 2024-08-06T20:50:22.1873935Z  break 2024-08-06T20:50:22.1874296Z  fi 2024-08-06T20:50:22.1874594Z  echo "." 2024-08-06T20:50:22.1874932Z  sleep 5 2024-08-06T20:50:22.1875246Z done 2024-08-06T20:50:22.1882842Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:50:22.1883321Z env: 2024-08-06T20:50:22.1883614Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:22.1884027Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:22.1884708Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:22.1885322Z ##[endgroup] 2024-08-06T20:50:22.1909092Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-06T20:50:22.1964304Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-06T20:50:22.1965000Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-06T20:50:22.1965519Z # shellcheck disable=SC2046 2024-08-06T20:50:22.1965965Z docker stop $(docker ps -q) || true 2024-08-06T20:50:22.1966411Z # Prune all of the docker images 2024-08-06T20:50:22.1967020Z docker system prune -af 2024-08-06T20:50:22.1974260Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:50:22.1974740Z env: 2024-08-06T20:50:22.1975047Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:50:22.1975465Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:50:22.1976216Z DOCKER_CONTAINER_ID: ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:22.1976852Z ##[endgroup] 2024-08-06T20:50:22.5378611Z ed694363f944 2024-08-06T20:50:23.0909559Z Deleted Containers: 2024-08-06T20:50:23.0910075Z ed694363f94407a7ccd9beed2affe79504deca73274e5d7689142a33933f800e 2024-08-06T20:50:23.0910548Z 2024-08-06T20:50:29.6683516Z Deleted Images: 2024-08-06T20:50:29.6684492Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:50:29.6686144Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9@sha256:66181a134b3ebc35388dabcc46dc11e767152fd72514dad76711590f50b650f7 2024-08-06T20:50:29.6687428Z deleted: sha256:8e4e19ae0377c273bc603b24bc49d406d6f755072d4534d040c86afbe761a129 2024-08-06T20:50:29.6688230Z deleted: sha256:bee5df81b6fe1ee0aae7e536a9bda16351d0515be5b789ad50bf0ac60fd41e33 2024-08-06T20:50:29.6689035Z deleted: sha256:8ee1ee099b65b2d73d66dc3330f82f8be2dd6cbdccf3d718b448a5e3419c214e 2024-08-06T20:50:29.6689815Z deleted: sha256:aed7424cdc4428e32e398a33e98e6e252075e6280e7068e20df7626451fa312a 2024-08-06T20:50:29.6690565Z deleted: sha256:61b7d2994ed16481c3de07839b5a3bb779dc2f3f716ad49acb91537042c3b7d1 2024-08-06T20:50:29.6691335Z deleted: sha256:79999dfec9b5df258486b014aa99c30979900c4131effd2a7bc18251cd5fc372 2024-08-06T20:50:29.6692112Z deleted: sha256:9020e3bab07b431af26c788ac2d12cbca9a76ffb1c923ea45a38e872942e1f34 2024-08-06T20:50:29.6693146Z deleted: sha256:387e2f2f359df5f69c500ae9589c224d8283ae5418520509b3c9ef0a45d116fa 2024-08-06T20:50:29.6693965Z deleted: sha256:5ddfc73c14e2e0a762c51ddecfb2710258366e1d048a8bf56d70268486907b99 2024-08-06T20:50:29.6694761Z deleted: sha256:4dffb251634d1c8105e68310ef1b2ffb923dac4c4139f494be0938ad032a1bca 2024-08-06T20:50:29.6695519Z deleted: sha256:e0df7617281f0e4396253d2f0c40d1418ad604dc0912cae13f153fe22e316fa0 2024-08-06T20:50:29.6696303Z deleted: sha256:d239b3ccf05c2850364c3f73acbc6e0017a25f8faa4c3abaa8524e63dc9f9c29 2024-08-06T20:50:29.6697090Z deleted: sha256:918d0912faf4251f3be58f9a5aaa80f59a180ace4771b9ce903cedf94a57723a 2024-08-06T20:50:29.6697839Z deleted: sha256:09d5696392736fa88c52a0656d1d831d2bcb4015630810412565a5a086eb98ff 2024-08-06T20:50:29.6698602Z deleted: sha256:67542af70388f69b345fc7f0965e1fce3c552ee987689f804c8406536f16534f 2024-08-06T20:50:29.6699383Z deleted: sha256:350337bafbf2976e8d4e0b7ce3cc66b2cb43ebb8622fa6e538b02f561f929393 2024-08-06T20:50:29.6700195Z deleted: sha256:00fe9fe12fefe10d5dbfb2f855aa385e840f360deeccbc9ee92cae51c6eea4ba 2024-08-06T20:50:29.6700976Z deleted: sha256:9895af0586ff7d94936462a92d7cec0d8ec3b01df27b664f7e4ac277ce7bbe9e 2024-08-06T20:50:29.6701774Z deleted: sha256:7dd7dbc3ec792cf9d043ff867afe169d15fe414138097ad9247a5f388d3eeccf 2024-08-06T20:50:29.6702560Z deleted: sha256:e0a3ace1b87c67d71f5db76d211740431660a7fbcd80415333c9e7b7b9a2e89b 2024-08-06T20:50:29.6703315Z deleted: sha256:b9c56b5040d5d5998961b788563b85e5be12f678bc6b3976d089a81f1c93b7e8 2024-08-06T20:50:29.6704094Z deleted: sha256:9232fe8591973938b7bdb0a7a31dcdbc79f9667d6e011a2facf4e17fb77d4922 2024-08-06T20:50:29.6704889Z deleted: sha256:52f5fbe778d735123327825c6a16dcec9bc84c2623087bb4c7c094ac99764ae7 2024-08-06T20:50:29.6705926Z deleted: sha256:4b7c4121b76b04d5c7e4a0b77add435c0caa99cccff506ea704eba27fd2b2166 2024-08-06T20:50:29.6706770Z deleted: sha256:795dd765ace5b617808fc40cd576b449ae01b8671290e6df6b49d86ea79f4145 2024-08-06T20:50:29.6707494Z deleted: sha256:51cc651f1ffbeef49ad010ab2d4cf2d5ce5f8f5767c2133346a959c773d837bb 2024-08-06T20:50:29.6708299Z deleted: sha256:4c3019549b40f7a268d344eae162d7a7cc78855880db865944da136e01c190e9 2024-08-06T20:50:29.6709330Z deleted: sha256:855b789ec056edaab8cc8dad50a82e9dbfb9b20bcf39c70d9c3bd40080b2b65f 2024-08-06T20:50:29.6710047Z deleted: sha256:cef853cb7e0f5d6351692a4b9a0413a2496ea3c7bfe5d8063528ac2d6c17696f 2024-08-06T20:50:29.6710811Z deleted: sha256:4f7b163ef5ff693ce9f7380bdf7b526131f2567489342f0630949c9341e1c63f 2024-08-06T20:50:29.6711586Z deleted: sha256:09f12e5e102232b4ec9fd5168f4aa4b20ee0dfc755b053ac57690657e72e3fe3 2024-08-06T20:50:29.6712365Z deleted: sha256:c9962be393e03353af0d6a26f5478a94cacb5e528ee3f97609fac594c9a5d387 2024-08-06T20:50:29.6713149Z deleted: sha256:ac7aa4ec25b0d9a035c438effaa309071729de3f3a0f96e8e481b9839e7460b8 2024-08-06T20:50:29.6713911Z deleted: sha256:63ff2106018988888d022ee38d6ce1805aa29d8368a18dec1ab8eb587799ab44 2024-08-06T20:50:29.6714689Z deleted: sha256:3298c1acdeb701f11351366cfe680a84b8290666fdcd76af5ac79f226422fc0d 2024-08-06T20:50:29.6715481Z deleted: sha256:f076cc29eff5dae969bc7c05003d680b6e683eaef971c0a3f67a8a4d12693ee4 2024-08-06T20:50:29.6716247Z deleted: sha256:2604655e98c557e6de2e650d21cd0306c01f12204f3538386a9f374d003a4d70 2024-08-06T20:50:29.6717016Z deleted: sha256:fb942194d6d4fc6f9c8a4c8cba17330563877c92369892edc9b005b8b491f204 2024-08-06T20:50:29.6717790Z deleted: sha256:d349cc387708a540eee839ccd78494458dc5f1ccf989db608d25c5f798ec8c76 2024-08-06T20:50:29.6718552Z deleted: sha256:269864865f9b12ee399ca9ee3e15a16cedb86fd4889e01a83567afb357a774e6 2024-08-06T20:50:29.6719326Z deleted: sha256:236a91ce743cd500427338a78915ff834ae80331fcc06afccfca82a35bffb63c 2024-08-06T20:50:29.6720144Z deleted: sha256:de968bbc1d8bc2ad420d6a1c78a9a6cf8a763bb855ca3aefe7de6c5c52201f7a 2024-08-06T20:50:29.6720897Z deleted: sha256:0f15ecc513ffdac89965bd76cd9f8c7f4e37bbe7be0bfa086353133e2dd4f086 2024-08-06T20:50:29.6721697Z deleted: sha256:754bb9b5a5af40b6ad88f0ab14be37b9e512e25edccbfc9fd5e0c5a88ca09c0f 2024-08-06T20:50:29.6722653Z deleted: sha256:427104437dfa1883feec2c441eb4e4f825481883c6d3d31759361cbc924a27fe 2024-08-06T20:50:29.6723528Z deleted: sha256:0b404434cf2c3087ef28ea3cdc5813cc555f6e261c2514a7ab5bf5d484b5b575 2024-08-06T20:50:29.6724353Z deleted: sha256:bfb91e062bab9dafb63d0d4ab366026692fbd63e642da9ab28f04c8bba9b56a3 2024-08-06T20:50:29.6725164Z deleted: sha256:54617b36a396e9e82f2092257d2f621c90de460e0e54b1cf8df065573fbaa0ba 2024-08-06T20:50:29.6725938Z deleted: sha256:d8bdc337db1750bb3ebc7c8e4e1bd48e722218f261cb35ec2056ceaa6d3c4e9c 2024-08-06T20:50:29.6726771Z deleted: sha256:25cbacf4be7b2bbcbeaf7375d0eff0eaa33e94c5bef20ff0993bf074fdaa54eb 2024-08-06T20:50:29.6727529Z deleted: sha256:43744f05dcf6dc44c2f5e06533b13e085319a78b6b5e17eba5e7bd4419f5c905 2024-08-06T20:50:29.6728304Z deleted: sha256:2ea7d74f63d16a8d6e14a87e0e916f67f99536b21ac53e80b27786dcd1ce36fd 2024-08-06T20:50:29.6729064Z deleted: sha256:dbfb6031f18e50c89ff1a268c1e80e2fa88e727c48656718b326b60e8ef0d383 2024-08-06T20:50:29.6729859Z deleted: sha256:3eb3bb1755fcc3d4aba9abe2d2270ed2a55ec6254b3e87ca7688370e67e707a3 2024-08-06T20:50:29.6730646Z deleted: sha256:04605aba2d51924a85499584edda24c28390eb777c1c88eff2f4191b48e3da8c 2024-08-06T20:50:29.6731413Z deleted: sha256:117ff9e5eddba4582a39d416380eadf1c245237612c5ae0d6781d36c36bcb87f 2024-08-06T20:50:29.6732197Z deleted: sha256:ebf86c84eafa12a81b8eb66541a56c8056f4f9bc30e4be761771e53c6e1342aa 2024-08-06T20:50:29.6732982Z deleted: sha256:23d7f7e0c9d7e60acb59e60da9b4ee4e1840c1c406bf2082822c49430975529d 2024-08-06T20:50:29.6733746Z deleted: sha256:3e7e5583645195af5eeb60fe49024a4aa2f0eb3be28ff95c9859bb6233c29c4b 2024-08-06T20:50:29.6734529Z deleted: sha256:45b8f0dd78e72f0156b1fbc4dcb14200c61954f37a75129c753275e5a45ccd9b 2024-08-06T20:50:29.6735317Z deleted: sha256:f6b7a042384e2f2bb3401da2b9f4e9b9f942adb70be7e7f89c46fa81a42774d8 2024-08-06T20:50:29.6736109Z deleted: sha256:eb547a7b8fe732bffe8cc9a66aa19f2b41f6e0616a8a27040307c022d00fbc0d 2024-08-06T20:50:29.6736876Z deleted: sha256:20f729283bb4f2063f3a8baa0b2afc17911da345bf367e1918d8cc09ddd6d964 2024-08-06T20:50:29.6737660Z deleted: sha256:8ce8da9ab0319625a66c35d24ddadf8d2688d16c5024ba2345099431e0aa7ff8 2024-08-06T20:50:29.6738571Z deleted: sha256:e7a91181c74f879360530f0bbe957b8ff96a1ec19adde8a41d9e3fa2038d0d16 2024-08-06T20:50:29.6739333Z deleted: sha256:212b7a0fad7208624a2bc56768d8c489d2690142d0db0fa20f69eb648234e8d1 2024-08-06T20:50:29.6740116Z deleted: sha256:72315e37f51ba8bcc88fc5b56f92e5bb6264456d9a755afa01fd19f9128c1df7 2024-08-06T20:50:29.6740907Z deleted: sha256:bfc5ab07cfab1698147152988eabdc7ebbdeb35b6d9aefd9360dd64104cd3888 2024-08-06T20:50:29.6741674Z deleted: sha256:89edf8c906678c217b9a88160e1561991f53909c3fd27a4a5e1f0ca0f7ff7888 2024-08-06T20:50:29.6742462Z deleted: sha256:d5ba67f8f9b0c2b6da06d2d5d9d5fedc9f17cbe1a5e13a1a4033a4619f4528ba 2024-08-06T20:50:29.6743260Z deleted: sha256:5d61311e1a293c0c9642e34871719525dfeec7028e4b45e08344a469e3fe24bf 2024-08-06T20:50:29.6744016Z deleted: sha256:b89bb70f745a94c0edd5e974aee294e160735f76fd666a1a06a0b9b1be3caf63 2024-08-06T20:50:29.6744811Z deleted: sha256:9e5477f1f21a38c3dfe21a56a7d77d1b1a42453aed3e11729afcbb9d5731770b 2024-08-06T20:50:29.6745771Z deleted: sha256:b76f4af91cffe2bb7b3f09960f5bdbb26f741419defda65c451c4407fc5b0a92 2024-08-06T20:50:29.6746582Z deleted: sha256:3f8748a00ebd394c0e9d9d481972d764dff58896841ef0150b3f7efaac0f4a4a 2024-08-06T20:50:29.6747335Z deleted: sha256:13989c771e3ba4d40f618344e55e5cb7062a35041b5efd69e2f740ba819f46ee 2024-08-06T20:50:29.6748119Z deleted: sha256:9ebb27837f6c67ad8c17e9af4e63c008faae03fa226c3641dd8bbbfeefb106c4 2024-08-06T20:50:29.6748896Z deleted: sha256:8c16907d735950f8093582bc3cf392b70e002456bf9c4224de5a5b86b78b3ea2 2024-08-06T20:50:29.6749646Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-08-06T20:50:29.6750125Z 2024-08-06T20:50:29.6750293Z Total reclaimed space: 26.03GB 2024-08-06T20:50:29.6814915Z Post job cleanup. 2024-08-06T20:50:29.6874702Z Post job cleanup. 2024-08-06T20:50:29.8047049Z [command]/usr/bin/git version 2024-08-06T20:50:29.8100621Z git version 2.40.1 2024-08-06T20:50:29.8164299Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/86567ca3-62b0-42bb-bbb1-5d63c46039a7' before making global git config changes 2024-08-06T20:50:29.8165801Z Adding repository directory to the temporary git global config as a safe directory 2024-08-06T20:50:29.8171963Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:50:29.8213632Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-06T20:50:29.8251551Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-06T20:50:29.8567812Z Entering 'android/libs/fbjni' 2024-08-06T20:50:29.8619566Z Entering 'third_party/FP16' 2024-08-06T20:50:29.8671096Z Entering 'third_party/FXdiv' 2024-08-06T20:50:29.8720827Z Entering 'third_party/NNPACK' 2024-08-06T20:50:29.8772199Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:50:29.8830022Z Entering 'third_party/XNNPACK' 2024-08-06T20:50:29.8899858Z Entering 'third_party/benchmark' 2024-08-06T20:50:29.8952452Z Entering 'third_party/cpp-httplib' 2024-08-06T20:50:29.9003626Z Entering 'third_party/cpuinfo' 2024-08-06T20:50:29.9056540Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:50:29.9110454Z Entering 'third_party/cutlass' 2024-08-06T20:50:29.9169866Z Entering 'third_party/eigen' 2024-08-06T20:50:29.9223435Z Entering 'third_party/fbgemm' 2024-08-06T20:50:29.9277522Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:50:29.9327566Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:50:29.9379021Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:50:29.9438324Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:50:29.9488880Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:50:29.9539726Z Entering 'third_party/flatbuffers' 2024-08-06T20:50:29.9594905Z Entering 'third_party/fmt' 2024-08-06T20:50:29.9646595Z Entering 'third_party/foxi' 2024-08-06T20:50:29.9697421Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:50:29.9748442Z Entering 'third_party/gloo' 2024-08-06T20:50:29.9799572Z Entering 'third_party/googletest' 2024-08-06T20:50:29.9851003Z Entering 'third_party/ideep' 2024-08-06T20:50:29.9901241Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:50:29.9961981Z Entering 'third_party/ittapi' 2024-08-06T20:50:30.0013535Z Entering 'third_party/kineto' 2024-08-06T20:50:30.0064603Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:50:30.0116604Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:50:30.0169599Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:50:30.0223092Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:50:30.0276603Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:50:30.0327769Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:50:30.0379925Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:50:30.0430257Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:50:30.0480852Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:50:30.0532282Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:50:30.0585520Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:50:30.0636598Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:50:30.0689078Z Entering 'third_party/mimalloc' 2024-08-06T20:50:30.0741622Z Entering 'third_party/nccl/nccl' 2024-08-06T20:50:30.0794635Z Entering 'third_party/nlohmann' 2024-08-06T20:50:30.0848286Z Entering 'third_party/onnx' 2024-08-06T20:50:30.0917965Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:50:30.0970367Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:50:30.1024084Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:50:30.1077634Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:50:30.1129422Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:50:30.1180017Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:50:30.1232154Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:50:30.1283792Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:50:30.1334311Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:50:30.1384831Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:50:30.1435357Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:50:30.1488375Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:50:30.1540766Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:50:30.1613560Z Entering 'third_party/pocketfft' 2024-08-06T20:50:30.1666201Z Entering 'third_party/protobuf' 2024-08-06T20:50:30.1720492Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:50:30.1771078Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:50:30.1823021Z Entering 'third_party/psimd' 2024-08-06T20:50:30.1874994Z Entering 'third_party/pthreadpool' 2024-08-06T20:50:30.1925782Z Entering 'third_party/pybind11' 2024-08-06T20:50:30.1978074Z Entering 'third_party/python-peachpy' 2024-08-06T20:50:30.2030244Z Entering 'third_party/sleef' 2024-08-06T20:50:30.2082395Z Entering 'third_party/tensorpipe' 2024-08-06T20:50:30.2133925Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:50:30.2185273Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:50:30.2236195Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:50:30.2287434Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:50:30.2337093Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:50:30.2407911Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-06T20:50:30.2435266Z http.https://github.com/.extraheader 2024-08-06T20:50:30.2446565Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-06T20:50:30.2485216Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-06T20:50:30.2795434Z Entering 'android/libs/fbjni' 2024-08-06T20:50:30.2828218Z http.https://github.com/.extraheader 2024-08-06T20:50:30.2861875Z Entering 'third_party/FP16' 2024-08-06T20:50:30.2894791Z http.https://github.com/.extraheader 2024-08-06T20:50:30.2928922Z Entering 'third_party/FXdiv' 2024-08-06T20:50:30.2962982Z http.https://github.com/.extraheader 2024-08-06T20:50:30.2998044Z Entering 'third_party/NNPACK' 2024-08-06T20:50:30.3030714Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3065238Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:50:30.3099012Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3133331Z Entering 'third_party/XNNPACK' 2024-08-06T20:50:30.3167102Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3220096Z Entering 'third_party/benchmark' 2024-08-06T20:50:30.3252722Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3286835Z Entering 'third_party/cpp-httplib' 2024-08-06T20:50:30.3320831Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3354407Z Entering 'third_party/cpuinfo' 2024-08-06T20:50:30.3387712Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3421543Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:50:30.3454679Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3488385Z Entering 'third_party/cutlass' 2024-08-06T20:50:30.3522461Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3564780Z Entering 'third_party/eigen' 2024-08-06T20:50:30.3597337Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3633887Z Entering 'third_party/fbgemm' 2024-08-06T20:50:30.3667123Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3701395Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:50:30.3733312Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3766281Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:50:30.3800318Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3834294Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:50:30.3866821Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3907990Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:50:30.3940257Z http.https://github.com/.extraheader 2024-08-06T20:50:30.3973648Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:50:30.4006244Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4040646Z Entering 'third_party/flatbuffers' 2024-08-06T20:50:30.4074340Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4112912Z Entering 'third_party/fmt' 2024-08-06T20:50:30.4145896Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4179333Z Entering 'third_party/foxi' 2024-08-06T20:50:30.4212755Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4246239Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:50:30.4279886Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4313388Z Entering 'third_party/gloo' 2024-08-06T20:50:30.4345883Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4378734Z Entering 'third_party/googletest' 2024-08-06T20:50:30.4412208Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4445920Z Entering 'third_party/ideep' 2024-08-06T20:50:30.4478484Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4510812Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:50:30.4543191Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4584949Z Entering 'third_party/ittapi' 2024-08-06T20:50:30.4617257Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4651150Z Entering 'third_party/kineto' 2024-08-06T20:50:30.4685157Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4719242Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:50:30.4751372Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4784884Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:50:30.4818568Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4855441Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:50:30.4888218Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4922491Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:50:30.4954949Z http.https://github.com/.extraheader 2024-08-06T20:50:30.4990389Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:50:30.5023087Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5056352Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:50:30.5089056Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5125658Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:50:30.5158107Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5192957Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:50:30.5225677Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5260092Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:50:30.5293661Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5329011Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:50:30.5361371Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5397206Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:50:30.5429088Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5462919Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:50:30.5495441Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5530999Z Entering 'third_party/mimalloc' 2024-08-06T20:50:30.5563547Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5598094Z Entering 'third_party/nccl/nccl' 2024-08-06T20:50:30.5630732Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5663974Z Entering 'third_party/nlohmann' 2024-08-06T20:50:30.5697136Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5734453Z Entering 'third_party/onnx' 2024-08-06T20:50:30.5767320Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5818519Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:50:30.5851168Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5885927Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:50:30.5920219Z http.https://github.com/.extraheader 2024-08-06T20:50:30.5956286Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:50:30.5989362Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6024201Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:50:30.6056830Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6089830Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:50:30.6122416Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6155937Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:50:30.6187894Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6221067Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:50:30.6253569Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6288469Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:50:30.6320980Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6355026Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:50:30.6387401Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6421114Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:50:30.6453139Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6486257Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:50:30.6518094Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6555439Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:50:30.6587462Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6622133Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:50:30.6654066Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6710570Z Entering 'third_party/pocketfft' 2024-08-06T20:50:30.6743393Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6776387Z Entering 'third_party/protobuf' 2024-08-06T20:50:30.6810828Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6847795Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:50:30.6880293Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6913768Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:50:30.6946420Z http.https://github.com/.extraheader 2024-08-06T20:50:30.6981882Z Entering 'third_party/psimd' 2024-08-06T20:50:30.7014771Z http.https://github.com/.extraheader 2024-08-06T20:50:30.7047836Z Entering 'third_party/pthreadpool' 2024-08-06T20:50:30.7081439Z http.https://github.com/.extraheader 2024-08-06T20:50:30.7115043Z Entering 'third_party/pybind11' 2024-08-06T20:50:30.7148007Z http.https://github.com/.extraheader 2024-08-06T20:50:30.7181325Z Entering 'third_party/python-peachpy' 2024-08-06T20:50:30.7214465Z http.https://github.com/.extraheader 2024-08-06T20:50:30.7249132Z Entering 'third_party/sleef' 2024-08-06T20:50:30.7282366Z http.https://github.com/.extraheader 2024-08-06T20:50:30.7316774Z Entering 'third_party/tensorpipe' 2024-08-06T20:50:30.7349358Z http.https://github.com/.extraheader 2024-08-06T20:50:30.7382968Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:50:30.7415051Z http.https://github.com/.extraheader 2024-08-06T20:50:30.7449434Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:50:30.7483329Z http.https://github.com/.extraheader 2024-08-06T20:50:30.7517242Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:50:30.7548929Z http.https://github.com/.extraheader 2024-08-06T20:50:30.7582736Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:50:30.7615059Z http.https://github.com/.extraheader 2024-08-06T20:50:30.7647598Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:50:30.7680378Z http.https://github.com/.extraheader 2024-08-06T20:50:30.7815156Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-06T20:50:30.7846619Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-06T20:50:30.7853623Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:50:30.7854137Z ##[endgroup] 2024-08-06T20:50:36.8137820Z Cleaning up orphan processes