2024-08-06T20:30:58.8472321Z Current runner version: '2.318.0' 2024-08-06T20:30:58.8482523Z Runner name: 'i-01c85e67ad0689ec5' 2024-08-06T20:30:58.8483891Z Runner group name: 'Default' 2024-08-06T20:30:58.8485331Z Machine name: 'ip-10-0-49-118' 2024-08-06T20:30:58.8491936Z ##[group]GITHUB_TOKEN Permissions 2024-08-06T20:30:58.8495401Z Actions: read 2024-08-06T20:30:58.8496384Z Attestations: read 2024-08-06T20:30:58.8497454Z Checks: read 2024-08-06T20:30:58.8498384Z Contents: read 2024-08-06T20:30:58.8499341Z Deployments: read 2024-08-06T20:30:58.8500334Z Discussions: read 2024-08-06T20:30:58.8501336Z Issues: read 2024-08-06T20:30:58.8502348Z Metadata: read 2024-08-06T20:30:58.8503324Z Packages: read 2024-08-06T20:30:58.8504256Z Pages: read 2024-08-06T20:30:58.8505189Z PullRequests: read 2024-08-06T20:30:58.8506200Z RepositoryProjects: read 2024-08-06T20:30:58.8507231Z SecurityEvents: read 2024-08-06T20:30:58.8508283Z Statuses: read 2024-08-06T20:30:58.8509243Z ##[endgroup] 2024-08-06T20:30:58.8513830Z Secret source: Actions 2024-08-06T20:30:58.8515321Z Prepare workflow directory 2024-08-06T20:30:59.2326915Z Prepare all required actions 2024-08-06T20:30:59.2380701Z Getting action download info 2024-08-06T20:30:59.4227902Z Download action repository 'pytorch/test-infra@main' (SHA:a1f5a89251fc4258ab59806094fe3108f7d6741a) 2024-08-06T20:30:59.9176986Z Download action repository 'pytorch/pytorch@main' (SHA:de00c7958301ce81b9716bdef5731ed40d4d14ca) 2024-08-06T20:31:03.5831772Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-06T20:31:03.7193676Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-06T20:31:03.9713792Z Getting action download info 2024-08-06T20:31:04.0866442Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-06T20:31:04.3251490Z Getting action download info 2024-08-06T20:31:04.4260222Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-06T20:31:04.5716184Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/132710/merge (bf5bb5a1585a03379137fab341e87c02c77e76cd) 2024-08-06T20:31:04.5719310Z ##[group] Inputs 2024-08-06T20:31:04.5719969Z build-environment: linux-focal-cuda11.8-py3.10-gcc9 2024-08-06T20:31:04.5721610Z test-matrix: {"include": [{"config": "distributed", "shard": 1, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 2, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 3, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}]} 2024-08-06T20:31:04.5723740Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:31:04.5724850Z sync-tag: 2024-08-06T20:31:04.5726138Z timeout-minutes: 360 2024-08-06T20:31:04.5726604Z use-gha: 2024-08-06T20:31:04.5727067Z dashboard-tag: 2024-08-06T20:31:04.5727551Z s3-bucket: gha-artifacts 2024-08-06T20:31:04.5727985Z aws-role-to-assume: 2024-08-06T20:31:04.5728458Z ##[endgroup] 2024-08-06T20:31:04.5729217Z Complete job name: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:31:04.6461929Z A job started hook has been configured by the self-hosted runner administrator 2024-08-06T20:31:04.6614971Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-06T20:31:04.6627008Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:31:04.6627996Z ##[endgroup] 2024-08-06T20:31:06.2421713Z Runner Type: linux.8xlarge.nvidia.gpu 2024-08-06T20:31:06.2422361Z Instance Type: g3.8xlarge 2024-08-06T20:31:06.2422812Z AMI Name: amzn2-ami-hvm-2.0.20240306.2-x86_64-ebs 2024-08-06T20:31:06.2423306Z AMI ID: ami-0ce0c36d7a00b20e2 2024-08-06T20:31:11.2366554Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-06T20:31:11.2367340Z with: 2024-08-06T20:31:11.2368151Z github-secret: *** 2024-08-06T20:31:11.2369061Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-06T20:31:11.2370000Z activate-with-label: false 2024-08-06T20:31:11.2370376Z label: with-ssh 2024-08-06T20:31:11.2370716Z remove-existing-keys: true 2024-08-06T20:31:11.2371075Z fail-silently: true 2024-08-06T20:31:11.2371400Z env: 2024-08-06T20:31:11.2371690Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:31:11.2372030Z ##[endgroup] 2024-08-06T20:31:11.3466730Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-06T20:31:11.7267758Z Grabbing public ssh keys from https://github.com/drisspg.keys 2024-08-06T20:31:11.8167597Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-06T20:31:11.8188737Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-06T20:31:11.8228679Z Login using: ssh ec2-user@ec2-3-91-17-27.compute-1.amazonaws.com 2024-08-06T20:31:11.8229394Z All testing is done inside the container, to start an interactive session run: 2024-08-06T20:31:11.8230070Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-06T20:31:11.8363407Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-06T20:31:11.8363992Z with: 2024-08-06T20:31:11.8364276Z submodules: recursive 2024-08-06T20:31:11.8364625Z fetch-depth: 0 2024-08-06T20:31:11.8364935Z env: 2024-08-06T20:31:11.8365215Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:31:11.8365565Z ##[endgroup] 2024-08-06T20:31:11.8467353Z ##[group]Run retry () { 2024-08-06T20:31:11.8467754Z retry () { 2024-08-06T20:31:11.8468190Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-06T20:31:11.8468696Z } 2024-08-06T20:31:11.8469034Z echo "${GITHUB_WORKSPACE}" 2024-08-06T20:31:11.8469490Z if [ -z "${NO_SUDO}" ]; then 2024-08-06T20:31:11.8469943Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-06T20:31:11.8470376Z else 2024-08-06T20:31:11.8470701Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-06T20:31:11.8471117Z fi 2024-08-06T20:31:11.8471481Z mkdir "${GITHUB_WORKSPACE}" 2024-08-06T20:31:11.8480193Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:31:11.8480689Z env: 2024-08-06T20:31:11.8480967Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:31:11.8481320Z NO_SUDO: 2024-08-06T20:31:11.8481609Z ##[endgroup] 2024-08-06T20:31:11.8507982Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:31:15.3860740Z ##[group]Run malfet/checkout@silent-checkout 2024-08-06T20:31:15.3861189Z with: 2024-08-06T20:31:15.3861504Z ref: b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:31:15.3861937Z fetch-depth: 0 2024-08-06T20:31:15.3862257Z submodules: recursive 2024-08-06T20:31:15.3862619Z quiet-checkout: true 2024-08-06T20:31:15.3862986Z repository: pytorch/pytorch 2024-08-06T20:31:15.3863496Z token: *** 2024-08-06T20:31:15.3863801Z ssh-strict: true 2024-08-06T20:31:15.3864119Z persist-credentials: true 2024-08-06T20:31:15.3864481Z clean: true 2024-08-06T20:31:15.3864804Z sparse-checkout-cone-mode: true 2024-08-06T20:31:15.3865190Z lfs: false 2024-08-06T20:31:15.3865497Z set-safe-directory: true 2024-08-06T20:31:15.3865829Z env: 2024-08-06T20:31:15.3866117Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:31:15.3866459Z ##[endgroup] 2024-08-06T20:31:15.5134086Z Syncing repository: pytorch/pytorch 2024-08-06T20:31:15.5135927Z ##[group]Getting Git version info 2024-08-06T20:31:15.5136559Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-06T20:31:15.5137418Z [command]/usr/bin/git version 2024-08-06T20:31:15.5137833Z git version 2.40.1 2024-08-06T20:31:15.5170391Z ##[endgroup] 2024-08-06T20:31:15.5191598Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/74e2eabb-7555-4a1f-9207-56907f8836b5' before making global git config changes 2024-08-06T20:31:15.5193060Z Adding repository directory to the temporary git global config as a safe directory 2024-08-06T20:31:15.5198963Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:31:15.5241835Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-06T20:31:15.5247875Z ##[group]Initializing the repository 2024-08-06T20:31:15.5251691Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:31:15.5281444Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-06T20:31:15.5282532Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-06T20:31:15.5283222Z hint: of your new repositories, which will suppress this warning, call: 2024-08-06T20:31:15.5283757Z hint: 2024-08-06T20:31:15.5284155Z hint: git config --global init.defaultBranch 2024-08-06T20:31:15.5284600Z hint: 2024-08-06T20:31:15.5285036Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-06T20:31:15.5285744Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-06T20:31:15.5286479Z hint: 2024-08-06T20:31:15.5286807Z hint: git branch -m 2024-08-06T20:31:15.5287452Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-06T20:31:15.5294221Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-06T20:31:15.5326900Z ##[endgroup] 2024-08-06T20:31:15.5327462Z ##[group]Disabling automatic garbage collection 2024-08-06T20:31:15.5330988Z [command]/usr/bin/git config --local gc.auto 0 2024-08-06T20:31:15.5361957Z ##[endgroup] 2024-08-06T20:31:15.5362470Z ##[group]Setting up auth 2024-08-06T20:31:15.5371414Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-06T20:31:15.5404318Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-06T20:31:15.5686203Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-06T20:31:15.5714407Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-06T20:31:15.5991991Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-06T20:31:15.6038185Z ##[endgroup] 2024-08-06T20:31:15.6038746Z ##[group]Fetching the repository 2024-08-06T20:31:15.6046274Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-06T20:31:17.8063432Z remote: Enumerating objects: 1008383 2024-08-06T20:31:17.8064018Z remote: Enumerating objects: 1008693, done. 2024-08-06T20:31:17.8065044Z remote: Counting objects: 0% (1/310) 2024-08-06T20:31:17.8065653Z remote: Counting objects: 1% (4/310) 2024-08-06T20:31:17.8066187Z remote: Counting objects: 2% (7/310) 2024-08-06T20:31:17.8066704Z remote: Counting objects: 3% (10/310) 2024-08-06T20:31:17.8067214Z remote: Counting objects: 4% (13/310) 2024-08-06T20:31:17.8067717Z remote: Counting objects: 5% (16/310) 2024-08-06T20:31:17.8068733Z remote: Counting objects: 6% (19/310) 2024-08-06T20:31:17.8069428Z remote: Counting objects: 7% (22/310) 2024-08-06T20:31:17.8069908Z remote: Counting objects: 8% (25/310) 2024-08-06T20:31:17.8070429Z remote: Counting objects: 9% (28/310) 2024-08-06T20:31:17.8071069Z remote: Counting objects: 10% (31/310) 2024-08-06T20:31:17.8072080Z remote: Counting objects: 11% (35/310) 2024-08-06T20:31:17.8072587Z remote: Counting objects: 12% (38/310) 2024-08-06T20:31:17.8073098Z remote: Counting objects: 13% (41/310) 2024-08-06T20:31:17.8073607Z remote: Counting objects: 14% (44/310) 2024-08-06T20:31:17.8074101Z remote: Counting objects: 15% (47/310) 2024-08-06T20:31:17.8074614Z remote: Counting objects: 16% (50/310) 2024-08-06T20:31:17.8075118Z remote: Counting objects: 17% (53/310) 2024-08-06T20:31:17.8075604Z remote: Counting objects: 18% (56/310) 2024-08-06T20:31:17.8076108Z remote: Counting objects: 19% (59/310) 2024-08-06T20:31:17.8076610Z remote: Counting objects: 20% (62/310) 2024-08-06T20:31:17.8077451Z remote: Counting objects: 21% (66/310) 2024-08-06T20:31:17.8078028Z remote: Counting objects: 22% (69/310) 2024-08-06T20:31:17.8078536Z remote: Counting objects: 23% (72/310) 2024-08-06T20:31:17.8079020Z remote: Counting objects: 24% (75/310) 2024-08-06T20:31:17.8079542Z remote: Counting objects: 25% (78/310) 2024-08-06T20:31:17.8080043Z remote: Counting objects: 26% (81/310) 2024-08-06T20:31:17.8080627Z remote: Counting objects: 27% (84/310) 2024-08-06T20:31:17.8081386Z remote: Counting objects: 28% (87/310) 2024-08-06T20:31:17.8081958Z remote: Counting objects: 29% (90/310) 2024-08-06T20:31:17.8082436Z remote: Counting objects: 30% (93/310) 2024-08-06T20:31:17.8083031Z remote: Counting objects: 31% (97/310) 2024-08-06T20:31:17.8083537Z remote: Counting objects: 32% (100/310) 2024-08-06T20:31:17.8084048Z remote: Counting objects: 33% (103/310) 2024-08-06T20:31:17.8084539Z remote: Counting objects: 34% (106/310) 2024-08-06T20:31:17.8085056Z remote: Counting objects: 35% (109/310) 2024-08-06T20:31:17.8085577Z remote: Counting objects: 36% (112/310) 2024-08-06T20:31:17.8086059Z remote: Counting objects: 37% (115/310) 2024-08-06T20:31:17.8086588Z remote: Counting objects: 38% (118/310) 2024-08-06T20:31:17.8087097Z remote: Counting objects: 39% (121/310) 2024-08-06T20:31:17.8087581Z remote: Counting objects: 40% (124/310) 2024-08-06T20:31:17.8088099Z remote: Counting objects: 41% (128/310) 2024-08-06T20:31:17.8088617Z remote: Counting objects: 42% (131/310) 2024-08-06T20:31:17.8089094Z remote: Counting objects: 43% (134/310) 2024-08-06T20:31:17.8089603Z remote: Counting objects: 44% (137/310) 2024-08-06T20:31:17.8090111Z remote: Counting objects: 45% (140/310) 2024-08-06T20:31:17.8090590Z remote: Counting objects: 46% (143/310) 2024-08-06T20:31:17.8091099Z remote: Counting objects: 47% (146/310) 2024-08-06T20:31:17.8091611Z remote: Counting objects: 48% (149/310) 2024-08-06T20:31:17.8092090Z remote: Counting objects: 49% (152/310) 2024-08-06T20:31:17.8092594Z remote: Counting objects: 50% (155/310) 2024-08-06T20:31:17.8093118Z remote: Counting objects: 51% (159/310) 2024-08-06T20:31:17.8093593Z remote: Counting objects: 52% (162/310) 2024-08-06T20:31:17.8094101Z remote: Counting objects: 53% (165/310) 2024-08-06T20:31:17.8094604Z remote: Counting objects: 54% (168/310) 2024-08-06T20:31:17.8095093Z remote: Counting objects: 55% (171/310) 2024-08-06T20:31:17.8095603Z remote: Counting objects: 56% (174/310) 2024-08-06T20:31:17.8096107Z remote: Counting objects: 57% (177/310) 2024-08-06T20:31:17.8096583Z remote: Counting objects: 58% (180/310) 2024-08-06T20:31:17.8097089Z remote: Counting objects: 59% (183/310) 2024-08-06T20:31:17.8097592Z remote: Counting objects: 60% (186/310) 2024-08-06T20:31:17.8098068Z remote: Counting objects: 61% (190/310) 2024-08-06T20:31:17.8098578Z remote: Counting objects: 62% (193/310) 2024-08-06T20:31:17.8099141Z remote: Counting objects: 63% (196/310) 2024-08-06T20:31:17.8099875Z remote: Counting objects: 64% (199/310) 2024-08-06T20:31:17.8100359Z remote: Counting objects: 65% (202/310) 2024-08-06T20:31:17.8100868Z remote: Counting objects: 66% (205/310) 2024-08-06T20:31:17.8101443Z remote: Counting objects: 67% (208/310) 2024-08-06T20:31:17.8101934Z remote: Counting objects: 68% (211/310) 2024-08-06T20:31:17.8102443Z remote: Counting objects: 69% (214/310) 2024-08-06T20:31:17.8102953Z remote: Counting objects: 70% (217/310) 2024-08-06T20:31:17.8103433Z remote: Counting objects: 71% (221/310) 2024-08-06T20:31:17.8103940Z remote: Counting objects: 72% (224/310) 2024-08-06T20:31:17.8104448Z remote: Counting objects: 73% (227/310) 2024-08-06T20:31:17.8104959Z remote: Counting objects: 74% (230/310) 2024-08-06T20:31:17.8105471Z remote: Counting objects: 75% (233/310) 2024-08-06T20:31:17.8105974Z remote: Counting objects: 76% (236/310) 2024-08-06T20:31:17.8106470Z remote: Counting objects: 77% (239/310) 2024-08-06T20:31:17.8106981Z remote: Counting objects: 78% (242/310) 2024-08-06T20:31:17.8107492Z remote: Counting objects: 79% (245/310) 2024-08-06T20:31:17.8107974Z remote: Counting objects: 80% (248/310) 2024-08-06T20:31:17.8108621Z remote: Counting objects: 81% (252/310) 2024-08-06T20:31:17.8109172Z remote: Counting objects: 82% (255/310) 2024-08-06T20:31:17.8109661Z remote: Counting objects: 83% (258/310) 2024-08-06T20:31:17.8110176Z remote: Counting objects: 84% (261/310) 2024-08-06T20:31:17.8110683Z remote: Counting objects: 85% (264/310) 2024-08-06T20:31:17.8111171Z remote: Counting objects: 86% (267/310) 2024-08-06T20:31:17.8111690Z remote: Counting objects: 87% (270/310) 2024-08-06T20:31:17.8112194Z remote: Counting objects: 88% (273/310) 2024-08-06T20:31:17.8112674Z remote: Counting objects: 89% (276/310) 2024-08-06T20:31:17.8113208Z remote: Counting objects: 90% (279/310) 2024-08-06T20:31:17.8113721Z remote: Counting objects: 91% (283/310) 2024-08-06T20:31:17.8114201Z remote: Counting objects: 92% (286/310) 2024-08-06T20:31:17.8114709Z remote: Counting objects: 93% (289/310) 2024-08-06T20:31:17.8115228Z remote: Counting objects: 94% (292/310) 2024-08-06T20:31:17.8115708Z remote: Counting objects: 95% (295/310) 2024-08-06T20:31:17.8116214Z remote: Counting objects: 96% (298/310) 2024-08-06T20:31:17.8116717Z remote: Counting objects: 97% (301/310) 2024-08-06T20:31:17.8117480Z remote: Counting objects: 98% (304/310) 2024-08-06T20:31:17.8118019Z remote: Counting objects: 99% (307/310) 2024-08-06T20:31:17.8118527Z remote: Counting objects: 100% (310/310) 2024-08-06T20:31:17.8119036Z remote: Counting objects: 100% (310/310), done. 2024-08-06T20:31:17.8119585Z remote: Compressing objects: 0% (1/177) 2024-08-06T20:31:17.8144661Z remote: Compressing objects: 1% (2/177) 2024-08-06T20:31:17.8171617Z remote: Compressing objects: 2% (4/177) 2024-08-06T20:31:17.8202214Z remote: Compressing objects: 3% (6/177) 2024-08-06T20:31:17.8291911Z remote: Compressing objects: 4% (8/177) 2024-08-06T20:31:17.8424000Z remote: Compressing objects: 5% (9/177) 2024-08-06T20:31:17.8498154Z remote: Compressing objects: 6% (11/177) 2024-08-06T20:31:17.8526801Z remote: Compressing objects: 7% (13/177) 2024-08-06T20:31:17.8561594Z remote: Compressing objects: 8% (15/177) 2024-08-06T20:31:17.8603008Z remote: Compressing objects: 9% (16/177) 2024-08-06T20:31:17.8770664Z remote: Compressing objects: 10% (18/177) 2024-08-06T20:31:17.8771228Z remote: Compressing objects: 11% (20/177) 2024-08-06T20:31:17.8771754Z remote: Compressing objects: 12% (22/177) 2024-08-06T20:31:17.8772294Z remote: Compressing objects: 13% (24/177) 2024-08-06T20:31:17.8773543Z remote: Compressing objects: 14% (25/177) 2024-08-06T20:31:17.8774071Z remote: Compressing objects: 15% (27/177) 2024-08-06T20:31:17.8774570Z remote: Compressing objects: 16% (29/177) 2024-08-06T20:31:17.8775100Z remote: Compressing objects: 17% (31/177) 2024-08-06T20:31:17.8775640Z remote: Compressing objects: 18% (32/177) 2024-08-06T20:31:17.8776142Z remote: Compressing objects: 19% (34/177) 2024-08-06T20:31:17.8776663Z remote: Compressing objects: 20% (36/177) 2024-08-06T20:31:17.8777192Z remote: Compressing objects: 21% (38/177) 2024-08-06T20:31:17.8777693Z remote: Compressing objects: 22% (39/177) 2024-08-06T20:31:17.8778207Z remote: Compressing objects: 23% (41/177) 2024-08-06T20:31:17.8778733Z remote: Compressing objects: 24% (43/177) 2024-08-06T20:31:17.8779232Z remote: Compressing objects: 25% (45/177) 2024-08-06T20:31:17.8779797Z remote: Compressing objects: 26% (47/177) 2024-08-06T20:31:17.8780349Z remote: Compressing objects: 27% (48/177) 2024-08-06T20:31:17.8780853Z remote: Compressing objects: 28% (50/177) 2024-08-06T20:31:17.8781381Z remote: Compressing objects: 29% (52/177) 2024-08-06T20:31:17.8781910Z remote: Compressing objects: 30% (54/177) 2024-08-06T20:31:17.8782594Z remote: Compressing objects: 31% (55/177) 2024-08-06T20:31:17.8785673Z remote: Compressing objects: 32% (57/177) 2024-08-06T20:31:17.8791030Z remote: Compressing objects: 33% (59/177) 2024-08-06T20:31:17.8793372Z remote: Compressing objects: 34% (61/177) 2024-08-06T20:31:17.8795787Z remote: Compressing objects: 35% (62/177) 2024-08-06T20:31:17.8798861Z remote: Compressing objects: 36% (64/177) 2024-08-06T20:31:17.8801505Z remote: Compressing objects: 37% (66/177) 2024-08-06T20:31:17.8804611Z remote: Compressing objects: 38% (68/177) 2024-08-06T20:31:17.8805167Z remote: Compressing objects: 39% (70/177) 2024-08-06T20:31:17.8806487Z remote: Compressing objects: 40% (71/177) 2024-08-06T20:31:17.8809352Z remote: Compressing objects: 41% (73/177) 2024-08-06T20:31:17.8809909Z remote: Compressing objects: 42% (75/177) 2024-08-06T20:31:17.8812055Z remote: Compressing objects: 43% (77/177) 2024-08-06T20:31:17.8812618Z remote: Compressing objects: 44% (78/177) 2024-08-06T20:31:17.8814090Z remote: Compressing objects: 45% (80/177) 2024-08-06T20:31:17.8817079Z remote: Compressing objects: 46% (82/177) 2024-08-06T20:31:17.8817661Z remote: Compressing objects: 47% (84/177) 2024-08-06T20:31:17.8818196Z remote: Compressing objects: 48% (85/177) 2024-08-06T20:31:17.8818702Z remote: Compressing objects: 49% (87/177) 2024-08-06T20:31:17.8819232Z remote: Compressing objects: 50% (89/177) 2024-08-06T20:31:17.8819763Z remote: Compressing objects: 51% (91/177) 2024-08-06T20:31:17.8820264Z remote: Compressing objects: 52% (93/177) 2024-08-06T20:31:17.8820817Z remote: Compressing objects: 53% (94/177) 2024-08-06T20:31:17.8821349Z remote: Compressing objects: 54% (96/177) 2024-08-06T20:31:17.8821855Z remote: Compressing objects: 55% (98/177) 2024-08-06T20:31:17.8822402Z remote: Compressing objects: 56% (100/177) 2024-08-06T20:31:17.8822943Z remote: Compressing objects: 57% (101/177) 2024-08-06T20:31:17.8823448Z remote: Compressing objects: 58% (103/177) 2024-08-06T20:31:17.8823978Z remote: Compressing objects: 59% (105/177) 2024-08-06T20:31:17.8824509Z remote: Compressing objects: 60% (107/177) 2024-08-06T20:31:17.8825042Z remote: Compressing objects: 61% (108/177) 2024-08-06T20:31:17.8825562Z remote: Compressing objects: 62% (110/177) 2024-08-06T20:31:17.8826098Z remote: Compressing objects: 63% (112/177) 2024-08-06T20:31:17.8826626Z remote: Compressing objects: 64% (114/177) 2024-08-06T20:31:17.8827407Z remote: Compressing objects: 65% (116/177) 2024-08-06T20:31:17.8827938Z remote: Compressing objects: 66% (117/177) 2024-08-06T20:31:17.8828460Z remote: Compressing objects: 67% (119/177) 2024-08-06T20:31:17.8828953Z remote: Compressing objects: 68% (121/177) 2024-08-06T20:31:17.8829489Z remote: Compressing objects: 69% (123/177) 2024-08-06T20:31:17.8830018Z remote: Compressing objects: 70% (124/177) 2024-08-06T20:31:17.8830522Z remote: Compressing objects: 71% (126/177) 2024-08-06T20:31:17.8831088Z remote: Compressing objects: 72% (128/177) 2024-08-06T20:31:17.8831613Z remote: Compressing objects: 73% (130/177) 2024-08-06T20:31:17.8832110Z remote: Compressing objects: 74% (131/177) 2024-08-06T20:31:17.8832642Z remote: Compressing objects: 75% (133/177) 2024-08-06T20:31:17.8833163Z remote: Compressing objects: 76% (135/177) 2024-08-06T20:31:17.8833658Z remote: Compressing objects: 77% (137/177) 2024-08-06T20:31:17.8834199Z remote: Compressing objects: 78% (139/177) 2024-08-06T20:31:17.8834729Z remote: Compressing objects: 79% (140/177) 2024-08-06T20:31:17.8835223Z remote: Compressing objects: 80% (142/177) 2024-08-06T20:31:17.8835902Z remote: Compressing objects: 81% (144/177) 2024-08-06T20:31:17.8836476Z remote: Compressing objects: 82% (146/177) 2024-08-06T20:31:17.8836971Z remote: Compressing objects: 83% (147/177) 2024-08-06T20:31:17.8837865Z remote: Compressing objects: 84% (149/177) 2024-08-06T20:31:17.8838396Z remote: Compressing objects: 85% (151/177) 2024-08-06T20:31:17.8838895Z remote: Compressing objects: 86% (153/177) 2024-08-06T20:31:17.8839466Z remote: Compressing objects: 87% (154/177) 2024-08-06T20:31:17.8839999Z remote: Compressing objects: 88% (156/177) 2024-08-06T20:31:17.8840590Z remote: Compressing objects: 89% (158/177) 2024-08-06T20:31:17.8841126Z remote: Compressing objects: 90% (160/177) 2024-08-06T20:31:17.8841655Z remote: Compressing objects: 91% (162/177) 2024-08-06T20:31:17.8842194Z remote: Compressing objects: 92% (163/177) 2024-08-06T20:31:17.8842735Z remote: Compressing objects: 93% (165/177) 2024-08-06T20:31:17.8843244Z remote: Compressing objects: 94% (167/177) 2024-08-06T20:31:17.8843782Z remote: Compressing objects: 95% (169/177) 2024-08-06T20:31:17.8844316Z remote: Compressing objects: 96% (170/177) 2024-08-06T20:31:17.8844822Z remote: Compressing objects: 97% (172/177) 2024-08-06T20:31:17.8845355Z remote: Compressing objects: 98% (174/177) 2024-08-06T20:31:17.8845897Z remote: Compressing objects: 99% (176/177) 2024-08-06T20:31:17.8846411Z remote: Compressing objects: 100% (177/177) 2024-08-06T20:31:17.8846986Z remote: Compressing objects: 100% (177/177), done. 2024-08-06T20:31:44.2726978Z remote: Total 1008693 (delta 148), reused 259 (delta 131), pack-reused 1008383 2024-08-06T20:32:06.6884132Z [command]/usr/bin/git rev-parse --verify --quiet b9d86fa89636e301796d4201f36d86c73f6e49bc^{object} 2024-08-06T20:32:06.6912627Z b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:32:06.6918666Z ##[endgroup] 2024-08-06T20:32:06.6919253Z ##[group]Determining the checkout info 2024-08-06T20:32:06.6921399Z ##[endgroup] 2024-08-06T20:32:06.6921938Z ##[group]Checking out the ref 2024-08-06T20:32:06.6925967Z [command]/usr/bin/git checkout --quiet --force b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:32:08.5360752Z ##[endgroup] 2024-08-06T20:32:08.5361337Z ##[group]Setting up auth for fetching submodules 2024-08-06T20:32:08.5367369Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-06T20:32:08.5425847Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-06T20:32:08.5458054Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-06T20:32:08.5491089Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-06T20:32:08.5519938Z ##[endgroup] 2024-08-06T20:32:08.5524394Z ##[group]Fetching submodules 2024-08-06T20:32:08.5528532Z [command]/usr/bin/git submodule sync --recursive 2024-08-06T20:32:08.5899674Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-06T20:32:08.6146412Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-06T20:32:08.6148782Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-06T20:32:08.6152087Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-06T20:32:08.6155258Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-06T20:32:08.6160163Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-06T20:32:08.6162753Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-06T20:32:08.6166442Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-06T20:32:08.6170220Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-06T20:32:08.6174031Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-06T20:32:08.6178212Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-06T20:32:08.6182182Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-06T20:32:08.6186459Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-06T20:32:08.6190880Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-06T20:32:08.6195500Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-06T20:32:08.6200484Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-06T20:32:08.6205270Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-08-06T20:32:08.6210406Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:32:08.6215303Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-06T20:32:08.6220503Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-06T20:32:08.6225669Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-06T20:32:08.6231164Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-06T20:32:08.6236825Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-06T20:32:08.6243215Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-06T20:32:08.6248955Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-06T20:32:08.6254914Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-06T20:32:08.6261046Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-06T20:32:08.6267492Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-06T20:32:08.6273675Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-06T20:32:08.6280643Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-06T20:32:08.6287194Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-06T20:32:08.6294027Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-06T20:32:08.6300846Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-06T20:32:08.6307853Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-06T20:32:08.6315138Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-06T20:32:08.6322784Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-06T20:32:08.6349613Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-06T20:32:08.9668987Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-06T20:32:09.1563676Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-06T20:32:09.3256241Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-06T20:32:09.5719664Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-06T20:32:11.5463706Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-06T20:32:23.1040431Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-06T20:32:23.5214287Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-06T20:32:23.9395402Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-06T20:32:24.5592209Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-06T20:32:25.9966527Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-06T20:32:28.3308707Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-06T20:32:35.5335196Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-06T20:32:36.9344731Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-06T20:32:39.0852604Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-06T20:32:40.3658136Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-08-06T20:32:40.5253960Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-06T20:32:40.9726970Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-06T20:32:41.4453848Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-06T20:32:42.5645969Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-06T20:32:42.8967011Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-06T20:32:43.1235483Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-06T20:32:45.0055409Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-06T20:32:45.7650235Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-06T20:32:46.4133200Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-06T20:32:53.1881705Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-06T20:32:55.3101100Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-06T20:32:58.9840956Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-06T20:32:59.1902017Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-06T20:33:08.9543486Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-06T20:33:09.1115868Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-06T20:33:09.3014177Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-06T20:33:10.3607521Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-06T20:33:10.6471665Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-06T20:33:11.3505454Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-06T20:33:11.8276332Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-06T20:33:11.8400410Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-06T20:33:11.8492308Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-06T20:33:11.8762443Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-06T20:33:11.9212106Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-06T20:33:13.0689428Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-06T20:33:13.0950147Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-06T20:33:13.1493941Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-06T20:33:13.2693498Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-06T20:33:13.3079304Z Submodule path 'third_party/cudnn_frontend': checked out '98ca4e1941fe3263f128f74f10063a3ea35c7019' 2024-08-06T20:33:13.9505060Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-06T20:33:14.2628702Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-06T20:33:14.3591205Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-06T20:33:14.3608206Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:33:14.3611090Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:33:14.3614132Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:33:14.3617353Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:33:14.3620943Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:33:14.3647730Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-06T20:33:15.4598255Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-06T20:33:16.1190262Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-06T20:33:18.4629686Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-06T20:33:19.6031115Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-06T20:33:19.9774023Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-06T20:33:20.0964746Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-06T20:33:20.6099374Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-06T20:33:20.6847181Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-06T20:33:20.6978236Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-06T20:33:20.8341351Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-06T20:33:20.8819804Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-06T20:33:20.8918176Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-08-06T20:33:20.9370969Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-06T20:33:20.9661645Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-06T20:33:21.0206378Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-06T20:33:21.0343384Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-06T20:33:21.0359607Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-06T20:33:21.0384028Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-06T20:33:35.5510800Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-06T20:33:35.5711867Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-06T20:33:35.6833614Z Submodule path 'third_party/kineto': checked out 'da2f2682cabaf95d601fa2a9b7e0979f84fe7667' 2024-08-06T20:33:35.6852230Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:33:35.6855959Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:33:35.6860051Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:33:35.6886477Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-06T20:33:37.7633889Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-06T20:33:38.9709022Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-06T20:33:40.1919377Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-06T20:33:40.1935733Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:33:40.1938759Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:33:40.1941924Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:33:40.1945335Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:33:40.1948710Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:33:40.1952346Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:33:40.1956048Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:33:40.1960383Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:33:40.1987043Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-06T20:33:41.1885975Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-06T20:33:41.5535902Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-06T20:33:42.8009929Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-06T20:33:43.0933865Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-06T20:33:43.6962301Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-06T20:33:44.8669913Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-06T20:33:51.7032236Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-06T20:33:52.1133691Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-06T20:33:52.1335747Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-06T20:33:52.1754649Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-06T20:33:52.1894932Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-06T20:33:52.1910630Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:33:52.1938145Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-06T20:33:52.6076122Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-06T20:33:52.6279880Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-06T20:33:52.6746397Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-06T20:33:52.7977046Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-06T20:33:52.8153374Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-06T20:33:52.8604085Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-06T20:33:52.9293703Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-06T20:33:52.9733760Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-06T20:33:53.0006513Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-06T20:33:53.1329195Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-06T20:33:53.5552829Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-06T20:33:53.5589568Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-06T20:33:53.5592278Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-06T20:33:53.5619746Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-06T20:33:53.9796704Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-06T20:33:54.9364428Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-06T20:33:54.9756235Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-06T20:33:55.0570206Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-06T20:33:55.0582063Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:33:55.0584423Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:33:55.0587804Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:33:55.0591117Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:33:55.0594690Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:33:55.0598258Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:33:55.0602235Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:33:55.0605805Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:33:55.0631352Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-06T20:33:55.5064105Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-06T20:33:56.6670975Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-06T20:33:56.9939347Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-06T20:34:03.8539343Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-06T20:34:04.1127340Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-06T20:34:04.3005678Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-06T20:34:04.6048215Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-06T20:34:11.5498073Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-06T20:34:11.5972092Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-06T20:34:11.6144971Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-06T20:34:11.7368624Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-06T20:34:11.7509801Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-06T20:34:11.7669713Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-06T20:34:11.7836433Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-06T20:34:11.7852341Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:34:11.7855444Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:34:11.7881466Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-06T20:34:13.7779872Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-06T20:34:15.2305359Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-06T20:34:15.2842054Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-06T20:34:15.8323419Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-06T20:34:15.8447743Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-06T20:34:16.1690211Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-06T20:34:16.1712282Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:34:16.1714994Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-06T20:34:16.1741516Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-06T20:34:16.6054909Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-06T20:34:17.7627562Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-06T20:34:17.8500216Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-06T20:34:17.8592783Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-06T20:34:17.8727041Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-06T20:34:17.9153321Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-06T20:34:17.9483722Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-06T20:34:17.9974229Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-06T20:34:18.0263666Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-06T20:34:18.0281755Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:34:18.0284933Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:34:18.0287803Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:34:18.0291121Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:34:18.0316462Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-06T20:34:19.2246178Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-06T20:34:19.5005315Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-06T20:34:20.7775586Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-06T20:34:21.8618796Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-06T20:34:21.8784710Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-06T20:34:21.9601792Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-06T20:34:21.9926233Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-06T20:34:21.9942472Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:34:21.9969269Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-06T20:34:22.2368354Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-06T20:34:22.2403129Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-06T20:34:22.2716099Z Entering 'android/libs/fbjni' 2024-08-06T20:34:22.2758287Z Entering 'third_party/FP16' 2024-08-06T20:34:22.2798460Z Entering 'third_party/FXdiv' 2024-08-06T20:34:22.2839224Z Entering 'third_party/NNPACK' 2024-08-06T20:34:22.2880703Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:34:22.2922239Z Entering 'third_party/XNNPACK' 2024-08-06T20:34:22.2982567Z Entering 'third_party/benchmark' 2024-08-06T20:34:22.3023389Z Entering 'third_party/cpp-httplib' 2024-08-06T20:34:22.3063220Z Entering 'third_party/cpuinfo' 2024-08-06T20:34:22.3104871Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:34:22.3145285Z Entering 'third_party/cutlass' 2024-08-06T20:34:22.3195231Z Entering 'third_party/eigen' 2024-08-06T20:34:22.3239126Z Entering 'third_party/fbgemm' 2024-08-06T20:34:22.3280452Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:34:22.3320346Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:34:22.3360191Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:34:22.3406377Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:34:22.3445514Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:34:22.3485340Z Entering 'third_party/flatbuffers' 2024-08-06T20:34:22.3528292Z Entering 'third_party/fmt' 2024-08-06T20:34:22.3567840Z Entering 'third_party/foxi' 2024-08-06T20:34:22.3608063Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:34:22.3647973Z Entering 'third_party/gloo' 2024-08-06T20:34:22.3689297Z Entering 'third_party/googletest' 2024-08-06T20:34:22.3729911Z Entering 'third_party/ideep' 2024-08-06T20:34:22.3768281Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:34:22.3816567Z Entering 'third_party/ittapi' 2024-08-06T20:34:22.3858680Z Entering 'third_party/kineto' 2024-08-06T20:34:22.3900508Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:34:22.3940805Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:34:22.3983526Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:34:22.4023480Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:34:22.4064062Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:34:22.4104205Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:34:22.4145830Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:34:22.4185452Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:34:22.4226697Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:34:22.4266976Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:34:22.4307781Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:34:22.4347049Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:34:22.4386691Z Entering 'third_party/mimalloc' 2024-08-06T20:34:22.4427490Z Entering 'third_party/nccl/nccl' 2024-08-06T20:34:22.4467050Z Entering 'third_party/nlohmann' 2024-08-06T20:34:22.4509877Z Entering 'third_party/onnx' 2024-08-06T20:34:22.4567887Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:34:22.4609034Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:34:22.4651568Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:34:22.4693734Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:34:22.4733498Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:34:22.4774137Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:34:22.4813862Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:34:22.4854127Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:34:22.4893859Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:34:22.4934242Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:34:22.4972746Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:34:22.5014603Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:34:22.5056140Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:34:22.5120014Z Entering 'third_party/pocketfft' 2024-08-06T20:34:22.5161361Z Entering 'third_party/protobuf' 2024-08-06T20:34:22.5206726Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:34:22.5246739Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:34:22.5288161Z Entering 'third_party/psimd' 2024-08-06T20:34:22.5328865Z Entering 'third_party/pthreadpool' 2024-08-06T20:34:22.5369353Z Entering 'third_party/pybind11' 2024-08-06T20:34:22.5410329Z Entering 'third_party/python-peachpy' 2024-08-06T20:34:22.5452032Z Entering 'third_party/sleef' 2024-08-06T20:34:22.5493111Z Entering 'third_party/tensorpipe' 2024-08-06T20:34:22.5533813Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:34:22.5573194Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:34:22.5612746Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:34:22.5652088Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:34:22.5690558Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:34:22.5745008Z ##[endgroup] 2024-08-06T20:34:22.5747948Z ##[group]Persisting credentials for submodules 2024-08-06T20:34:22.5753127Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-06T20:34:22.6062622Z Entering 'android/libs/fbjni' 2024-08-06T20:34:22.6112974Z Entering 'third_party/FP16' 2024-08-06T20:34:22.6163351Z Entering 'third_party/FXdiv' 2024-08-06T20:34:22.6211965Z Entering 'third_party/NNPACK' 2024-08-06T20:34:22.6263003Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:34:22.6313149Z Entering 'third_party/XNNPACK' 2024-08-06T20:34:22.6382184Z Entering 'third_party/benchmark' 2024-08-06T20:34:22.6432937Z Entering 'third_party/cpp-httplib' 2024-08-06T20:34:22.6483374Z Entering 'third_party/cpuinfo' 2024-08-06T20:34:22.6532361Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:34:22.6583955Z Entering 'third_party/cutlass' 2024-08-06T20:34:22.6642725Z Entering 'third_party/eigen' 2024-08-06T20:34:22.6693455Z Entering 'third_party/fbgemm' 2024-08-06T20:34:22.6743826Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:34:22.6792600Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:34:22.6842366Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:34:22.6898521Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:34:22.6947542Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:34:22.6998310Z Entering 'third_party/flatbuffers' 2024-08-06T20:34:22.7050631Z Entering 'third_party/fmt' 2024-08-06T20:34:22.7101099Z Entering 'third_party/foxi' 2024-08-06T20:34:22.7150534Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:34:22.7201493Z Entering 'third_party/gloo' 2024-08-06T20:34:22.7250472Z Entering 'third_party/googletest' 2024-08-06T20:34:22.7301285Z Entering 'third_party/ideep' 2024-08-06T20:34:22.7349649Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:34:22.7405939Z Entering 'third_party/ittapi' 2024-08-06T20:34:22.7455418Z Entering 'third_party/kineto' 2024-08-06T20:34:22.7504998Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:34:22.7553915Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:34:22.7606588Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:34:22.7655686Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:34:22.7705132Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:34:22.7752873Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:34:22.7803252Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:34:22.7851988Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:34:22.7901421Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:34:22.7952553Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:34:22.8003904Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:34:22.8053085Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:34:22.8103004Z Entering 'third_party/mimalloc' 2024-08-06T20:34:22.8153942Z Entering 'third_party/nccl/nccl' 2024-08-06T20:34:22.8204195Z Entering 'third_party/nlohmann' 2024-08-06T20:34:22.8255620Z Entering 'third_party/onnx' 2024-08-06T20:34:22.8323040Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:34:22.8372159Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:34:22.8425455Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:34:22.8476904Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:34:22.8526445Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:34:22.8575531Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:34:22.8625406Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:34:22.8676030Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:34:22.8725382Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:34:22.8773972Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:34:22.8822056Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:34:22.8873926Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:34:22.8925361Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:34:22.8995915Z Entering 'third_party/pocketfft' 2024-08-06T20:34:22.9047649Z Entering 'third_party/protobuf' 2024-08-06T20:34:22.9104080Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:34:22.9152722Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:34:22.9205549Z Entering 'third_party/psimd' 2024-08-06T20:34:22.9256904Z Entering 'third_party/pthreadpool' 2024-08-06T20:34:22.9306494Z Entering 'third_party/pybind11' 2024-08-06T20:34:22.9357848Z Entering 'third_party/python-peachpy' 2024-08-06T20:34:22.9407008Z Entering 'third_party/sleef' 2024-08-06T20:34:22.9457367Z Entering 'third_party/tensorpipe' 2024-08-06T20:34:22.9507178Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:34:22.9556253Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:34:22.9605519Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:34:22.9654454Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:34:22.9702616Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:34:22.9769633Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-06T20:34:23.0075774Z Entering 'android/libs/fbjni' 2024-08-06T20:34:23.0123188Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-06T20:34:23.0140907Z Entering 'third_party/FP16' 2024-08-06T20:34:23.0188664Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-06T20:34:23.0207615Z Entering 'third_party/FXdiv' 2024-08-06T20:34:23.0254915Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-06T20:34:23.0272597Z Entering 'third_party/NNPACK' 2024-08-06T20:34:23.0319866Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-06T20:34:23.0338567Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:34:23.0385663Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-06T20:34:23.0404230Z Entering 'third_party/XNNPACK' 2024-08-06T20:34:23.0450292Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-06T20:34:23.0487705Z Entering 'third_party/benchmark' 2024-08-06T20:34:23.0533904Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-06T20:34:23.0551813Z Entering 'third_party/cpp-httplib' 2024-08-06T20:34:23.0598315Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-06T20:34:23.0618360Z Entering 'third_party/cpuinfo' 2024-08-06T20:34:23.0664559Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-06T20:34:23.0683405Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:34:23.0730717Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-06T20:34:23.0748494Z Entering 'third_party/cutlass' 2024-08-06T20:34:23.0795485Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-06T20:34:23.0821303Z Entering 'third_party/eigen' 2024-08-06T20:34:23.0867997Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-06T20:34:23.0889109Z Entering 'third_party/fbgemm' 2024-08-06T20:34:23.0935650Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-06T20:34:23.0953082Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:34:23.0999280Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-06T20:34:23.1016772Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:34:23.1062053Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-06T20:34:23.1080455Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:34:23.1125538Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-06T20:34:23.1149366Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:34:23.1196307Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:34:23.1213990Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:34:23.1260252Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-06T20:34:23.1278781Z Entering 'third_party/flatbuffers' 2024-08-06T20:34:23.1325107Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-06T20:34:23.1345910Z Entering 'third_party/fmt' 2024-08-06T20:34:23.1391997Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-06T20:34:23.1410054Z Entering 'third_party/foxi' 2024-08-06T20:34:23.1455795Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-08-06T20:34:23.1473740Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:34:23.1520576Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-06T20:34:23.1538313Z Entering 'third_party/gloo' 2024-08-06T20:34:23.1584129Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-06T20:34:23.1602847Z Entering 'third_party/googletest' 2024-08-06T20:34:23.1649292Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:34:23.1667529Z Entering 'third_party/ideep' 2024-08-06T20:34:23.1714040Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-06T20:34:23.1731097Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:34:23.1777098Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-06T20:34:23.1803002Z Entering 'third_party/ittapi' 2024-08-06T20:34:23.1849115Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-06T20:34:23.1866988Z Entering 'third_party/kineto' 2024-08-06T20:34:23.1913969Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-06T20:34:23.1933070Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:34:23.1979833Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-06T20:34:23.1996749Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:34:23.2045226Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-06T20:34:23.2063559Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:34:23.2110633Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-06T20:34:23.2129190Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:34:23.2175724Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-06T20:34:23.2193408Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:34:23.2240652Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-06T20:34:23.2258026Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:34:23.2304893Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-06T20:34:23.2325100Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:34:23.2371605Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-06T20:34:23.2389286Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:34:23.2437575Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:34:23.2455542Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:34:23.2501607Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-06T20:34:23.2521338Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:34:23.2567905Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-06T20:34:23.2586863Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:34:23.2633286Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-06T20:34:23.2650960Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:34:23.2696905Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-06T20:34:23.2716293Z Entering 'third_party/mimalloc' 2024-08-06T20:34:23.2763192Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-06T20:34:23.2781388Z Entering 'third_party/nccl/nccl' 2024-08-06T20:34:23.2827579Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-06T20:34:23.2846286Z Entering 'third_party/nlohmann' 2024-08-06T20:34:23.2892145Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-06T20:34:23.2911547Z Entering 'third_party/onnx' 2024-08-06T20:34:23.2957381Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-06T20:34:23.2992814Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:34:23.3040034Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-06T20:34:23.3057463Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:34:23.3104259Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-06T20:34:23.3124489Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:34:23.3171338Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-06T20:34:23.3190831Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:34:23.3236989Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-06T20:34:23.3254765Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:34:23.3300480Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:34:23.3318464Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:34:23.3364495Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-06T20:34:23.3381653Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:34:23.3428171Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-06T20:34:23.3447540Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:34:23.3493912Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-06T20:34:23.3511096Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:34:23.3557644Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-06T20:34:23.3575008Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:34:23.3620913Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-06T20:34:23.3637344Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:34:23.3684433Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-06T20:34:23.3703988Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:34:23.3749974Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-06T20:34:23.3769156Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:34:23.3815045Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-06T20:34:23.3855214Z Entering 'third_party/pocketfft' 2024-08-06T20:34:23.3901325Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-06T20:34:23.3919875Z Entering 'third_party/protobuf' 2024-08-06T20:34:23.3966016Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-06T20:34:23.3987113Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:34:23.4033297Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-06T20:34:23.4050887Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:34:23.4096813Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:34:23.4116209Z Entering 'third_party/psimd' 2024-08-06T20:34:23.4164198Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-06T20:34:23.4182029Z Entering 'third_party/pthreadpool' 2024-08-06T20:34:23.4229605Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-06T20:34:23.4247579Z Entering 'third_party/pybind11' 2024-08-06T20:34:23.4294310Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-06T20:34:23.4312554Z Entering 'third_party/python-peachpy' 2024-08-06T20:34:23.4359580Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-06T20:34:23.4377722Z Entering 'third_party/sleef' 2024-08-06T20:34:23.4423376Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-06T20:34:23.4442340Z Entering 'third_party/tensorpipe' 2024-08-06T20:34:23.4488404Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-06T20:34:23.4506017Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:34:23.4552350Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:34:23.4570624Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:34:23.4616789Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-06T20:34:23.4634131Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:34:23.4681127Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-06T20:34:23.4698796Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:34:23.4744306Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-06T20:34:23.4762179Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:34:23.4808945Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-06T20:34:23.5805986Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-06T20:34:23.6112712Z Entering 'android/libs/fbjni' 2024-08-06T20:34:23.6154109Z Entering 'third_party/FP16' 2024-08-06T20:34:23.6195998Z Entering 'third_party/FXdiv' 2024-08-06T20:34:23.6236802Z Entering 'third_party/NNPACK' 2024-08-06T20:34:23.6277737Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:34:23.6319794Z Entering 'third_party/XNNPACK' 2024-08-06T20:34:23.6378913Z Entering 'third_party/benchmark' 2024-08-06T20:34:23.6420605Z Entering 'third_party/cpp-httplib' 2024-08-06T20:34:23.6461824Z Entering 'third_party/cpuinfo' 2024-08-06T20:34:23.6503938Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:34:23.6544892Z Entering 'third_party/cutlass' 2024-08-06T20:34:23.6593892Z Entering 'third_party/eigen' 2024-08-06T20:34:23.6636830Z Entering 'third_party/fbgemm' 2024-08-06T20:34:23.6678167Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:34:23.6718323Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:34:23.6758332Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:34:23.6804837Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:34:23.6844607Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:34:23.6885917Z Entering 'third_party/flatbuffers' 2024-08-06T20:34:23.6929908Z Entering 'third_party/fmt' 2024-08-06T20:34:23.6971591Z Entering 'third_party/foxi' 2024-08-06T20:34:23.7012009Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:34:23.7054358Z Entering 'third_party/gloo' 2024-08-06T20:34:23.7097043Z Entering 'third_party/googletest' 2024-08-06T20:34:23.7137904Z Entering 'third_party/ideep' 2024-08-06T20:34:23.7177703Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:34:23.7224989Z Entering 'third_party/ittapi' 2024-08-06T20:34:23.7266005Z Entering 'third_party/kineto' 2024-08-06T20:34:23.7307371Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:34:23.7347902Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:34:23.7389513Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:34:23.7430426Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:34:23.7470647Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:34:23.7510566Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:34:23.7554486Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:34:23.7594122Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:34:23.7635051Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:34:23.7677882Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:34:23.7718983Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:34:23.7757885Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:34:23.7799206Z Entering 'third_party/mimalloc' 2024-08-06T20:34:23.7840483Z Entering 'third_party/nccl/nccl' 2024-08-06T20:34:23.7882008Z Entering 'third_party/nlohmann' 2024-08-06T20:34:23.7924019Z Entering 'third_party/onnx' 2024-08-06T20:34:23.7981751Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:34:23.8022312Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:34:23.8064460Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:34:23.8107178Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:34:23.8148452Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:34:23.8188230Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:34:23.8226977Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:34:23.8267564Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:34:23.8307471Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:34:23.8348132Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:34:23.8387487Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:34:23.8429827Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:34:23.8470799Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:34:23.8534167Z Entering 'third_party/pocketfft' 2024-08-06T20:34:23.8574786Z Entering 'third_party/protobuf' 2024-08-06T20:34:23.8618963Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:34:23.8658602Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:34:23.8700151Z Entering 'third_party/psimd' 2024-08-06T20:34:23.8739957Z Entering 'third_party/pthreadpool' 2024-08-06T20:34:23.8780356Z Entering 'third_party/pybind11' 2024-08-06T20:34:23.8820944Z Entering 'third_party/python-peachpy' 2024-08-06T20:34:23.8862213Z Entering 'third_party/sleef' 2024-08-06T20:34:23.8902740Z Entering 'third_party/tensorpipe' 2024-08-06T20:34:23.8943134Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:34:23.8984036Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:34:23.9023204Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:34:23.9062759Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:34:23.9101801Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:34:23.9159554Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-06T20:34:23.9468274Z Entering 'android/libs/fbjni' 2024-08-06T20:34:23.9509358Z Entering 'third_party/FP16' 2024-08-06T20:34:23.9551632Z Entering 'third_party/FXdiv' 2024-08-06T20:34:23.9591777Z Entering 'third_party/NNPACK' 2024-08-06T20:34:23.9634073Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:34:23.9676368Z Entering 'third_party/XNNPACK' 2024-08-06T20:34:23.9736353Z Entering 'third_party/benchmark' 2024-08-06T20:34:23.9777926Z Entering 'third_party/cpp-httplib' 2024-08-06T20:34:23.9818791Z Entering 'third_party/cpuinfo' 2024-08-06T20:34:23.9860670Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:34:23.9901332Z Entering 'third_party/cutlass' 2024-08-06T20:34:23.9950875Z Entering 'third_party/eigen' 2024-08-06T20:34:23.9994550Z Entering 'third_party/fbgemm' 2024-08-06T20:34:24.0035644Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:34:24.0075830Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:34:24.0116928Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:34:24.0164291Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:34:24.0204679Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:34:24.0246023Z Entering 'third_party/flatbuffers' 2024-08-06T20:34:24.0290216Z Entering 'third_party/fmt' 2024-08-06T20:34:24.0330461Z Entering 'third_party/foxi' 2024-08-06T20:34:24.0369967Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:34:24.0410832Z Entering 'third_party/gloo' 2024-08-06T20:34:24.0451094Z Entering 'third_party/googletest' 2024-08-06T20:34:24.0491886Z Entering 'third_party/ideep' 2024-08-06T20:34:24.0531507Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:34:24.0578527Z Entering 'third_party/ittapi' 2024-08-06T20:34:24.0619062Z Entering 'third_party/kineto' 2024-08-06T20:34:24.0660146Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:34:24.0702082Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:34:24.0742849Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:34:24.0783485Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:34:24.0824659Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:34:24.0864129Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:34:24.0906093Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:34:24.0946821Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:34:24.0986601Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:34:24.1027550Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:34:24.1069267Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:34:24.1111128Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:34:24.1151619Z Entering 'third_party/mimalloc' 2024-08-06T20:34:24.1192448Z Entering 'third_party/nccl/nccl' 2024-08-06T20:34:24.1233803Z Entering 'third_party/nlohmann' 2024-08-06T20:34:24.1278084Z Entering 'third_party/onnx' 2024-08-06T20:34:24.1336565Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:34:24.1377485Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:34:24.1420762Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:34:24.1463674Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:34:24.1504121Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:34:24.1543839Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:34:24.1584107Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:34:24.1626319Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:34:24.1666185Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:34:24.1706476Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:34:24.1745357Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:34:24.1787977Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:34:24.1830711Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:34:24.1890763Z Entering 'third_party/pocketfft' 2024-08-06T20:34:24.1930480Z Entering 'third_party/protobuf' 2024-08-06T20:34:24.1974631Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:34:24.2014934Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:34:24.2056583Z Entering 'third_party/psimd' 2024-08-06T20:34:24.2097769Z Entering 'third_party/pthreadpool' 2024-08-06T20:34:24.2138222Z Entering 'third_party/pybind11' 2024-08-06T20:34:24.2179435Z Entering 'third_party/python-peachpy' 2024-08-06T20:34:24.2220034Z Entering 'third_party/sleef' 2024-08-06T20:34:24.2260420Z Entering 'third_party/tensorpipe' 2024-08-06T20:34:24.2300821Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:34:24.2340143Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:34:24.2379300Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:34:24.2419105Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:34:24.2458766Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:34:24.2513360Z ##[endgroup] 2024-08-06T20:34:24.2555499Z [command]/usr/bin/git log -1 --format='%H' 2024-08-06T20:34:24.2581916Z 'b9d86fa89636e301796d4201f36d86c73f6e49bc' 2024-08-06T20:34:24.2766367Z Prepare all required actions 2024-08-06T20:34:24.2767038Z Getting action download info 2024-08-06T20:34:24.4354725Z ##[group]Run ./.github/actions/setup-linux 2024-08-06T20:34:24.4355149Z env: 2024-08-06T20:34:24.4355422Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:24.4355766Z ##[endgroup] 2024-08-06T20:34:24.4436635Z ##[group]Run set -euo pipefail 2024-08-06T20:34:24.4437340Z set -euo pipefail 2024-08-06T20:34:24.4437945Z function get_ec2_metadata() { 2024-08-06T20:34:24.4438416Z  # Pulled from instance metadata endpoint for EC2 2024-08-06T20:34:24.4439198Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-06T20:34:24.4439932Z  category=$1 2024-08-06T20:34:24.4440383Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-06T20:34:24.4440931Z  runner_name_str=i-01c85e67ad0689ec5 2024-08-06T20:34:24.4441442Z  if [[ -f /.inarc ]]; then 2024-08-06T20:34:24.4441883Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-06T20:34:24.4442381Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-06T20:34:24.4442968Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-06T20:34:24.4443488Z  else 2024-08-06T20:34:24.4443934Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-06T20:34:24.4444462Z  fi 2024-08-06T20:34:24.4444742Z } 2024-08-06T20:34:24.4445087Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-06T20:34:24.4445627Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-06T20:34:24.4446206Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-06T20:34:24.4446733Z echo "system info $(uname -a)" 2024-08-06T20:34:24.4454748Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:24.4455209Z env: 2024-08-06T20:34:24.4455507Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:24.4455854Z ##[endgroup] 2024-08-06T20:34:24.4596260Z ami-id: ami-0ce0c36d7a00b20e2 2024-08-06T20:34:24.4664222Z instance-id: i-01c85e67ad0689ec5 2024-08-06T20:34:24.4730260Z instance-type: g3.8xlarge 2024-08-06T20:34:24.4738586Z system info Linux ip-10-0-49-118.ec2.internal 4.14.336-257.562.amzn2.x86_64 #1 SMP Sat Feb 24 09:50:35 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-06T20:34:24.4997920Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-06T20:34:24.4998749Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-06T20:34:24.5006964Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:24.5007454Z env: 2024-08-06T20:34:24.5007751Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:24.5008086Z ##[endgroup] 2024-08-06T20:34:24.5414772Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-06T20:34:24.5415365Z if systemctl is-active --quiet docker; then 2024-08-06T20:34:24.5415861Z  echo "Docker daemon is running..."; 2024-08-06T20:34:24.5416285Z else 2024-08-06T20:34:24.5416726Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-06T20:34:24.5417259Z fi 2024-08-06T20:34:24.5424941Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:24.5425403Z env: 2024-08-06T20:34:24.5425719Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:24.5426068Z ##[endgroup] 2024-08-06T20:34:24.5472948Z Docker daemon is running... 2024-08-06T20:34:24.5833019Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-06T20:34:24.5833541Z with: 2024-08-06T20:34:24.5833828Z shell: bash 2024-08-06T20:34:24.5834134Z timeout_minutes: 5 2024-08-06T20:34:24.5834443Z max_attempts: 3 2024-08-06T20:34:24.5834764Z retry_wait_seconds: 30 2024-08-06T20:34:24.5836217Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-08-06T20:34:24.5837954Z polling_interval_seconds: 1 2024-08-06T20:34:24.5838317Z warning_on_retry: true 2024-08-06T20:34:24.5838668Z continue_on_error: false 2024-08-06T20:34:24.5839004Z env: 2024-08-06T20:34:24.5839271Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:24.5839802Z AWS_RETRY_MODE: standard 2024-08-06T20:34:24.5840148Z AWS_MAX_ATTEMPTS: 5 2024-08-06T20:34:24.5840471Z AWS_DEFAULT_REGION: us-east-1 2024-08-06T20:34:24.5840837Z ##[endgroup] 2024-08-06T20:34:25.5723954Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-06T20:34:25.5724733Z Configure a credential helper to remove this warning. See 2024-08-06T20:34:25.5725461Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-06T20:34:25.5725947Z 2024-08-06T20:34:25.5726115Z Login Succeeded 2024-08-06T20:34:25.6546883Z Command completed after 1 attempt(s). 2024-08-06T20:34:25.6610189Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-06T20:34:25.6610865Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-06T20:34:25.6611439Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-06T20:34:25.6619676Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:25.6620181Z env: 2024-08-06T20:34:25.6620473Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:25.6620808Z ##[endgroup] 2024-08-06T20:34:25.6703481Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-06T20:34:25.6704189Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-06T20:34:25.6704711Z # shellcheck disable=SC2046 2024-08-06T20:34:25.6705144Z docker stop $(docker ps -q) || true 2024-08-06T20:34:25.6705605Z # Prune all of the docker images 2024-08-06T20:34:25.6706017Z docker system prune -af 2024-08-06T20:34:25.6713400Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:25.6713880Z env: 2024-08-06T20:34:25.6714155Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:25.6714506Z ##[endgroup] 2024-08-06T20:34:25.7040466Z "docker stop" requires at least 1 argument. 2024-08-06T20:34:25.7040925Z See 'docker stop --help'. 2024-08-06T20:34:25.7041182Z 2024-08-06T20:34:25.7041422Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-06T20:34:25.7041766Z 2024-08-06T20:34:25.7041943Z Stop one or more running containers 2024-08-06T20:34:25.7223898Z Total reclaimed space: 0B 2024-08-06T20:34:25.7263096Z ##[group]Run set +e 2024-08-06T20:34:25.7263480Z set +e 2024-08-06T20:34:25.7263794Z set -x 2024-08-06T20:34:25.7264095Z  2024-08-06T20:34:25.7264420Z PT_DOMAIN=download.pytorch.org 2024-08-06T20:34:25.7265162Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-06T20:34:25.7266074Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-06T20:34:25.7266735Z # one is returned at random 2024-08-06T20:34:25.7267247Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-06T20:34:25.7267715Z  2024-08-06T20:34:25.7268031Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-06T20:34:25.7268599Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-06T20:34:25.7269269Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-06T20:34:25.7269760Z  2024-08-06T20:34:25.7270078Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-06T20:34:25.7270575Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-06T20:34:25.7271024Z  exit 1 2024-08-06T20:34:25.7271340Z  fi 2024-08-06T20:34:25.7271633Z fi 2024-08-06T20:34:25.7272090Z  2024-08-06T20:34:25.7272457Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-06T20:34:25.7272930Z  # Clean up any old records first 2024-08-06T20:34:25.7273378Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-06T20:34:25.7273801Z fi 2024-08-06T20:34:25.7274091Z  2024-08-06T20:34:25.7274481Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-06T20:34:25.7275148Z cat /etc/hosts 2024-08-06T20:34:25.7283283Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:25.7283767Z env: 2024-08-06T20:34:25.7284056Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:25.7284387Z ##[endgroup] 2024-08-06T20:34:25.7307624Z + PT_DOMAIN=download.pytorch.org 2024-08-06T20:34:25.7313164Z ++ dig -4 +short download.pytorch.org 2024-08-06T20:34:25.7313581Z ++ tail -n1 2024-08-06T20:34:25.7398432Z + RESOLVED_IP=18.160.10.76 2024-08-06T20:34:25.7398871Z + '[' -z 18.160.10.76 ']' 2024-08-06T20:34:25.7399494Z + grep -r download.pytorch.org /etc/hosts 2024-08-06T20:34:25.7408039Z 18.160.10.28 download.pytorch.org 2024-08-06T20:34:25.7409068Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-06T20:34:25.7528780Z + echo '18.160.10.76 download.pytorch.org' 2024-08-06T20:34:25.7529527Z + sudo tee -a /etc/hosts 2024-08-06T20:34:25.7619732Z 18.160.10.76 download.pytorch.org 2024-08-06T20:34:25.7635099Z + cat /etc/hosts 2024-08-06T20:34:25.7641932Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-06T20:34:25.7648066Z ::1 localhost6 localhost6.localdomain6 2024-08-06T20:34:25.7648542Z 18.160.10.76 download.pytorch.org 2024-08-06T20:34:25.7801779Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-06T20:34:25.7802370Z with: 2024-08-06T20:34:25.7803276Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:25.7804309Z docker-build-dir: .ci/docker 2024-08-06T20:34:25.7804691Z working-directory: . 2024-08-06T20:34:25.7805134Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:25.7805642Z force-push: false 2024-08-06T20:34:25.7805948Z env: 2024-08-06T20:34:25.7806213Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:25.7806574Z ##[endgroup] 2024-08-06T20:34:25.7834023Z ##[group]Run set -ex 2024-08-06T20:34:25.7834433Z set -ex 2024-08-06T20:34:25.7834740Z  2024-08-06T20:34:25.7835242Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-06T20:34:25.7836103Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-06T20:34:25.7836799Z # job could then download the pre-built image as usual 2024-08-06T20:34:25.7838547Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-06T20:34:25.7839151Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:25.7839724Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:25.7840225Z  2024-08-06T20:34:25.7840684Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-06T20:34:25.7841236Z  exit 0 2024-08-06T20:34:25.7841530Z else 2024-08-06T20:34:25.7841883Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:25.7842317Z fi 2024-08-06T20:34:25.7842590Z  2024-08-06T20:34:25.7843029Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-06T20:34:25.7843777Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-06T20:34:25.7844432Z  # use it as it is, but first let's extract the tag 2024-08-06T20:34:25.7845040Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-06T20:34:25.7845683Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:25.7846293Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:25.7846811Z else 2024-08-06T20:34:25.7847222Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-06T20:34:25.7847791Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:25.7848757Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:25.7849442Z fi 2024-08-06T20:34:25.7856978Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:25.7857469Z env: 2024-08-06T20:34:25.7857761Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:25.7858103Z REPO_NAME: pytorch 2024-08-06T20:34:25.7859036Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:25.7860032Z DOCKER_BUILD_DIR: .ci/docker 2024-08-06T20:34:25.7860496Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:25.7860989Z ##[endgroup] 2024-08-06T20:34:25.7884508Z + [[ ! -d .ci/docker ]] 2024-08-06T20:34:25.7884883Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-06T20:34:25.7885264Z + echo skip=false 2024-08-06T20:34:25.7886467Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-06T20:34:25.7891636Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:25.7892621Z ++ awk -F '[:,]' '{print $2}' 2024-08-06T20:34:25.7902032Z + DOCKER_TAG=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:25.7902591Z + echo docker-tag=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:25.7903923Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:25.7940421Z ##[group]Run set +e 2024-08-06T20:34:25.7940834Z set +e 2024-08-06T20:34:25.7941128Z set -x 2024-08-06T20:34:25.7941427Z  2024-08-06T20:34:25.7941711Z login() { 2024-08-06T20:34:25.7942310Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-06T20:34:25.7942957Z } 2024-08-06T20:34:25.7943243Z  2024-08-06T20:34:25.7943520Z retry () { 2024-08-06T20:34:25.7943882Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-06T20:34:25.7944290Z } 2024-08-06T20:34:25.7944559Z  2024-08-06T20:34:25.7944874Z retry login "${DOCKER_REGISTRY}" 2024-08-06T20:34:25.7945259Z  2024-08-06T20:34:25.7945703Z # Check if image already exists, if it does then skip building it 2024-08-06T20:34:25.7946325Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-06T20:34:25.7946795Z  exit 0 2024-08-06T20:34:25.7947087Z fi 2024-08-06T20:34:25.7947368Z  2024-08-06T20:34:25.7947802Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-06T20:34:25.7948524Z # be empty. The default action would be to continue rebuild the image 2024-08-06T20:34:25.7949180Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-06T20:34:25.7949760Z  # if we're on the base branch then use the parent commit 2024-08-06T20:34:25.7950273Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-06T20:34:25.7950687Z else 2024-08-06T20:34:25.7951098Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-06T20:34:25.7951705Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-06T20:34:25.7952169Z fi 2024-08-06T20:34:25.7952439Z  2024-08-06T20:34:25.7952752Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-06T20:34:25.7953219Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:25.7953632Z  2024-08-06T20:34:25.7954213Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-06T20:34:25.7955033Z  exit 0 2024-08-06T20:34:25.7955321Z fi 2024-08-06T20:34:25.7955608Z  2024-08-06T20:34:25.7956008Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-06T20:34:25.7956851Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-06T20:34:25.7957878Z  exit 1 2024-08-06T20:34:25.7958185Z fi 2024-08-06T20:34:25.7958472Z  2024-08-06T20:34:25.7958921Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-06T20:34:25.7959724Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-06T20:34:25.7960458Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-06T20:34:25.7961274Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-06T20:34:25.7962214Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-06T20:34:25.7962789Z fi 2024-08-06T20:34:25.7963058Z  2024-08-06T20:34:25.7963527Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-06T20:34:25.7970645Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:25.7971106Z env: 2024-08-06T20:34:25.7971398Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:25.7971759Z DOCKER_BUILD_DIR: .ci/docker 2024-08-06T20:34:25.7972183Z BASE_REVISION: 1736af7cf736184c356be1bb00f59fb2feea6d7d 2024-08-06T20:34:25.7973229Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:25.7974273Z DOCKER_TAG: 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:25.7974848Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:25.7975326Z ##[endgroup] 2024-08-06T20:34:25.7997822Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:25.7998390Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:25.8000548Z + aws ecr get-login-password --region us-east-1 2024-08-06T20:34:25.8001486Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:26.2642516Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-06T20:34:26.2643573Z Configure a credential helper to remove this warning. See 2024-08-06T20:34:26.2644540Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-06T20:34:26.2645254Z 2024-08-06T20:34:26.2645404Z Login Succeeded 2024-08-06T20:34:26.2657379Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:26.4687658Z { 2024-08-06T20:34:26.4688228Z "schemaVersion": 2, 2024-08-06T20:34:26.4688799Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-06T20:34:26.4689364Z "config": { 2024-08-06T20:34:26.4689817Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-06T20:34:26.4690321Z "size": 47454, 2024-08-06T20:34:26.4690842Z "digest": "sha256:8e4e19ae0377c273bc603b24bc49d406d6f755072d4534d040c86afbe761a129" 2024-08-06T20:34:26.4691427Z }, 2024-08-06T20:34:26.4691679Z "layers": [ 2024-08-06T20:34:26.4691959Z { 2024-08-06T20:34:26.4692396Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4692920Z "size": 28580681, 2024-08-06T20:34:26.4693472Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-08-06T20:34:26.4694113Z }, 2024-08-06T20:34:26.4694367Z { 2024-08-06T20:34:26.4694820Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4695378Z "size": 7943353, 2024-08-06T20:34:26.4695953Z "digest": "sha256:09d415c238d76b32a7ea4a6e6add9542db9a5641f7f183af70aae185d0709e58" 2024-08-06T20:34:26.4696865Z }, 2024-08-06T20:34:26.4697119Z { 2024-08-06T20:34:26.4697576Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4698226Z "size": 56226389, 2024-08-06T20:34:26.4698804Z "digest": "sha256:9fe6e2e61518cba6844870c03b285737daec35e62baf25ae7744629ed3a7b470" 2024-08-06T20:34:26.4699546Z }, 2024-08-06T20:34:26.4699982Z { 2024-08-06T20:34:26.4700570Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4701170Z "size": 187, 2024-08-06T20:34:26.4701810Z "digest": "sha256:41f16248e682693ff20b3032c1d5e5541cc87c5af898ae2ff9b24d2940e59100" 2024-08-06T20:34:26.4702529Z }, 2024-08-06T20:34:26.4702897Z { 2024-08-06T20:34:26.4703367Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4704000Z "size": 6885, 2024-08-06T20:34:26.4704641Z "digest": "sha256:95d7b781703928cf3c4eece39d800cccb76728c375fedf51ecd83833fb25e458" 2024-08-06T20:34:26.4705283Z }, 2024-08-06T20:34:26.4705648Z { 2024-08-06T20:34:26.4706190Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4706910Z "size": 1383662475, 2024-08-06T20:34:26.4707561Z "digest": "sha256:8f6c9048534734f4c873935293b7296225846ceb31c1a158400a67ea170dde7f" 2024-08-06T20:34:26.4708288Z }, 2024-08-06T20:34:26.4708590Z { 2024-08-06T20:34:26.4709108Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4709748Z "size": 62282, 2024-08-06T20:34:26.4710305Z "digest": "sha256:ab17245097e491b9368790714f9d90ed447bf0973bd677cfe6f2456d62b72a13" 2024-08-06T20:34:26.4728296Z }, 2024-08-06T20:34:26.4728592Z { 2024-08-06T20:34:26.4729036Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4729581Z "size": 1684, 2024-08-06T20:34:26.4730098Z "digest": "sha256:dfecd7e9912b76ed460b8edd5a85f1943666e38a973ab5458177cf2c7c3110e3" 2024-08-06T20:34:26.4730711Z }, 2024-08-06T20:34:26.4730971Z { 2024-08-06T20:34:26.4731379Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4731908Z "size": 1522, 2024-08-06T20:34:26.4732441Z "digest": "sha256:464a8f74544589bf7b57f9a4cadcb6681e5ed00758f6c35025e691df4e88e890" 2024-08-06T20:34:26.4733013Z }, 2024-08-06T20:34:26.4733390Z + exit 0 2024-08-06T20:34:26.4733664Z { 2024-08-06T20:34:26.4734073Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4734603Z "size": 2445884909, 2024-08-06T20:34:26.4735134Z "digest": "sha256:d67c111fa58886d8904335b5122039816c8e51af5844cb6fe4e14154d9a7c016" 2024-08-06T20:34:26.4735694Z }, 2024-08-06T20:34:26.4735954Z { 2024-08-06T20:34:26.4736372Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4736881Z "size": 85882, 2024-08-06T20:34:26.4737399Z "digest": "sha256:0723abde19c09431db80b5d0f0e003541f6232c2da5c4f35c635c861875937ae" 2024-08-06T20:34:26.4737990Z }, 2024-08-06T20:34:26.4738232Z { 2024-08-06T20:34:26.4738653Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4739176Z "size": 1820, 2024-08-06T20:34:26.4739689Z "digest": "sha256:ceceb076946ee95300822daccb5d3e4e45e6857d34996ef89305d9a078410f8e" 2024-08-06T20:34:26.4740274Z }, 2024-08-06T20:34:26.4740533Z { 2024-08-06T20:34:26.4740940Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4741468Z "size": 654860881, 2024-08-06T20:34:26.4741995Z "digest": "sha256:889eb8142c5187639e02cdc07adc75024843fd85a1f5c77d24c1a1123f5246bb" 2024-08-06T20:34:26.4742558Z }, 2024-08-06T20:34:26.4742819Z { 2024-08-06T20:34:26.4743243Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4743750Z "size": 542, 2024-08-06T20:34:26.4744276Z "digest": "sha256:af8081143f008c5cb1bd61f6d92fde22553cfc007b6a77c2fd2aaf6a9f0088a9" 2024-08-06T20:34:26.4744876Z }, 2024-08-06T20:34:26.4745313Z { 2024-08-06T20:34:26.4745743Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4746252Z "size": 1284, 2024-08-06T20:34:26.4746779Z "digest": "sha256:cc64a79090ad4059e2f3dbc1949b010b2733ab614bc9518ff6256ce2bed52115" 2024-08-06T20:34:26.4747367Z }, 2024-08-06T20:34:26.4747608Z { 2024-08-06T20:34:26.4748027Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4748547Z "size": 483, 2024-08-06T20:34:26.4749037Z "digest": "sha256:ad26542e0d01609840496a12691483f2c50eb8e23002a205becb5b695f9159fb" 2024-08-06T20:34:26.4749605Z }, 2024-08-06T20:34:26.4749866Z { 2024-08-06T20:34:26.4750270Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4750795Z "size": 91711921, 2024-08-06T20:34:26.4751328Z "digest": "sha256:7b42d8ae23671f21f6523a7e71cc50d5f87c4e993027a9bd924985cda6b57de9" 2024-08-06T20:34:26.4751895Z }, 2024-08-06T20:34:26.4752154Z { 2024-08-06T20:34:26.4752582Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4753095Z "size": 3229, 2024-08-06T20:34:26.4753732Z "digest": "sha256:6d46e3b97d9fd2bd0e7ceec44e4884756b26c594a1c1c51d1426ddd13f749db5" 2024-08-06T20:34:26.4754347Z }, 2024-08-06T20:34:26.4754586Z { 2024-08-06T20:34:26.4755006Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4755526Z "size": 1907, 2024-08-06T20:34:26.4756025Z "digest": "sha256:91f9dddffbee46981d9e74382704c1b53c87f1808350cf1dd518687805d3aa59" 2024-08-06T20:34:26.4756608Z }, 2024-08-06T20:34:26.4756868Z { 2024-08-06T20:34:26.4757822Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4758362Z "size": 699, 2024-08-06T20:34:26.4758887Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:34:26.4759465Z }, 2024-08-06T20:34:26.4759720Z { 2024-08-06T20:34:26.4760144Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4760672Z "size": 2813413845, 2024-08-06T20:34:26.4761236Z "digest": "sha256:eeedca475f8cafec096dfa0abc5c6f08938eb47c5d9d96fb001fd7f13a2531ca" 2024-08-06T20:34:26.4761848Z }, 2024-08-06T20:34:26.4762093Z { 2024-08-06T20:34:26.4762519Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4763026Z "size": 378, 2024-08-06T20:34:26.4763552Z "digest": "sha256:202c71391ade27762d5e9cafde793db67b4d5e4d6869debe2643d5092101c461" 2024-08-06T20:34:26.4764135Z }, 2024-08-06T20:34:26.4764379Z { 2024-08-06T20:34:26.4764805Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4765328Z "size": 12893, 2024-08-06T20:34:26.4765835Z "digest": "sha256:5af882ef27a2db3ee5e89c8730de161c0718b951f3f8477e63fe21517c9167f7" 2024-08-06T20:34:26.4766419Z }, 2024-08-06T20:34:26.4766677Z { 2024-08-06T20:34:26.4767083Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4767623Z "size": 802, 2024-08-06T20:34:26.4768149Z "digest": "sha256:bac001ac2fee5df3367ff9332bc1fcd706a8de3ad05a1885903f338197619520" 2024-08-06T20:34:26.4768726Z }, 2024-08-06T20:34:26.4768990Z { 2024-08-06T20:34:26.4769418Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4769925Z "size": 106, 2024-08-06T20:34:26.4770442Z "digest": "sha256:05896f1d52f7a8e2c4a53777e5575da3dc8fca8a786d14075b5bb204bcb4cf04" 2024-08-06T20:34:26.4771028Z }, 2024-08-06T20:34:26.4771270Z { 2024-08-06T20:34:26.4771691Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4772210Z "size": 502, 2024-08-06T20:34:26.4772705Z "digest": "sha256:49a0e418a175ac12f1b4b503e80e22679bc321e6163ed5b203af848204f0606a" 2024-08-06T20:34:26.4773285Z }, 2024-08-06T20:34:26.4773544Z { 2024-08-06T20:34:26.4773947Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4774475Z "size": 121477364, 2024-08-06T20:34:26.4775171Z "digest": "sha256:b5632121ff7a146647683888294bcca31252dd90ca681099dfb81d75c4710d52" 2024-08-06T20:34:26.4775733Z }, 2024-08-06T20:34:26.4775992Z { 2024-08-06T20:34:26.4776419Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4776931Z "size": 109, 2024-08-06T20:34:26.4777442Z "digest": "sha256:deceb46806b535404e025b9b532009eb211e79f85021171997af825253327d1a" 2024-08-06T20:34:26.4778067Z }, 2024-08-06T20:34:26.4778344Z { 2024-08-06T20:34:26.4778761Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4779290Z "size": 487, 2024-08-06T20:34:26.4779814Z "digest": "sha256:a2630e5fe166ae8ff146e4c14db0bdc1299ed2b6120108351c961adcd2375288" 2024-08-06T20:34:26.4780386Z }, 2024-08-06T20:34:26.4780647Z { 2024-08-06T20:34:26.4781051Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4781577Z "size": 297, 2024-08-06T20:34:26.4782098Z "digest": "sha256:603261fb379fff6cf0eb63de1787e7c98932f81e44e6af3f1645c9a977e0ca8c" 2024-08-06T20:34:26.4782678Z }, 2024-08-06T20:34:26.4782941Z { 2024-08-06T20:34:26.4783478Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4784008Z "size": 103, 2024-08-06T20:34:26.4784535Z "digest": "sha256:d8ac71337d62da7c4624f267380da27d834fcf174b873c8c53a963e6cc7193a4" 2024-08-06T20:34:26.4785124Z }, 2024-08-06T20:34:26.4785367Z { 2024-08-06T20:34:26.4785802Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4786308Z "size": 1470, 2024-08-06T20:34:26.4786818Z "digest": "sha256:2c2e35b979f5fc0ecccef6efc157a97f33db01a77958fa33855dc367544a5c46" 2024-08-06T20:34:26.4787412Z }, 2024-08-06T20:34:26.4787675Z { 2024-08-06T20:34:26.4788079Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4788606Z "size": 424146101, 2024-08-06T20:34:26.4789142Z "digest": "sha256:e559222c0ec62d839f78e4215ae553de028be354b7031e7393d1c6be6e91fd70" 2024-08-06T20:34:26.4789718Z }, 2024-08-06T20:34:26.4789978Z { 2024-08-06T20:34:26.4790404Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4790917Z "size": 159, 2024-08-06T20:34:26.4791444Z "digest": "sha256:c4dcb7e9d189eca2e414c757b5cf6d158055b29b0e71dbd082a5fb3585dba665" 2024-08-06T20:34:26.4792038Z }, 2024-08-06T20:34:26.4792277Z { 2024-08-06T20:34:26.4792699Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4793223Z "size": 563, 2024-08-06T20:34:26.4793720Z "digest": "sha256:2b9731d295d60278e361d491fc274e5d558a89ec5adeb9fdf342d7a8cc17bfec" 2024-08-06T20:34:26.4794301Z }, 2024-08-06T20:34:26.4794558Z { 2024-08-06T20:34:26.4794960Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4795485Z "size": 35807525, 2024-08-06T20:34:26.4796005Z "digest": "sha256:19da43312ebac2831028913760327e53d9812e80ab6b806cd7a920c549a3b554" 2024-08-06T20:34:26.4796572Z }, 2024-08-06T20:34:26.4796830Z { 2024-08-06T20:34:26.4797720Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4798258Z "size": 104, 2024-08-06T20:34:26.4798761Z "digest": "sha256:b846263d066405189d21552b7288e3efbc47e6cd2f236aa39f39d88ca576a33a" 2024-08-06T20:34:26.4799332Z }, 2024-08-06T20:34:26.4799613Z { 2024-08-06T20:34:26.4800021Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4800531Z "size": 424, 2024-08-06T20:34:26.4801044Z "digest": "sha256:3886e8f57dc887a87bb710fbda080ef4fd521f1ed2e560a895b000c05f6e9b0a" 2024-08-06T20:34:26.4801605Z }, 2024-08-06T20:34:26.4801862Z { 2024-08-06T20:34:26.4802269Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4802795Z "size": 20262048, 2024-08-06T20:34:26.4803329Z "digest": "sha256:58e13e7b6d7b78400d84805c843f08fb8f1d6583bccdb3373c8122b317e4d15a" 2024-08-06T20:34:26.4803900Z }, 2024-08-06T20:34:26.4804303Z { 2024-08-06T20:34:26.4804731Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4805239Z "size": 438, 2024-08-06T20:34:26.4805767Z "digest": "sha256:0eb80a0ac0006a45aa6b4581263cc1613ddf9545bd4a1d5a544310efe898ad81" 2024-08-06T20:34:26.4806356Z }, 2024-08-06T20:34:26.4806599Z { 2024-08-06T20:34:26.4807025Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4807551Z "size": 699, 2024-08-06T20:34:26.4808063Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:34:26.4808658Z }, 2024-08-06T20:34:26.4808924Z { 2024-08-06T20:34:26.4809326Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4809848Z "size": 141, 2024-08-06T20:34:26.4810368Z "digest": "sha256:a9f11b2b3622f5530c5cfefe973e96e028323fab1ff02e7a6337eb86fadebeb5" 2024-08-06T20:34:26.4810942Z }, 2024-08-06T20:34:26.4811200Z { 2024-08-06T20:34:26.4811629Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4812145Z "size": 135, 2024-08-06T20:34:26.4812755Z "digest": "sha256:d7e94279f8b2fe549e3f23637275d9dca8c831563ce84f1d44117eb297ddc871" 2024-08-06T20:34:26.4813349Z }, 2024-08-06T20:34:26.4813588Z { 2024-08-06T20:34:26.4814004Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4814524Z "size": 32, 2024-08-06T20:34:26.4815033Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:34:26.4815621Z }, 2024-08-06T20:34:26.4815877Z { 2024-08-06T20:34:26.4816281Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4816804Z "size": 190, 2024-08-06T20:34:26.4817324Z "digest": "sha256:0a3560fcd26c387cb3ff0638718fa9da02d3b08ad75b23c434b06d6a6d21cb55" 2024-08-06T20:34:26.4817896Z }, 2024-08-06T20:34:26.4818153Z { 2024-08-06T20:34:26.4818557Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4819090Z "size": 562, 2024-08-06T20:34:26.4819611Z "digest": "sha256:adf9d181cfbca5b0544f23797995d4ea6500cc4a635db9f7e18954c9044e9f4a" 2024-08-06T20:34:26.4820180Z }, 2024-08-06T20:34:26.4820450Z { 2024-08-06T20:34:26.4820873Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4821369Z "size": 43163050, 2024-08-06T20:34:26.4821889Z "digest": "sha256:bc18dd8315979255e24cf91e83757510baeb22f29a64d7579b02aca642215b58" 2024-08-06T20:34:26.4822465Z }, 2024-08-06T20:34:26.4822708Z { 2024-08-06T20:34:26.4823133Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4823655Z "size": 106, 2024-08-06T20:34:26.4824162Z "digest": "sha256:d62f2a6086195c833ef952a43eed18216c6cde79b07f8deeb4fb3aeea648f868" 2024-08-06T20:34:26.4824754Z }, 2024-08-06T20:34:26.4825013Z { 2024-08-06T20:34:26.4825419Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4825999Z "size": 1210, 2024-08-06T20:34:26.4826528Z "digest": "sha256:719a1c2abcfd7e4113aa2f03adeb54350a5e381c720688ea53bc0e18401ad33d" 2024-08-06T20:34:26.4827101Z }, 2024-08-06T20:34:26.4827359Z { 2024-08-06T20:34:26.4827787Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4828295Z "size": 699, 2024-08-06T20:34:26.4828818Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:34:26.4829412Z }, 2024-08-06T20:34:26.4829654Z { 2024-08-06T20:34:26.4830077Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4830601Z "size": 138, 2024-08-06T20:34:26.4831090Z "digest": "sha256:8e1355d516c5e0527c25402c664fc648a8a2b814090e6ee0fdd60f49a8826ac1" 2024-08-06T20:34:26.4831663Z }, 2024-08-06T20:34:26.4831920Z { 2024-08-06T20:34:26.4832316Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4832839Z "size": 117, 2024-08-06T20:34:26.4833441Z "digest": "sha256:701898e66f54fcf3b57d837ab651a876666391d9f9b51b9969ec842a23fd86f0" 2024-08-06T20:34:26.4833999Z }, 2024-08-06T20:34:26.4834254Z { 2024-08-06T20:34:26.4834682Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4835194Z "size": 1916643487, 2024-08-06T20:34:26.4835734Z "digest": "sha256:129b38cfed12360bb7c4d882141b5fb7f7337791383b9f75e4ffed521548f060" 2024-08-06T20:34:26.4836319Z }, 2024-08-06T20:34:26.4836561Z { 2024-08-06T20:34:26.4836983Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4838001Z "size": 172, 2024-08-06T20:34:26.4838500Z "digest": "sha256:1751651a072f518b34289de9dfa631c4e82c7958a5331061379684f9b9b11720" 2024-08-06T20:34:26.4839069Z }, 2024-08-06T20:34:26.4839312Z { 2024-08-06T20:34:26.4839734Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4840258Z "size": 905, 2024-08-06T20:34:26.4840771Z "digest": "sha256:fc956bf4e5b49119cab7cc7bda867c61bed76c76c1e113dfa7c7b184251f8173" 2024-08-06T20:34:26.4841381Z }, 2024-08-06T20:34:26.4841641Z { 2024-08-06T20:34:26.4842049Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4842692Z "size": 699, 2024-08-06T20:34:26.4843237Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:34:26.4843817Z }, 2024-08-06T20:34:26.4844077Z { 2024-08-06T20:34:26.4844501Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4845010Z "size": 134, 2024-08-06T20:34:26.4845522Z "digest": "sha256:874491333fb5b81d9cbbd71520163a6baf07e6f011dc148b1634b94e301972b8" 2024-08-06T20:34:26.4846105Z }, 2024-08-06T20:34:26.4846347Z { 2024-08-06T20:34:26.4846769Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4847295Z "size": 32, 2024-08-06T20:34:26.4847798Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:34:26.4848394Z }, 2024-08-06T20:34:26.4848645Z { 2024-08-06T20:34:26.4849052Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4849576Z "size": 156, 2024-08-06T20:34:26.4850113Z "digest": "sha256:365b4a0eec21fb9dfc58baf7b43a229ae3f8b638b17e9b4db5ee1cfaecb82de4" 2024-08-06T20:34:26.4850698Z }, 2024-08-06T20:34:26.4850953Z { 2024-08-06T20:34:26.4851375Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4851884Z "size": 1839, 2024-08-06T20:34:26.4852401Z "digest": "sha256:c1d4cac78640fc7d364011da807a363d6b248a2c46ccf99a6e1488c4974c17e3" 2024-08-06T20:34:26.4852984Z }, 2024-08-06T20:34:26.4853228Z { 2024-08-06T20:34:26.4853647Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4854171Z "size": 7529788, 2024-08-06T20:34:26.4854679Z "digest": "sha256:29afe2181ca6174db390fd0d9e8c4b8675fb99bb961c93936d4080ae92c8af57" 2024-08-06T20:34:26.4855279Z }, 2024-08-06T20:34:26.4855518Z { 2024-08-06T20:34:26.4855939Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4856469Z "size": 161, 2024-08-06T20:34:26.4856965Z "digest": "sha256:190dccfa687dc1e698f27965d8065a183168d7c66361505f368765456ff9f4bc" 2024-08-06T20:34:26.4857544Z }, 2024-08-06T20:34:26.4857801Z { 2024-08-06T20:34:26.4858207Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4858730Z "size": 7942, 2024-08-06T20:34:26.4859261Z "digest": "sha256:b01bea4b6634f47c51ede2b4a95ceade4e61356ca29551168b493ae6d4492f44" 2024-08-06T20:34:26.4859836Z }, 2024-08-06T20:34:26.4860095Z { 2024-08-06T20:34:26.4860514Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4861019Z "size": 8066, 2024-08-06T20:34:26.4861555Z "digest": "sha256:f13a9cc5968861df73c96e4ad67eef7f58576eeba8cdbee3d39e42ed3d6fe96d" 2024-08-06T20:34:26.4862150Z }, 2024-08-06T20:34:26.4862508Z { 2024-08-06T20:34:26.4862932Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4863457Z "size": 302, 2024-08-06T20:34:26.4863967Z "digest": "sha256:96d6dcd01b9f6c5557ba405d028442d808dd329f4f93e75f2c3ba0554e703a52" 2024-08-06T20:34:26.4864555Z }, 2024-08-06T20:34:26.4864813Z { 2024-08-06T20:34:26.4865217Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4865746Z "size": 7629762, 2024-08-06T20:34:26.4866267Z "digest": "sha256:fe134554f1b8a0509798c1f49e6f973f9b8b7f954eb1fda146f35a37c9c71599" 2024-08-06T20:34:26.4866832Z }, 2024-08-06T20:34:26.4867093Z { 2024-08-06T20:34:26.4867517Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4868026Z "size": 108, 2024-08-06T20:34:26.4868550Z "digest": "sha256:8068d364963fe87857fbb7c3aad42ddee962351a57eddc01b4282be12c69a074" 2024-08-06T20:34:26.4869133Z }, 2024-08-06T20:34:26.4869374Z { 2024-08-06T20:34:26.4869801Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4870335Z "size": 54145777, 2024-08-06T20:34:26.4870850Z "digest": "sha256:f307f545c1a5e086f18e23247c890ddd5b135b1cb7157807aeeaef38ceb3b7dc" 2024-08-06T20:34:26.4871530Z }, 2024-08-06T20:34:26.4871802Z { 2024-08-06T20:34:26.4872211Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4872738Z "size": 471, 2024-08-06T20:34:26.4873232Z "digest": "sha256:51c0b7eab50584e37e08f303a4bf3ba3141998800971452b2024a8b897858877" 2024-08-06T20:34:26.4873845Z }, 2024-08-06T20:34:26.4874122Z { 2024-08-06T20:34:26.4874566Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4875103Z "size": 1373229322, 2024-08-06T20:34:26.4875623Z "digest": "sha256:60aba96f31c695ec1e1433578783bab1257233794a6e8b5cc5c6365fdf68669e" 2024-08-06T20:34:26.4876202Z }, 2024-08-06T20:34:26.4876458Z { 2024-08-06T20:34:26.4876863Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4877876Z "size": 106, 2024-08-06T20:34:26.4878400Z "digest": "sha256:46db894a106ccd6c28198b9236ee84ac2973b37ae608caeb380dfaef91f13961" 2024-08-06T20:34:26.4878970Z }, 2024-08-06T20:34:26.4879232Z { 2024-08-06T20:34:26.4879655Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4880163Z "size": 557, 2024-08-06T20:34:26.4880662Z "digest": "sha256:0052583728e40920c255902250fe4a615b10ff00a5c0e7ed51a0075121d42fad" 2024-08-06T20:34:26.4881228Z }, 2024-08-06T20:34:26.4881468Z { 2024-08-06T20:34:26.4881891Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4882418Z "size": 36532090, 2024-08-06T20:34:26.4882946Z "digest": "sha256:8c6d352a3b51edbb546310cc3d97ded6aa8fd29a7d610e4d13fc54b5828fdd05" 2024-08-06T20:34:26.4883535Z }, 2024-08-06T20:34:26.4883777Z { 2024-08-06T20:34:26.4884202Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4884736Z "size": 111, 2024-08-06T20:34:26.4885248Z "digest": "sha256:380c4e16278f4e3bf0eb6aee3ff4a730e3a1adb82aeb1cfb8e4114f1759a2bcf" 2024-08-06T20:34:26.4885852Z }, 2024-08-06T20:34:26.4886112Z { 2024-08-06T20:34:26.4886533Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4887069Z "size": 32, 2024-08-06T20:34:26.4887590Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:34:26.4888166Z }, 2024-08-06T20:34:26.4888429Z { 2024-08-06T20:34:26.4888853Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4889362Z "size": 32, 2024-08-06T20:34:26.4889884Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:34:26.4890477Z }, 2024-08-06T20:34:26.4890721Z { 2024-08-06T20:34:26.4891144Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4891669Z "size": 32, 2024-08-06T20:34:26.4892306Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:34:26.4892899Z }, 2024-08-06T20:34:26.4893158Z { 2024-08-06T20:34:26.4893576Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:34:26.4894102Z "size": 32, 2024-08-06T20:34:26.4894630Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:34:26.4895211Z } 2024-08-06T20:34:26.4895476Z ] 2024-08-06T20:34:26.4895733Z } 2024-08-06T20:34:26.5042480Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-06T20:34:26.5043045Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-06T20:34:26.5043563Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-06T20:34:26.5051817Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:26.5052282Z env: 2024-08-06T20:34:26.5052569Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:26.5053519Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:26.5054503Z ##[endgroup] 2024-08-06T20:34:26.5081323Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9-02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:26.5136708Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-06T20:34:26.5137291Z with: 2024-08-06T20:34:26.5138168Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:26.5139241Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:26.5139739Z env: 2024-08-06T20:34:26.5140024Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:26.5140356Z ##[endgroup] 2024-08-06T20:34:26.5164741Z ##[group]Run set -x 2024-08-06T20:34:26.5165099Z set -x 2024-08-06T20:34:26.5165386Z set +e 2024-08-06T20:34:26.5165690Z  2024-08-06T20:34:26.5165983Z login() { 2024-08-06T20:34:26.5166562Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-06T20:34:26.5167221Z } 2024-08-06T20:34:26.5167504Z  2024-08-06T20:34:26.5167833Z retry () { 2024-08-06T20:34:26.5168182Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-06T20:34:26.5168594Z } 2024-08-06T20:34:26.5168873Z  2024-08-06T20:34:26.5169167Z retry login "${DOCKER_REGISTRY}" 2024-08-06T20:34:26.5169565Z  2024-08-06T20:34:26.5169843Z set -e 2024-08-06T20:34:26.5170265Z # ignore output since only exit code is used for conditional 2024-08-06T20:34:26.5170879Z # only pull docker image if it's not available locally 2024-08-06T20:34:26.5171569Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-06T20:34:26.5172172Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-06T20:34:26.5172598Z fi 2024-08-06T20:34:26.5179544Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:34:26.5180010Z env: 2024-08-06T20:34:26.5180302Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:34:26.5181243Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:26.5182339Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:26.5182817Z ##[endgroup] 2024-08-06T20:34:26.5204990Z + set +e 2024-08-06T20:34:26.5205648Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:26.5206208Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:26.5208414Z + aws ecr get-login-password --region us-east-1 2024-08-06T20:34:26.5209705Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:34:26.9784466Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-06T20:34:26.9785552Z Configure a credential helper to remove this warning. See 2024-08-06T20:34:26.9786484Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-06T20:34:26.9786996Z 2024-08-06T20:34:26.9787160Z Login Succeeded 2024-08-06T20:34:26.9799261Z + set -e 2024-08-06T20:34:26.9800452Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:26.9971884Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:26.9973449Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:34:27.2328904Z 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9: Pulling from pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9 2024-08-06T20:34:27.2330053Z 7a2c55901189: Pulling fs layer 2024-08-06T20:34:27.2330591Z 09d415c238d7: Pulling fs layer 2024-08-06T20:34:27.2331299Z 9fe6e2e61518: Pulling fs layer 2024-08-06T20:34:27.2331998Z 41f16248e682: Pulling fs layer 2024-08-06T20:34:27.2333016Z 95d7b7817039: Pulling fs layer 2024-08-06T20:34:27.2333746Z 8f6c90485347: Pulling fs layer 2024-08-06T20:34:27.2334349Z ab17245097e4: Pulling fs layer 2024-08-06T20:34:27.2334725Z dfecd7e9912b: Pulling fs layer 2024-08-06T20:34:27.2335173Z 464a8f745445: Pulling fs layer 2024-08-06T20:34:27.2335886Z d67c111fa588: Pulling fs layer 2024-08-06T20:34:27.2336446Z 0723abde19c0: Pulling fs layer 2024-08-06T20:34:27.2336800Z ceceb076946e: Pulling fs layer 2024-08-06T20:34:27.2337166Z 889eb8142c51: Pulling fs layer 2024-08-06T20:34:27.2337651Z af8081143f00: Pulling fs layer 2024-08-06T20:34:27.2338179Z 41f16248e682: Waiting 2024-08-06T20:34:27.2338740Z cc64a79090ad: Pulling fs layer 2024-08-06T20:34:27.2339134Z ad26542e0d01: Pulling fs layer 2024-08-06T20:34:27.2339504Z 7b42d8ae2367: Pulling fs layer 2024-08-06T20:34:27.2340194Z 6d46e3b97d9f: Pulling fs layer 2024-08-06T20:34:27.2340867Z 95d7b7817039: Waiting 2024-08-06T20:34:27.2341375Z 91f9dddffbee: Pulling fs layer 2024-08-06T20:34:27.2342022Z 03fb78f207e3: Pulling fs layer 2024-08-06T20:34:27.2342484Z 8f6c90485347: Waiting 2024-08-06T20:34:27.2342820Z eeedca475f8c: Pulling fs layer 2024-08-06T20:34:27.2343175Z 202c71391ade: Pulling fs layer 2024-08-06T20:34:27.2343530Z ab17245097e4: Waiting 2024-08-06T20:34:27.2343860Z 5af882ef27a2: Pulling fs layer 2024-08-06T20:34:27.2344211Z bac001ac2fee: Pulling fs layer 2024-08-06T20:34:27.2344574Z 05896f1d52f7: Pulling fs layer 2024-08-06T20:34:27.2344914Z dfecd7e9912b: Waiting 2024-08-06T20:34:27.2345244Z 49a0e418a175: Pulling fs layer 2024-08-06T20:34:27.2345597Z 889eb8142c51: Waiting 2024-08-06T20:34:27.2345916Z 464a8f745445: Waiting 2024-08-06T20:34:27.2346432Z ceceb076946e: Waiting 2024-08-06T20:34:27.2346755Z d67c111fa588: Waiting 2024-08-06T20:34:27.2347069Z 7b42d8ae2367: Waiting 2024-08-06T20:34:27.2347389Z 0723abde19c0: Waiting 2024-08-06T20:34:27.2347718Z b5632121ff7a: Pulling fs layer 2024-08-06T20:34:27.2348056Z ad26542e0d01: Waiting 2024-08-06T20:34:27.2348370Z 6d46e3b97d9f: Waiting 2024-08-06T20:34:27.2348686Z af8081143f00: Waiting 2024-08-06T20:34:27.2348999Z deceb46806b5: Pulling fs layer 2024-08-06T20:34:27.2349360Z 91f9dddffbee: Waiting 2024-08-06T20:34:27.2349690Z a2630e5fe166: Pulling fs layer 2024-08-06T20:34:27.2350029Z 5af882ef27a2: Waiting 2024-08-06T20:34:27.2350342Z 49a0e418a175: Waiting 2024-08-06T20:34:27.2350667Z 603261fb379f: Pulling fs layer 2024-08-06T20:34:27.2351002Z deceb46806b5: Waiting 2024-08-06T20:34:27.2351335Z d8ac71337d62: Pulling fs layer 2024-08-06T20:34:27.2351670Z 05896f1d52f7: Waiting 2024-08-06T20:34:27.2351995Z 2c2e35b979f5: Pulling fs layer 2024-08-06T20:34:27.2352353Z eeedca475f8c: Waiting 2024-08-06T20:34:27.2352651Z 603261fb379f: Waiting 2024-08-06T20:34:27.2352966Z bac001ac2fee: Waiting 2024-08-06T20:34:27.2353498Z e559222c0ec6: Pulling fs layer 2024-08-06T20:34:27.2353833Z cc64a79090ad: Waiting 2024-08-06T20:34:27.2354149Z 2c2e35b979f5: Waiting 2024-08-06T20:34:27.2354480Z c4dcb7e9d189: Pulling fs layer 2024-08-06T20:34:27.2354828Z 2b9731d295d6: Pulling fs layer 2024-08-06T20:34:27.2355200Z 19da43312eba: Pulling fs layer 2024-08-06T20:34:27.2355556Z c4dcb7e9d189: Waiting 2024-08-06T20:34:27.2355853Z 2b9731d295d6: Waiting 2024-08-06T20:34:27.2356180Z b846263d0664: Pulling fs layer 2024-08-06T20:34:27.2356546Z 3886e8f57dc8: Pulling fs layer 2024-08-06T20:34:27.2356879Z 19da43312eba: Waiting 2024-08-06T20:34:27.2357429Z 58e13e7b6d7b: Pulling fs layer 2024-08-06T20:34:27.2357820Z 0eb80a0ac000: Pulling fs layer 2024-08-06T20:34:27.2358175Z b846263d0664: Waiting 2024-08-06T20:34:27.2358473Z 03fb78f207e3: Waiting 2024-08-06T20:34:27.2358791Z 0eb80a0ac000: Waiting 2024-08-06T20:34:27.2359104Z a9f11b2b3622: Pulling fs layer 2024-08-06T20:34:27.2359476Z d7e94279f8b2: Pulling fs layer 2024-08-06T20:34:27.2359842Z a9f11b2b3622: Waiting 2024-08-06T20:34:27.2360155Z 4f4fb700ef54: Pulling fs layer 2024-08-06T20:34:27.2360530Z 0a3560fcd26c: Pulling fs layer 2024-08-06T20:34:27.2360903Z adf9d181cfbc: Pulling fs layer 2024-08-06T20:34:27.2361245Z 0a3560fcd26c: Waiting 2024-08-06T20:34:27.2361722Z b5632121ff7a: Waiting 2024-08-06T20:34:27.2362074Z bc18dd831597: Pulling fs layer 2024-08-06T20:34:27.2362416Z adf9d181cfbc: Waiting 2024-08-06T20:34:27.2362754Z d62f2a608619: Pulling fs layer 2024-08-06T20:34:27.2363129Z 719a1c2abcfd: Pulling fs layer 2024-08-06T20:34:27.2363725Z d62f2a608619: Waiting 2024-08-06T20:34:27.2364049Z 719a1c2abcfd: Waiting 2024-08-06T20:34:27.2364617Z 202c71391ade: Waiting 2024-08-06T20:34:27.2365214Z 8e1355d516c5: Pulling fs layer 2024-08-06T20:34:27.2365889Z 701898e66f54: Pulling fs layer 2024-08-06T20:34:27.2366548Z 8e1355d516c5: Waiting 2024-08-06T20:34:27.2367123Z 129b38cfed12: Pulling fs layer 2024-08-06T20:34:27.2367816Z d7e94279f8b2: Waiting 2024-08-06T20:34:27.2368389Z 58e13e7b6d7b: Waiting 2024-08-06T20:34:27.2369013Z 1751651a072f: Pulling fs layer 2024-08-06T20:34:27.2369459Z bc18dd831597: Waiting 2024-08-06T20:34:27.2369776Z fc956bf4e5b4: Pulling fs layer 2024-08-06T20:34:27.2370132Z 129b38cfed12: Waiting 2024-08-06T20:34:27.2370446Z 701898e66f54: Waiting 2024-08-06T20:34:27.2370747Z 1751651a072f: Waiting 2024-08-06T20:34:27.2371067Z 874491333fb5: Pulling fs layer 2024-08-06T20:34:27.2371582Z fc956bf4e5b4: Waiting 2024-08-06T20:34:27.2372142Z 874491333fb5: Waiting 2024-08-06T20:34:27.2372758Z 365b4a0eec21: Pulling fs layer 2024-08-06T20:34:27.2373392Z c1d4cac78640: Pulling fs layer 2024-08-06T20:34:27.2373750Z 29afe2181ca6: Pulling fs layer 2024-08-06T20:34:27.2374118Z 190dccfa687d: Pulling fs layer 2024-08-06T20:34:27.2374488Z b01bea4b6634: Pulling fs layer 2024-08-06T20:34:27.2374827Z c1d4cac78640: Waiting 2024-08-06T20:34:27.2375142Z 365b4a0eec21: Waiting 2024-08-06T20:34:27.2375471Z f13a9cc59688: Pulling fs layer 2024-08-06T20:34:27.2375808Z 29afe2181ca6: Waiting 2024-08-06T20:34:27.2376155Z 96d6dcd01b9f: Pulling fs layer 2024-08-06T20:34:27.2376513Z b01bea4b6634: Waiting 2024-08-06T20:34:27.2376827Z fe134554f1b8: Pulling fs layer 2024-08-06T20:34:27.2377180Z f13a9cc59688: Waiting 2024-08-06T20:34:27.2377488Z 8068d364963f: Pulling fs layer 2024-08-06T20:34:27.2377848Z 8068d364963f: Waiting 2024-08-06T20:34:27.2378179Z f307f545c1a5: Pulling fs layer 2024-08-06T20:34:27.2378535Z 51c0b7eab505: Pulling fs layer 2024-08-06T20:34:27.2378896Z fe134554f1b8: Waiting 2024-08-06T20:34:27.2379212Z 96d6dcd01b9f: Waiting 2024-08-06T20:34:27.2379512Z f307f545c1a5: Waiting 2024-08-06T20:34:27.2379852Z 60aba96f31c6: Pulling fs layer 2024-08-06T20:34:27.2380223Z 46db894a106c: Pulling fs layer 2024-08-06T20:34:27.2380585Z 0052583728e4: Pulling fs layer 2024-08-06T20:34:27.2380950Z 8c6d352a3b51: Pulling fs layer 2024-08-06T20:34:27.2381317Z 380c4e16278f: Pulling fs layer 2024-08-06T20:34:27.2381652Z 46db894a106c: Waiting 2024-08-06T20:34:27.2381972Z 60aba96f31c6: Waiting 2024-08-06T20:34:27.2382286Z 8c6d352a3b51: Waiting 2024-08-06T20:34:27.2382835Z 380c4e16278f: Waiting 2024-08-06T20:34:27.2383152Z 0052583728e4: Waiting 2024-08-06T20:34:27.4088279Z 09d415c238d7: Verifying Checksum 2024-08-06T20:34:27.4088746Z 09d415c238d7: Download complete 2024-08-06T20:34:27.4930688Z 41f16248e682: Verifying Checksum 2024-08-06T20:34:27.4931469Z 41f16248e682: Download complete 2024-08-06T20:34:27.5757182Z 7a2c55901189: Verifying Checksum 2024-08-06T20:34:27.5758173Z 7a2c55901189: Download complete 2024-08-06T20:34:27.5937409Z 95d7b7817039: Verifying Checksum 2024-08-06T20:34:27.5938203Z 95d7b7817039: Download complete 2024-08-06T20:34:27.6959388Z ab17245097e4: Verifying Checksum 2024-08-06T20:34:27.6959878Z ab17245097e4: Download complete 2024-08-06T20:34:27.7879474Z dfecd7e9912b: Download complete 2024-08-06T20:34:27.9773016Z 464a8f745445: Download complete 2024-08-06T20:34:28.0093855Z 9fe6e2e61518: Verifying Checksum 2024-08-06T20:34:28.0094282Z 9fe6e2e61518: Download complete 2024-08-06T20:34:28.1162810Z 0723abde19c0: Verifying Checksum 2024-08-06T20:34:28.1163316Z 0723abde19c0: Download complete 2024-08-06T20:34:28.2243803Z ceceb076946e: Verifying Checksum 2024-08-06T20:34:28.2244295Z ceceb076946e: Download complete 2024-08-06T20:34:28.8211304Z 7a2c55901189: Pull complete 2024-08-06T20:34:29.1664233Z 09d415c238d7: Pull complete 2024-08-06T20:34:30.3008606Z 9fe6e2e61518: Pull complete 2024-08-06T20:34:30.3425234Z 41f16248e682: Pull complete 2024-08-06T20:34:30.3889318Z 95d7b7817039: Pull complete 2024-08-06T20:34:34.8229966Z 889eb8142c51: Verifying Checksum 2024-08-06T20:34:34.8230447Z 889eb8142c51: Download complete 2024-08-06T20:34:34.9525700Z af8081143f00: Download complete 2024-08-06T20:34:35.0353316Z cc64a79090ad: Verifying Checksum 2024-08-06T20:34:35.0353738Z cc64a79090ad: Download complete 2024-08-06T20:34:35.1092831Z ad26542e0d01: Verifying Checksum 2024-08-06T20:34:35.1093265Z ad26542e0d01: Download complete 2024-08-06T20:34:36.1282217Z 7b42d8ae2367: Verifying Checksum 2024-08-06T20:34:36.1282699Z 7b42d8ae2367: Download complete 2024-08-06T20:34:36.2413212Z 6d46e3b97d9f: Verifying Checksum 2024-08-06T20:34:36.2413682Z 6d46e3b97d9f: Download complete 2024-08-06T20:34:36.3094464Z 91f9dddffbee: Verifying Checksum 2024-08-06T20:34:36.3094905Z 91f9dddffbee: Download complete 2024-08-06T20:34:36.4631018Z 03fb78f207e3: Verifying Checksum 2024-08-06T20:34:36.4631452Z 03fb78f207e3: Download complete 2024-08-06T20:34:41.5501747Z 8f6c90485347: Verifying Checksum 2024-08-06T20:34:41.5502206Z 8f6c90485347: Download complete 2024-08-06T20:34:41.6560314Z 202c71391ade: Verifying Checksum 2024-08-06T20:34:41.6560788Z 202c71391ade: Download complete 2024-08-06T20:34:41.7332957Z 5af882ef27a2: Download complete 2024-08-06T20:34:41.8339440Z bac001ac2fee: Verifying Checksum 2024-08-06T20:34:41.8339928Z bac001ac2fee: Download complete 2024-08-06T20:34:41.9111572Z 05896f1d52f7: Download complete 2024-08-06T20:34:42.0047563Z 49a0e418a175: Verifying Checksum 2024-08-06T20:34:42.0048010Z 49a0e418a175: Download complete 2024-08-06T20:34:43.3076026Z b5632121ff7a: Verifying Checksum 2024-08-06T20:34:43.3076483Z b5632121ff7a: Download complete 2024-08-06T20:34:43.4236561Z deceb46806b5: Download complete 2024-08-06T20:34:43.4989481Z a2630e5fe166: Download complete 2024-08-06T20:34:43.5718752Z 603261fb379f: Verifying Checksum 2024-08-06T20:34:43.5719275Z 603261fb379f: Download complete 2024-08-06T20:34:43.6539832Z d8ac71337d62: Verifying Checksum 2024-08-06T20:34:43.6540569Z d8ac71337d62: Download complete 2024-08-06T20:34:43.7437223Z 2c2e35b979f5: Download complete 2024-08-06T20:34:48.1589681Z e559222c0ec6: Verifying Checksum 2024-08-06T20:34:48.1590351Z e559222c0ec6: Download complete 2024-08-06T20:34:48.2385460Z c4dcb7e9d189: Verifying Checksum 2024-08-06T20:34:48.2386162Z c4dcb7e9d189: Download complete 2024-08-06T20:34:48.3135316Z 2b9731d295d6: Verifying Checksum 2024-08-06T20:34:48.3136144Z 2b9731d295d6: Download complete 2024-08-06T20:34:48.7791256Z 19da43312eba: Verifying Checksum 2024-08-06T20:34:48.7791742Z 19da43312eba: Download complete 2024-08-06T20:34:48.8511333Z b846263d0664: Verifying Checksum 2024-08-06T20:34:48.8511815Z b846263d0664: Download complete 2024-08-06T20:34:48.9403658Z 3886e8f57dc8: Verifying Checksum 2024-08-06T20:34:48.9404098Z 3886e8f57dc8: Download complete 2024-08-06T20:34:49.2159987Z 58e13e7b6d7b: Verifying Checksum 2024-08-06T20:34:49.2160473Z 58e13e7b6d7b: Download complete 2024-08-06T20:34:49.2860281Z 0eb80a0ac000: Verifying Checksum 2024-08-06T20:34:49.2860949Z 0eb80a0ac000: Download complete 2024-08-06T20:34:49.3782375Z a9f11b2b3622: Verifying Checksum 2024-08-06T20:34:49.3783041Z a9f11b2b3622: Download complete 2024-08-06T20:34:49.4807443Z d7e94279f8b2: Verifying Checksum 2024-08-06T20:34:49.4807898Z d7e94279f8b2: Download complete 2024-08-06T20:34:49.4871954Z 4f4fb700ef54: Verifying Checksum 2024-08-06T20:34:49.4872460Z 4f4fb700ef54: Download complete 2024-08-06T20:34:49.5481071Z 0a3560fcd26c: Verifying Checksum 2024-08-06T20:34:49.5481502Z 0a3560fcd26c: Download complete 2024-08-06T20:34:49.6223447Z adf9d181cfbc: Verifying Checksum 2024-08-06T20:34:49.6223951Z adf9d181cfbc: Download complete 2024-08-06T20:34:50.2119428Z bc18dd831597: Verifying Checksum 2024-08-06T20:34:50.2119884Z bc18dd831597: Download complete 2024-08-06T20:34:50.2901429Z d62f2a608619: Verifying Checksum 2024-08-06T20:34:50.2902188Z d62f2a608619: Download complete 2024-08-06T20:34:50.3720996Z 719a1c2abcfd: Verifying Checksum 2024-08-06T20:34:50.3721441Z 719a1c2abcfd: Download complete 2024-08-06T20:34:50.4720323Z 8e1355d516c5: Download complete 2024-08-06T20:34:50.5509238Z 701898e66f54: Verifying Checksum 2024-08-06T20:34:50.5509690Z 701898e66f54: Download complete 2024-08-06T20:34:52.4960720Z d67c111fa588: Verifying Checksum 2024-08-06T20:34:52.4961488Z d67c111fa588: Download complete 2024-08-06T20:34:52.5768638Z 1751651a072f: Verifying Checksum 2024-08-06T20:34:52.5769143Z 1751651a072f: Download complete 2024-08-06T20:34:52.6501937Z fc956bf4e5b4: Verifying Checksum 2024-08-06T20:34:52.6503069Z fc956bf4e5b4: Download complete 2024-08-06T20:34:52.7277701Z 874491333fb5: Verifying Checksum 2024-08-06T20:34:52.7278502Z 874491333fb5: Download complete 2024-08-06T20:34:52.8212568Z 365b4a0eec21: Verifying Checksum 2024-08-06T20:34:52.8213401Z 365b4a0eec21: Download complete 2024-08-06T20:34:52.9329474Z c1d4cac78640: Verifying Checksum 2024-08-06T20:34:52.9330645Z c1d4cac78640: Download complete 2024-08-06T20:34:53.1208402Z 29afe2181ca6: Verifying Checksum 2024-08-06T20:34:53.1208859Z 29afe2181ca6: Download complete 2024-08-06T20:34:53.1939346Z 190dccfa687d: Verifying Checksum 2024-08-06T20:34:53.1939812Z 190dccfa687d: Download complete 2024-08-06T20:34:53.2568132Z b01bea4b6634: Download complete 2024-08-06T20:34:53.3274905Z f13a9cc59688: Verifying Checksum 2024-08-06T20:34:53.3275364Z f13a9cc59688: Download complete 2024-08-06T20:34:53.3969652Z 96d6dcd01b9f: Verifying Checksum 2024-08-06T20:34:53.3970074Z 96d6dcd01b9f: Download complete 2024-08-06T20:34:53.5592686Z fe134554f1b8: Verifying Checksum 2024-08-06T20:34:53.5595596Z fe134554f1b8: Download complete 2024-08-06T20:34:53.6199340Z 8068d364963f: Verifying Checksum 2024-08-06T20:34:53.6200164Z 8068d364963f: Download complete 2024-08-06T20:34:54.2072245Z f307f545c1a5: Verifying Checksum 2024-08-06T20:34:54.2072720Z f307f545c1a5: Download complete 2024-08-06T20:34:54.2822046Z 51c0b7eab505: Verifying Checksum 2024-08-06T20:34:54.2822516Z 51c0b7eab505: Download complete 2024-08-06T20:34:59.8737201Z 8f6c90485347: Pull complete 2024-08-06T20:34:59.9183290Z ab17245097e4: Pull complete 2024-08-06T20:34:59.9943649Z dfecd7e9912b: Pull complete 2024-08-06T20:35:00.0517779Z 464a8f745445: Pull complete 2024-08-06T20:35:04.6497437Z eeedca475f8c: Download complete 2024-08-06T20:35:04.7412788Z 46db894a106c: Verifying Checksum 2024-08-06T20:35:04.7413640Z 46db894a106c: Download complete 2024-08-06T20:35:04.8204522Z 0052583728e4: Download complete 2024-08-06T20:35:05.2635859Z 8c6d352a3b51: Verifying Checksum 2024-08-06T20:35:05.2636316Z 8c6d352a3b51: Download complete 2024-08-06T20:35:05.3510916Z 380c4e16278f: Verifying Checksum 2024-08-06T20:35:05.3512180Z 380c4e16278f: Download complete 2024-08-06T20:35:08.0696772Z 60aba96f31c6: Verifying Checksum 2024-08-06T20:35:08.0697246Z 60aba96f31c6: Download complete 2024-08-06T20:35:09.7714982Z 129b38cfed12: Verifying Checksum 2024-08-06T20:35:09.7715485Z 129b38cfed12: Download complete 2024-08-06T20:35:37.8832756Z d67c111fa588: Pull complete 2024-08-06T20:35:40.0814108Z 0723abde19c0: Pull complete 2024-08-06T20:35:42.7753854Z ceceb076946e: Pull complete 2024-08-06T20:35:59.2395463Z 889eb8142c51: Pull complete 2024-08-06T20:36:01.6796219Z af8081143f00: Pull complete 2024-08-06T20:36:04.1254564Z cc64a79090ad: Pull complete 2024-08-06T20:36:06.5625808Z ad26542e0d01: Pull complete 2024-08-06T20:36:12.6911106Z 7b42d8ae2367: Pull complete 2024-08-06T20:36:15.3707647Z 6d46e3b97d9f: Pull complete 2024-08-06T20:36:17.8135048Z 91f9dddffbee: Pull complete 2024-08-06T20:36:20.5969074Z 03fb78f207e3: Pull complete 2024-08-06T20:37:25.7348279Z eeedca475f8c: Pull complete 2024-08-06T20:37:29.6097253Z 202c71391ade: Pull complete 2024-08-06T20:37:33.3512241Z 5af882ef27a2: Pull complete 2024-08-06T20:37:37.3853260Z bac001ac2fee: Pull complete 2024-08-06T20:37:40.1617106Z 05896f1d52f7: Pull complete 2024-08-06T20:37:40.6176655Z 49a0e418a175: Pull complete 2024-08-06T20:37:44.8219797Z b5632121ff7a: Pull complete 2024-08-06T20:37:44.8646939Z deceb46806b5: Pull complete 2024-08-06T20:37:44.9181974Z a2630e5fe166: Pull complete 2024-08-06T20:37:44.9668807Z 603261fb379f: Pull complete 2024-08-06T20:37:45.0086445Z d8ac71337d62: Pull complete 2024-08-06T20:37:45.0496226Z 2c2e35b979f5: Pull complete 2024-08-06T20:37:56.7661245Z e559222c0ec6: Pull complete 2024-08-06T20:37:59.5081488Z c4dcb7e9d189: Pull complete 2024-08-06T20:38:00.5734585Z 2b9731d295d6: Pull complete 2024-08-06T20:38:01.5391010Z 19da43312eba: Pull complete 2024-08-06T20:38:01.5859532Z b846263d0664: Pull complete 2024-08-06T20:38:01.6313715Z 3886e8f57dc8: Pull complete 2024-08-06T20:38:02.0125324Z 58e13e7b6d7b: Pull complete 2024-08-06T20:38:02.0648443Z 0eb80a0ac000: Pull complete 2024-08-06T20:38:02.1860842Z a9f11b2b3622: Pull complete 2024-08-06T20:38:02.2357024Z d7e94279f8b2: Pull complete 2024-08-06T20:38:02.2813019Z 4f4fb700ef54: Pull complete 2024-08-06T20:38:02.3179412Z 0a3560fcd26c: Pull complete 2024-08-06T20:38:02.3549222Z adf9d181cfbc: Pull complete 2024-08-06T20:38:04.7796295Z bc18dd831597: Pull complete 2024-08-06T20:38:04.8166478Z d62f2a608619: Pull complete 2024-08-06T20:38:04.8549222Z 719a1c2abcfd: Pull complete 2024-08-06T20:38:04.9602800Z 8e1355d516c5: Pull complete 2024-08-06T20:38:05.0054100Z 701898e66f54: Pull complete 2024-08-06T20:38:48.6314242Z 129b38cfed12: Pull complete 2024-08-06T20:38:51.0952524Z 1751651a072f: Pull complete 2024-08-06T20:38:53.5361995Z fc956bf4e5b4: Pull complete 2024-08-06T20:38:58.4079209Z 874491333fb5: Pull complete 2024-08-06T20:39:03.3817530Z 365b4a0eec21: Pull complete 2024-08-06T20:39:05.8249069Z c1d4cac78640: Pull complete 2024-08-06T20:39:08.9695782Z 29afe2181ca6: Pull complete 2024-08-06T20:39:11.3630278Z 190dccfa687d: Pull complete 2024-08-06T20:39:13.8841650Z b01bea4b6634: Pull complete 2024-08-06T20:39:16.2311286Z f13a9cc59688: Pull complete 2024-08-06T20:39:19.0905127Z 96d6dcd01b9f: Pull complete 2024-08-06T20:39:20.6975640Z fe134554f1b8: Pull complete 2024-08-06T20:39:20.7453116Z 8068d364963f: Pull complete 2024-08-06T20:39:23.1205200Z f307f545c1a5: Pull complete 2024-08-06T20:39:23.1699979Z 51c0b7eab505: Pull complete 2024-08-06T20:39:39.7914389Z 60aba96f31c6: Pull complete 2024-08-06T20:39:39.8989489Z 46db894a106c: Pull complete 2024-08-06T20:39:39.9371482Z 0052583728e4: Pull complete 2024-08-06T20:39:40.6136113Z 8c6d352a3b51: Pull complete 2024-08-06T20:39:40.6445808Z 380c4e16278f: Pull complete 2024-08-06T20:39:40.9528285Z Digest: sha256:66181a134b3ebc35388dabcc46dc11e767152fd72514dad76711590f50b650f7 2024-08-06T20:39:40.9564764Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:39:40.9599277Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:39:40.9649943Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-06T20:39:40.9650812Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-06T20:39:40.9660022Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:39:40.9660506Z env: 2024-08-06T20:39:40.9660782Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:39:40.9661131Z ##[endgroup] 2024-08-06T20:39:40.9926223Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-06T20:39:40.9926780Z with: 2024-08-06T20:39:40.9927076Z driver-version: 550.54.15 2024-08-06T20:39:40.9927507Z env: 2024-08-06T20:39:40.9927797Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:39:40.9928143Z ##[endgroup] 2024-08-06T20:39:40.9977325Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-06T20:39:40.9978302Z with: 2024-08-06T20:39:40.9978884Z timeout_minutes: 10 2024-08-06T20:39:40.9979466Z max_attempts: 3 2024-08-06T20:39:41.0010692Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-06T20:39:41.0043823Z retry_wait_seconds: 10 2024-08-06T20:39:41.0044196Z polling_interval_seconds: 1 2024-08-06T20:39:41.0044555Z warning_on_retry: true 2024-08-06T20:39:41.0044906Z continue_on_error: false 2024-08-06T20:39:41.0045247Z env: 2024-08-06T20:39:41.0045513Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:39:41.0045869Z DRIVER_VERSION: 550.54.15 2024-08-06T20:39:41.0046209Z ##[endgroup] 2024-08-06T20:39:41.0799168Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-06T20:39:41.0801216Z + pre_install_nvidia_driver_amzn2 2024-08-06T20:39:41.0801671Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-06T20:39:41.5901774Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-08-06T20:39:41.6492464Z No Match for argument: nvidia-driver-latest-dkms 2024-08-06T20:39:41.6860708Z No Packages marked for removal 2024-08-06T20:39:41.7020834Z + install_nvidia_driver_common 2024-08-06T20:39:41.7024316Z + echo 'Before installing NVIDIA driver' 2024-08-06T20:39:41.7024909Z + lspci 2024-08-06T20:39:41.7027122Z Before installing NVIDIA driver 2024-08-06T20:39:41.7202882Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-08-06T20:39:41.7203602Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-06T20:39:41.7204287Z 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 2024-08-06T20:39:41.7205232Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 2024-08-06T20:39:41.7205812Z 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 2024-08-06T20:39:41.7206413Z 00:03.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-06T20:39:41.7207155Z 00:1d.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-08-06T20:39:41.7207912Z 00:1e.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-08-06T20:39:41.7208634Z 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 2024-08-06T20:39:41.7209335Z + lsmod 2024-08-06T20:39:41.7226350Z Module Size Used by 2024-08-06T20:39:41.7226775Z ib_core 266240 0 2024-08-06T20:39:41.7227121Z veth 16384 0 2024-08-06T20:39:41.7227485Z nvidia_modeset 1339392 0 2024-08-06T20:39:41.7227848Z nvidia_uvm 4599808 0 2024-08-06T20:39:41.7228260Z nvidia 53989376 11 nvidia_uvm,nvidia_modeset 2024-08-06T20:39:41.7228738Z drm 421888 1 nvidia 2024-08-06T20:39:41.7229152Z i2c_core 77824 2 nvidia,drm 2024-08-06T20:39:41.7229579Z backlight 16384 1 nvidia_modeset 2024-08-06T20:39:41.7230015Z xt_conntrack 16384 1 2024-08-06T20:39:41.7230385Z ipt_MASQUERADE 16384 1 2024-08-06T20:39:41.7230787Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-08-06T20:39:41.7231241Z nf_conntrack_netlink 49152 0 2024-08-06T20:39:41.7231669Z nfnetlink 16384 2 nf_conntrack_netlink 2024-08-06T20:39:41.7232105Z xfrm_user 45056 1 2024-08-06T20:39:41.7232490Z xfrm_algo 16384 1 xfrm_user 2024-08-06T20:39:41.7232893Z iptable_nat 16384 1 2024-08-06T20:39:41.7233246Z nf_conntrack_ipv4 16384 3 2024-08-06T20:39:41.7233660Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-08-06T20:39:41.7234115Z nf_nat_ipv4 16384 1 iptable_nat 2024-08-06T20:39:41.7234595Z nf_nat 32768 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 2024-08-06T20:39:41.7235467Z nf_conntrack 155648 7 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink 2024-08-06T20:39:41.7236259Z xt_addrtype 16384 2 2024-08-06T20:39:41.7236620Z iptable_filter 16384 1 2024-08-06T20:39:41.7236993Z br_netfilter 24576 0 2024-08-06T20:39:41.7237862Z bridge 172032 1 br_netfilter 2024-08-06T20:39:41.7238282Z stp 16384 1 bridge 2024-08-06T20:39:41.7238678Z llc 16384 2 bridge,stp 2024-08-06T20:39:41.7239082Z overlay 86016 0 2024-08-06T20:39:41.7239444Z sunrpc 393216 1 2024-08-06T20:39:41.7239815Z dm_mirror 28672 0 2024-08-06T20:39:41.7240196Z dm_region_hash 20480 1 dm_mirror 2024-08-06T20:39:41.7240632Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-08-06T20:39:41.7241104Z dm_mod 143360 2 dm_log,dm_mirror 2024-08-06T20:39:41.7241539Z dax 69632 1 dm_mod 2024-08-06T20:39:41.7241911Z sb_edac 24576 0 2024-08-06T20:39:41.7242275Z crc32_pclmul 16384 0 2024-08-06T20:39:41.7242648Z ghash_clmulni_intel 16384 0 2024-08-06T20:39:41.7243002Z pcbc 16384 0 2024-08-06T20:39:41.7243365Z aesni_intel 188416 0 2024-08-06T20:39:41.7243744Z aes_x86_64 20480 1 aesni_intel 2024-08-06T20:39:41.7244152Z crypto_simd 16384 1 aesni_intel 2024-08-06T20:39:41.7244582Z glue_helper 16384 1 aesni_intel 2024-08-06T20:39:41.7245107Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-08-06T20:39:41.7245603Z mousedev 24576 0 2024-08-06T20:39:41.7245968Z evdev 20480 3 2024-08-06T20:39:41.7246333Z psmouse 32768 0 2024-08-06T20:39:41.7246681Z pcc_cpufreq 16384 0 2024-08-06T20:39:41.7247046Z button 16384 0 2024-08-06T20:39:41.7247570Z ena 139264 0 2024-08-06T20:39:41.7247911Z ptp 20480 1 ena 2024-08-06T20:39:41.7248303Z pps_core 20480 1 ptp 2024-08-06T20:39:41.7248687Z ata_piix 36864 0 2024-08-06T20:39:41.7249041Z libata 266240 1 ata_piix 2024-08-06T20:39:41.7249447Z xen_blkfront 49152 2 2024-08-06T20:39:41.7249810Z crc32c_intel 24576 0 2024-08-06T20:39:41.7250159Z scsi_mod 245760 1 libata 2024-08-06T20:39:41.7250545Z autofs4 49152 2 2024-08-06T20:39:41.7250900Z + modinfo nvidia 2024-08-06T20:39:41.7251518Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-08-06T20:39:41.7252139Z alias: char-major-195-* 2024-08-06T20:39:41.7252516Z version: 550.54.15 2024-08-06T20:39:41.7252847Z supported: external 2024-08-06T20:39:41.7253186Z license: NVIDIA 2024-08-06T20:39:41.7253542Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-06T20:39:41.7254020Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-06T20:39:41.7254458Z srcversion: 833721318DA517F0C2FEC97 2024-08-06T20:39:41.7254882Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-06T20:39:41.7255352Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-06T20:39:41.7255812Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-06T20:39:41.7256230Z depends: i2c-core,drm 2024-08-06T20:39:41.7256592Z retpoline: Y 2024-08-06T20:39:41.7256902Z name: nvidia 2024-08-06T20:39:41.7257464Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-08-06T20:39:41.7258072Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-06T20:39:41.7258680Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-06T20:39:41.7259316Z parm: NVreg_ResmanDebugLevel:int 2024-08-06T20:39:41.7259906Z parm: NVreg_RmLogonRC:int 2024-08-06T20:39:41.7260345Z parm: NVreg_ModifyDeviceFiles:int 2024-08-06T20:39:41.7260774Z parm: NVreg_DeviceFileUID:int 2024-08-06T20:39:41.7261201Z parm: NVreg_DeviceFileGID:int 2024-08-06T20:39:41.7261627Z parm: NVreg_DeviceFileMode:int 2024-08-06T20:39:41.7262113Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-06T20:39:41.7262649Z parm: NVreg_UsePageAttributeTable:int 2024-08-06T20:39:41.7263111Z parm: NVreg_EnablePCIeGen3:int 2024-08-06T20:39:41.7263518Z parm: NVreg_EnableMSI:int 2024-08-06T20:39:41.7263928Z parm: NVreg_TCEBypassMode:int 2024-08-06T20:39:41.7264385Z parm: NVreg_EnableStreamMemOPs:int 2024-08-06T20:39:41.7264874Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-06T20:39:41.7265423Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-06T20:39:41.7265950Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-06T20:39:41.7266497Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-06T20:39:41.7267068Z parm: NVreg_DynamicPowerManagement:int 2024-08-06T20:39:41.7267643Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-06T20:39:41.7268190Z parm: NVreg_EnableGpuFirmware:int 2024-08-06T20:39:41.7268663Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-06T20:39:41.7269176Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-06T20:39:41.7269681Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-06T20:39:41.7270156Z parm: NVreg_MemoryPoolSize:int 2024-08-06T20:39:41.7270613Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-06T20:39:41.7271060Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-06T20:39:41.7271520Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-06T20:39:41.7271956Z parm: NVreg_NvLinkDisable:int 2024-08-06T20:39:41.7272421Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-06T20:39:41.7272954Z parm: NVreg_RegisterPCIDriver:int 2024-08-06T20:39:41.7273547Z parm: NVreg_EnableResizableBar:int 2024-08-06T20:39:41.7274030Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-06T20:39:41.7274509Z parm: NVreg_EnableNonblockingOpen:int 2024-08-06T20:39:41.7274963Z parm: NVreg_RegistryDwords:charp 2024-08-06T20:39:41.7275484Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-06T20:39:41.7275948Z parm: NVreg_RmMsg:charp 2024-08-06T20:39:41.7276394Z parm: NVreg_GpuBlacklist:charp 2024-08-06T20:39:41.7276854Z parm: NVreg_TemporaryFilePath:charp 2024-08-06T20:39:41.7277921Z parm: NVreg_ExcludedGpus:charp 2024-08-06T20:39:41.7278416Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-06T20:39:41.7278875Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-06T20:39:41.7279325Z parm: NVreg_ImexChannelCount:int 2024-08-06T20:39:41.7279765Z parm: rm_firmware_active:charp 2024-08-06T20:39:41.7280185Z + HAS_NVIDIA_DRIVER=0 2024-08-06T20:39:41.7280525Z ++ command -v nvidia-smi 2024-08-06T20:39:41.7280894Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-06T20:39:41.7281246Z + set +e 2024-08-06T20:39:41.7281682Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-06T20:39:41.7510789Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-06T20:39:41.7511220Z + NVIDIA_SMI_STATUS=0 2024-08-06T20:39:41.7511537Z + '[' 0 -ne 0 ']' 2024-08-06T20:39:41.7511849Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-06T20:39:41.7512203Z + HAS_NVIDIA_DRIVER=1 2024-08-06T20:39:41.7512783Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-06T20:39:41.7513435Z + set -e 2024-08-06T20:39:41.7513702Z + '[' 1 -eq 0 ']' 2024-08-06T20:39:41.7514227Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-06T20:39:41.7514862Z + post_install_nvidia_driver_common 2024-08-06T20:39:41.7517648Z + sudo modprobe nvidia 2024-08-06T20:39:41.7632738Z + echo 'After installing NVIDIA driver' 2024-08-06T20:39:41.7633166Z + lspci 2024-08-06T20:39:41.7634172Z After installing NVIDIA driver 2024-08-06T20:39:41.7804934Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-08-06T20:39:41.7805651Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-06T20:39:41.7806344Z 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 2024-08-06T20:39:41.7806994Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 2024-08-06T20:39:41.7807569Z 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 2024-08-06T20:39:41.7808207Z 00:03.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-06T20:39:41.7808935Z 00:1d.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-08-06T20:39:41.7809699Z 00:1e.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-08-06T20:39:41.7810443Z 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 2024-08-06T20:39:41.7810994Z + lsmod 2024-08-06T20:39:41.7826207Z Module Size Used by 2024-08-06T20:39:41.7826649Z ib_core 266240 0 2024-08-06T20:39:41.7827011Z veth 16384 0 2024-08-06T20:39:41.7827355Z nvidia_modeset 1339392 0 2024-08-06T20:39:41.7827721Z nvidia_uvm 4599808 0 2024-08-06T20:39:41.7828146Z nvidia 53989376 11 nvidia_uvm,nvidia_modeset 2024-08-06T20:39:41.7828588Z drm 421888 1 nvidia 2024-08-06T20:39:41.7828992Z i2c_core 77824 2 nvidia,drm 2024-08-06T20:39:41.7829445Z backlight 16384 1 nvidia_modeset 2024-08-06T20:39:41.7829862Z xt_conntrack 16384 1 2024-08-06T20:39:41.7830239Z ipt_MASQUERADE 16384 1 2024-08-06T20:39:41.7830656Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-08-06T20:39:41.7831088Z nf_conntrack_netlink 49152 0 2024-08-06T20:39:41.7831521Z nfnetlink 16384 2 nf_conntrack_netlink 2024-08-06T20:39:41.7832194Z xfrm_user 45056 1 2024-08-06T20:39:41.7832556Z xfrm_algo 16384 1 xfrm_user 2024-08-06T20:39:41.7832960Z iptable_nat 16384 1 2024-08-06T20:39:41.7833330Z nf_conntrack_ipv4 16384 3 2024-08-06T20:39:41.7833725Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-08-06T20:39:41.7834187Z nf_nat_ipv4 16384 1 iptable_nat 2024-08-06T20:39:41.7834674Z nf_nat 32768 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 2024-08-06T20:39:41.7835658Z nf_conntrack 155648 7 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink 2024-08-06T20:39:41.7836486Z xt_addrtype 16384 2 2024-08-06T20:39:41.7836864Z iptable_filter 16384 1 2024-08-06T20:39:41.7837534Z br_netfilter 24576 0 2024-08-06T20:39:41.7837932Z bridge 172032 1 br_netfilter 2024-08-06T20:39:41.7838348Z stp 16384 1 bridge 2024-08-06T20:39:41.7838734Z llc 16384 2 bridge,stp 2024-08-06T20:39:41.7839145Z overlay 86016 0 2024-08-06T20:39:41.7839508Z sunrpc 393216 1 2024-08-06T20:39:41.7839858Z dm_mirror 28672 0 2024-08-06T20:39:41.7840306Z dm_region_hash 20480 1 dm_mirror 2024-08-06T20:39:41.7840761Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-08-06T20:39:41.7841206Z dm_mod 143360 2 dm_log,dm_mirror 2024-08-06T20:39:41.7841626Z dax 69632 1 dm_mod 2024-08-06T20:39:41.7842013Z sb_edac 24576 0 2024-08-06T20:39:41.7842362Z crc32_pclmul 16384 0 2024-08-06T20:39:41.7842737Z ghash_clmulni_intel 16384 0 2024-08-06T20:39:41.7843107Z pcbc 16384 0 2024-08-06T20:39:41.7843449Z aesni_intel 188416 0 2024-08-06T20:39:41.7843823Z aes_x86_64 20480 1 aesni_intel 2024-08-06T20:39:41.7844254Z crypto_simd 16384 1 aesni_intel 2024-08-06T20:39:41.7844665Z glue_helper 16384 1 aesni_intel 2024-08-06T20:39:41.7845192Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-08-06T20:39:41.7845704Z mousedev 24576 0 2024-08-06T20:39:41.7846047Z evdev 20480 3 2024-08-06T20:39:41.7846410Z psmouse 32768 0 2024-08-06T20:39:41.7846773Z pcc_cpufreq 16384 0 2024-08-06T20:39:41.7847117Z button 16384 0 2024-08-06T20:39:41.7847512Z ena 139264 0 2024-08-06T20:39:41.7847869Z ptp 20480 1 ena 2024-08-06T20:39:41.7848233Z pps_core 20480 1 ptp 2024-08-06T20:39:41.7848623Z ata_piix 36864 0 2024-08-06T20:39:41.7849001Z libata 266240 1 ata_piix 2024-08-06T20:39:41.7849386Z xen_blkfront 49152 2 2024-08-06T20:39:41.7849749Z crc32c_intel 24576 0 2024-08-06T20:39:41.7850215Z scsi_mod 245760 1 libata 2024-08-06T20:39:41.7850588Z autofs4 49152 2 2024-08-06T20:39:41.7850954Z + modinfo nvidia 2024-08-06T20:39:41.7851467Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-08-06T20:39:41.7852226Z alias: char-major-195-* 2024-08-06T20:39:41.7852607Z version: 550.54.15 2024-08-06T20:39:41.7852956Z supported: external 2024-08-06T20:39:41.7853288Z license: NVIDIA 2024-08-06T20:39:41.7853662Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-06T20:39:41.7854137Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-06T20:39:41.7854561Z srcversion: 833721318DA517F0C2FEC97 2024-08-06T20:39:41.7855024Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-06T20:39:41.7855496Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-06T20:39:41.7855950Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-06T20:39:41.7856386Z depends: i2c-core,drm 2024-08-06T20:39:41.7856751Z retpoline: Y 2024-08-06T20:39:41.7857053Z name: nvidia 2024-08-06T20:39:41.7857509Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-08-06T20:39:41.7858253Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-06T20:39:41.7858846Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-06T20:39:41.7859419Z parm: NVreg_ResmanDebugLevel:int 2024-08-06T20:39:41.7859856Z parm: NVreg_RmLogonRC:int 2024-08-06T20:39:41.7860265Z parm: NVreg_ModifyDeviceFiles:int 2024-08-06T20:39:41.7860718Z parm: NVreg_DeviceFileUID:int 2024-08-06T20:39:41.7861147Z parm: NVreg_DeviceFileGID:int 2024-08-06T20:39:41.7861673Z parm: NVreg_DeviceFileMode:int 2024-08-06T20:39:41.7862194Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-06T20:39:41.7862730Z parm: NVreg_UsePageAttributeTable:int 2024-08-06T20:39:41.7863182Z parm: NVreg_EnablePCIeGen3:int 2024-08-06T20:39:41.7863613Z parm: NVreg_EnableMSI:int 2024-08-06T20:39:41.7864025Z parm: NVreg_TCEBypassMode:int 2024-08-06T20:39:41.7864469Z parm: NVreg_EnableStreamMemOPs:int 2024-08-06T20:39:41.7864979Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-06T20:39:41.7865527Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-06T20:39:41.7866040Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-06T20:39:41.7866604Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-06T20:39:41.7867163Z parm: NVreg_DynamicPowerManagement:int 2024-08-06T20:39:41.7867717Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-06T20:39:41.7868284Z parm: NVreg_EnableGpuFirmware:int 2024-08-06T20:39:41.7868759Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-06T20:39:41.7869256Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-06T20:39:41.7869768Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-06T20:39:41.7870244Z parm: NVreg_MemoryPoolSize:int 2024-08-06T20:39:41.7870684Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-06T20:39:41.7871153Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-06T20:39:41.7871605Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-06T20:39:41.7872027Z parm: NVreg_NvLinkDisable:int 2024-08-06T20:39:41.7872511Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-06T20:39:41.7873008Z parm: NVreg_RegisterPCIDriver:int 2024-08-06T20:39:41.7873451Z parm: NVreg_EnableResizableBar:int 2024-08-06T20:39:41.7873914Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-06T20:39:41.7874391Z parm: NVreg_EnableNonblockingOpen:int 2024-08-06T20:39:41.7874854Z parm: NVreg_RegistryDwords:charp 2024-08-06T20:39:41.7875331Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-06T20:39:41.7875787Z parm: NVreg_RmMsg:charp 2024-08-06T20:39:41.7876178Z parm: NVreg_GpuBlacklist:charp 2024-08-06T20:39:41.7876635Z parm: NVreg_TemporaryFilePath:charp 2024-08-06T20:39:41.7877404Z parm: NVreg_ExcludedGpus:charp 2024-08-06T20:39:41.7877856Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-06T20:39:41.7878319Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-06T20:39:41.7878782Z parm: NVreg_ImexChannelCount:int 2024-08-06T20:39:41.7879210Z parm: rm_firmware_active:charp 2024-08-06T20:39:41.7879608Z + set +e 2024-08-06T20:39:41.7879891Z + nvidia-smi 2024-08-06T20:39:41.8095805Z Tue Aug 6 20:39:41 2024 2024-08-06T20:39:41.8096317Z +-----------------------------------------------------------------------------------------+ 2024-08-06T20:39:41.8097014Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-06T20:39:41.8097715Z |-----------------------------------------+------------------------+----------------------+ 2024-08-06T20:39:41.8098399Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-06T20:39:41.8099132Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-06T20:39:41.8099908Z | | | MIG M. | 2024-08-06T20:39:41.8100388Z |=========================================+========================+======================| 2024-08-06T20:39:41.8178191Z | 0 Tesla M60 On | 00000000:00:1D.0 Off | Off | 2024-08-06T20:39:41.8178809Z | N/A 35C P8 14W / 150W | 0MiB / 8192MiB | 0% Default | 2024-08-06T20:39:41.8179479Z | | | N/A | 2024-08-06T20:39:41.8180064Z +-----------------------------------------+------------------------+----------------------+ 2024-08-06T20:39:41.8242076Z | 1 Tesla M60 On | 00000000:00:1E.0 Off | Off | 2024-08-06T20:39:41.8242668Z | N/A 38C P8 13W / 150W | 0MiB / 8192MiB | 0% Default | 2024-08-06T20:39:41.8243213Z | | | N/A | 2024-08-06T20:39:41.8243771Z +-----------------------------------------+------------------------+----------------------+ 2024-08-06T20:39:41.8244327Z 2024-08-06T20:39:41.8244861Z +-----------------------------------------------------------------------------------------+ 2024-08-06T20:39:41.8245471Z | Processes: | 2024-08-06T20:39:41.8246101Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-06T20:39:41.8246692Z | ID ID Usage | 2024-08-06T20:39:41.8247181Z |=========================================================================================| 2024-08-06T20:39:41.8248309Z | No running processes found | 2024-08-06T20:39:41.8248977Z +-----------------------------------------------------------------------------------------+ 2024-08-06T20:39:41.9361381Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-06T20:39:41.9549951Z Tesla M60 2024-08-06T20:39:41.9625478Z + NVIDIA_SMI_STATUS=0 2024-08-06T20:39:41.9625827Z + '[' 0 -eq 0 ']' 2024-08-06T20:39:41.9626171Z + echo 'INFO: Ignoring allowed status 0' 2024-08-06T20:39:41.9626588Z + set -e 2024-08-06T20:39:41.9627216Z INFO: Ignoring allowed status 0 2024-08-06T20:39:41.9632243Z == Installing nvidia container toolkit for amzn2 == 2024-08-06T20:39:41.9635910Z + sudo yum install -y yum-utils 2024-08-06T20:39:42.4540066Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-08-06T20:39:44.1589595Z Package yum-utils-1.1.31-46.amzn2.0.1.noarch already installed and latest version 2024-08-06T20:39:44.1590243Z Nothing to do 2024-08-06T20:39:44.3015893Z + [[ amzn2 == \a\m\z\n\2\0\2\3 ]] 2024-08-06T20:39:44.3016581Z + YUM_REPO_URL=https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-08-06T20:39:44.3017517Z + sudo yum-config-manager --add-repo https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-08-06T20:39:44.8069864Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-08-06T20:39:44.8406032Z adding repo from: https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-08-06T20:39:44.8407066Z grabbing file https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo to /etc/yum.repos.d/nvidia-docker.repo 2024-08-06T20:39:44.8407941Z repo saved to /etc/yum.repos.d/nvidia-docker.repo 2024-08-06T20:39:44.8553034Z + sudo yum install -y nvidia-docker2 2024-08-06T20:39:45.3411008Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-08-06T20:39:47.0145118Z Package nvidia-docker2-2.14.0-1.noarch already installed and latest version 2024-08-06T20:39:47.0145747Z Nothing to do 2024-08-06T20:39:47.1540097Z + sudo systemctl restart docker 2024-08-06T20:40:06.2012537Z nvidia-persistenced failed to initialize. Check syslog for more details. 2024-08-06T20:40:06.2491660Z Tue Aug 6 20:40:06 2024 2024-08-06T20:40:06.2492195Z +-----------------------------------------------------------------------------------------+ 2024-08-06T20:40:06.2492884Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-06T20:40:06.2493535Z |-----------------------------------------+------------------------+----------------------+ 2024-08-06T20:40:06.2494485Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-06T20:40:06.2495243Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-06T20:40:06.2495851Z | | | MIG M. | 2024-08-06T20:40:06.2496316Z |=========================================+========================+======================| 2024-08-06T20:40:06.2573575Z | 0 Tesla M60 On | 00000000:00:1D.0 Off | Off | 2024-08-06T20:40:06.2574189Z | N/A 35C P8 15W / 150W | 0MiB / 8192MiB | 0% Default | 2024-08-06T20:40:06.2574724Z | | | N/A | 2024-08-06T20:40:06.2575286Z +-----------------------------------------+------------------------+----------------------+ 2024-08-06T20:40:06.2635995Z | 1 Tesla M60 On | 00000000:00:1E.0 Off | Off | 2024-08-06T20:40:06.2636667Z | N/A 38C P8 13W / 150W | 0MiB / 8192MiB | 0% Default | 2024-08-06T20:40:06.2637609Z | | | N/A | 2024-08-06T20:40:06.2638179Z +-----------------------------------------+------------------------+----------------------+ 2024-08-06T20:40:06.2638754Z 2024-08-06T20:40:06.2639301Z +-----------------------------------------------------------------------------------------+ 2024-08-06T20:40:06.2639878Z | Processes: | 2024-08-06T20:40:06.2640500Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-06T20:40:06.2641086Z | ID ID Usage | 2024-08-06T20:40:06.2641566Z |=========================================================================================| 2024-08-06T20:40:06.2642587Z | No running processes found | 2024-08-06T20:40:06.2643247Z +-----------------------------------------------------------------------------------------+ 2024-08-06T20:40:07.1035271Z Command completed after 1 attempt(s). 2024-08-06T20:40:07.1104981Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-06T20:40:07.1105727Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-06T20:40:07.1106375Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-06T20:40:07.1106979Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:40:07.1115001Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:07.1115490Z env: 2024-08-06T20:40:07.1115782Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:07.1116203Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:07.1116667Z ##[endgroup] 2024-08-06T20:40:07.4072374Z Defaulting to user installation because normal site-packages is not writeable 2024-08-06T20:40:07.4283046Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.7/site-packages (5.9.1) 2024-08-06T20:40:07.4406732Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.7/site-packages (11.525.84) 2024-08-06T20:40:07.5529134Z Prepare all required actions 2024-08-06T20:40:07.5529802Z Getting action download info 2024-08-06T20:40:07.6684056Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-06T20:40:07.8922700Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-06T20:40:08.1509050Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-06T20:40:08.1509521Z with: 2024-08-06T20:40:08.1509832Z name: linux-focal-cuda11.8-py3.10-gcc9 2024-08-06T20:40:08.1510261Z s3-bucket: gha-artifacts 2024-08-06T20:40:08.1510605Z env: 2024-08-06T20:40:08.1510871Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:08.1511304Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:08.1511830Z ##[endgroup] 2024-08-06T20:40:08.1555519Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-06T20:40:08.1555959Z with: 2024-08-06T20:40:08.1556349Z name: linux-focal-cuda11.8-py3.10-gcc9 2024-08-06T20:40:08.1556764Z s3-bucket: gha-artifacts 2024-08-06T20:40:08.1557432Z region: us-east-1 2024-08-06T20:40:08.1557764Z env: 2024-08-06T20:40:08.1558036Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:08.1558469Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:08.1558933Z ##[endgroup] 2024-08-06T20:40:08.7553030Z (node:104223) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-06T20:40:08.7553655Z 2024-08-06T20:40:08.7553911Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-06T20:40:08.7554595Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-06T20:40:08.7555310Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-06T20:40:08.8700133Z Found 1 objects with prefix pytorch/pytorch/10273124344/linux-focal-cuda11.8-py3.10-gcc9/ 2024-08-06T20:40:08.8701111Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-06T20:40:36.8614230Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-06T20:40:36.8624925Z Artifact download has finished successfully 2024-08-06T20:40:36.8860809Z ##[group]Run unzip -o artifacts.zip 2024-08-06T20:40:36.8861271Z unzip -o artifacts.zip 2024-08-06T20:40:36.8869235Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:36.8869719Z env: 2024-08-06T20:40:36.8869998Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:36.8870434Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:36.8870896Z ##[endgroup] 2024-08-06T20:40:36.8907721Z Archive: artifacts.zip 2024-08-06T20:40:36.8909698Z creating: dist/ 2024-08-06T20:40:39.3451467Z inflating: dist/torch-2.5.0a0+gitb9d86fa-cp310-cp310-linux_x86_64.whl 2024-08-06T20:40:39.3453060Z creating: build/custom_test_artifacts/ 2024-08-06T20:40:39.3453615Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-06T20:40:39.3454309Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-06T20:40:39.3455056Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-06T20:40:39.3461210Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-06T20:40:39.3462054Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-06T20:40:39.3463224Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-06T20:40:39.3464144Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-06T20:40:39.3465014Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-06T20:40:39.3467380Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-06T20:40:39.3469349Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-06T20:40:39.3470435Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-06T20:40:39.3471306Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-06T20:40:39.3472530Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-06T20:40:39.3474316Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-06T20:40:39.3476549Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-06T20:40:39.3477947Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-06T20:40:39.3479393Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-06T20:40:39.3480487Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-06T20:40:39.3481429Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-06T20:40:39.3482307Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-06T20:40:39.3539574Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-06T20:40:39.3540859Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-06T20:40:39.3542133Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-06T20:40:39.3543462Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-06T20:40:39.3544781Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-06T20:40:39.3546035Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-06T20:40:39.3547267Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-06T20:40:39.3548519Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-06T20:40:39.3549762Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-06T20:40:39.3595446Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-06T20:40:39.3645646Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-06T20:40:39.3646922Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-06T20:40:39.3648090Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-06T20:40:39.3649216Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-06T20:40:39.3650317Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-06T20:40:39.3651431Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-06T20:40:39.3652520Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-06T20:40:39.3653622Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-06T20:40:39.3728725Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-06T20:40:39.3804280Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-06T20:40:39.3805357Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-06T20:40:39.3806257Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-06T20:40:39.3807024Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-06T20:40:39.3807834Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-06T20:40:39.3808674Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-06T20:40:39.3809581Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-06T20:40:39.3810646Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-06T20:40:39.3811673Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-06T20:40:39.3812596Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-06T20:40:39.3813569Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-06T20:40:39.3814547Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-06T20:40:39.3815523Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-06T20:40:39.3816476Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-06T20:40:39.3817439Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-06T20:40:39.3839005Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-06T20:40:39.4017754Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-06T20:40:39.4018652Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-06T20:40:39.4019631Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-06T20:40:39.4020730Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-06T20:40:39.4021764Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-06T20:40:39.4022742Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-06T20:40:39.4023762Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-06T20:40:39.4024791Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-06T20:40:39.4025804Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-06T20:40:39.4026825Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-06T20:40:39.4027828Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-06T20:40:39.4050304Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-06T20:40:39.4154269Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-06T20:40:39.4155348Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-06T20:40:39.4156325Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-06T20:40:39.4157740Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-06T20:40:39.4158711Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-06T20:40:39.4159521Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-06T20:40:39.4160334Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-06T20:40:39.4162628Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-06T20:40:39.4163360Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-06T20:40:39.4164267Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-06T20:40:39.4313360Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-06T20:40:39.4391670Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-06T20:40:39.4392325Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-06T20:40:39.4392952Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-06T20:40:39.4393657Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-06T20:40:39.4401751Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-06T20:40:39.4402584Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-06T20:40:39.4403384Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-06T20:40:39.4404274Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-06T20:40:39.4405127Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-06T20:40:39.4406125Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-06T20:40:39.4407842Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-06T20:40:39.4408742Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-06T20:40:39.4409613Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-06T20:40:39.4411586Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-06T20:40:39.4413158Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-06T20:40:39.4415349Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-06T20:40:39.4416368Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-06T20:40:39.4417931Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-06T20:40:39.4418989Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-06T20:40:39.4419932Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-06T20:40:39.4420821Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-06T20:40:39.4477951Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-06T20:40:39.4479208Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-06T20:40:39.4480491Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-06T20:40:39.4481795Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-06T20:40:39.4483079Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-06T20:40:39.4484601Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-06T20:40:39.4485846Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-06T20:40:39.4487088Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-06T20:40:39.4488301Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-06T20:40:39.4534409Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-06T20:40:39.4584207Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-06T20:40:39.4585694Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-06T20:40:39.4586898Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-06T20:40:39.4587993Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-06T20:40:39.4589097Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-06T20:40:39.4590199Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-06T20:40:39.4591267Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-06T20:40:39.4592333Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-06T20:40:39.4666917Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-06T20:40:39.4742631Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-06T20:40:39.4743691Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-06T20:40:39.4744559Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-06T20:40:39.4745326Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-06T20:40:39.4746124Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-06T20:40:39.4746949Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-06T20:40:39.4747901Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-06T20:40:39.4748971Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-06T20:40:39.4750014Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-06T20:40:39.4750960Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-06T20:40:39.4751949Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-06T20:40:39.4752951Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-06T20:40:39.4753937Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-06T20:40:39.4754942Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-06T20:40:39.4755924Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-06T20:40:39.4778143Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-06T20:40:39.4858666Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-06T20:40:39.4859752Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-06T20:40:39.4860718Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-06T20:40:39.4861612Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-06T20:40:39.4862404Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-06T20:40:39.4863207Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-06T20:40:39.4864011Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-06T20:40:39.4866195Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-06T20:40:39.4867080Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-06T20:40:39.4867965Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-06T20:40:39.4929787Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-06T20:40:39.4930466Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-06T20:40:39.4931130Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-06T20:40:39.4931919Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-06T20:40:39.4939779Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-06T20:40:39.4940652Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-06T20:40:39.4941535Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-06T20:40:39.4942474Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-06T20:40:39.4943419Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-06T20:40:39.4944448Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-06T20:40:39.4945822Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-06T20:40:39.4946776Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-06T20:40:39.4947703Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-06T20:40:39.4949427Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-06T20:40:39.4951221Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-06T20:40:39.4953498Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-06T20:40:39.4954598Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-06T20:40:39.4956011Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-06T20:40:39.4957375Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-06T20:40:39.4958393Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-06T20:40:39.4959328Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-06T20:40:39.5016157Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-06T20:40:39.5017497Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-06T20:40:39.5019140Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-06T20:40:39.5020537Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-06T20:40:39.5021918Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-06T20:40:39.5023235Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-06T20:40:39.5024555Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-06T20:40:39.5025873Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-06T20:40:39.5027233Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-06T20:40:39.5072310Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-06T20:40:39.5122299Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-06T20:40:39.5123603Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-06T20:40:39.5124836Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-06T20:40:39.5126050Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-06T20:40:39.5127223Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-06T20:40:39.5128404Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-06T20:40:39.5129558Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-06T20:40:39.5130725Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-06T20:40:39.5205303Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-06T20:40:39.5281028Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-06T20:40:39.5282177Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-06T20:40:39.5283142Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-06T20:40:39.5283960Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-06T20:40:39.5284835Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-06T20:40:39.5285748Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-06T20:40:39.5286773Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-06T20:40:39.5287911Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-06T20:40:39.5289032Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-06T20:40:39.5290085Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-06T20:40:39.5291148Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-06T20:40:39.5292419Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-06T20:40:39.5293634Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-06T20:40:39.5294716Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-06T20:40:39.5295763Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-06T20:40:39.5296945Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-06T20:40:39.5453600Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-06T20:40:39.5454665Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-06T20:40:39.5455722Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-06T20:40:39.5456943Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-06T20:40:39.5458108Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-06T20:40:39.5459195Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-06T20:40:39.5460321Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-06T20:40:39.5461458Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-06T20:40:39.5462590Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-06T20:40:39.5463708Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-06T20:40:39.5464837Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-06T20:40:39.5486075Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-06T20:40:39.5555901Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-06T20:40:39.5557073Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-06T20:40:39.5558348Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-06T20:40:39.5559296Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-06T20:40:39.5560153Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-06T20:40:39.5561027Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-06T20:40:39.5561913Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-06T20:40:39.5564016Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-06T20:40:39.5564778Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-06T20:40:39.5565594Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-06T20:40:39.5694614Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-06T20:40:39.5748152Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-06T20:40:39.5748759Z creating: build/lib/ 2024-08-06T20:40:39.5760438Z inflating: build/lib/libpthreadpool.a 2024-08-06T20:40:39.5770586Z inflating: build/lib/libcpuinfo.a 2024-08-06T20:40:39.5780260Z inflating: build/lib/libcpuinfo_internals.a 2024-08-06T20:40:39.5781450Z inflating: build/lib/libclog.a 2024-08-06T20:40:39.5892501Z inflating: build/lib/libprotobuf-lite.a 2024-08-06T20:40:39.5894954Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-06T20:40:39.6461799Z inflating: build/lib/libprotobuf.a 2024-08-06T20:40:39.6542990Z inflating: build/lib/libgtest.a 2024-08-06T20:40:39.6635970Z inflating: build/lib/libbenchmark.a 2024-08-06T20:40:39.6645513Z inflating: build/lib/libittnotify.a 2024-08-06T20:40:39.6679914Z inflating: build/lib/libtensorpipe_uv.a 2024-08-06T20:40:39.6702707Z inflating: build/lib/libnnpack.a 2024-08-06T20:40:39.6782115Z inflating: build/lib/libasmjit.a 2024-08-06T20:40:39.6940386Z inflating: build/lib/libgloo.a 2024-08-06T20:40:39.6966486Z inflating: build/lib/libfmt.a 2024-08-06T20:40:39.7083753Z inflating: build/lib/libc10.so 2024-08-06T20:40:39.7085444Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-06T20:40:39.7086191Z inflating: build/lib/libfoxi_loader.a 2024-08-06T20:40:39.7088224Z inflating: build/lib/libtorch_global_deps.so 2024-08-06T20:40:39.7112265Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-06T20:40:39.7134831Z inflating: build/lib/libgmock.a 2024-08-06T20:40:39.7135636Z inflating: build/lib/libgtest_main.a 2024-08-06T20:40:39.7136584Z inflating: build/lib/libbenchmark_main.a 2024-08-06T20:40:39.7762536Z inflating: build/lib/libprotoc.a 2024-08-06T20:40:41.0373036Z inflating: build/lib/libdnnl.a 2024-08-06T20:40:41.1085493Z inflating: build/lib/libtensorpipe.a 2024-08-06T20:40:41.1156650Z inflating: build/lib/libc10_cuda.so 2024-08-06T20:40:41.1157880Z inflating: build/lib/libgmock_main.a 2024-08-06T20:40:41.2714992Z inflating: build/lib/libfbgemm.a 2024-08-06T20:40:41.3342989Z inflating: build/lib/libkineto.a 2024-08-06T20:40:41.3660789Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-06T20:40:41.3889931Z inflating: build/lib/libXNNPACK.a 2024-08-06T20:40:41.3942659Z inflating: build/lib/libonnx_proto.a 2024-08-06T20:40:41.4426109Z inflating: build/lib/libgloo_cuda.a 2024-08-06T20:40:41.5286735Z inflating: build/lib/libonnx.a 2024-08-06T20:40:44.5402544Z inflating: build/lib/libtorch_cpu.so 2024-08-06T20:40:44.5406933Z inflating: build/lib/libshm.so 2024-08-06T20:40:44.5413042Z inflating: build/lib/libunbox_lib.a 2024-08-06T20:40:47.0112203Z inflating: build/lib/libtorch_cuda.so 2024-08-06T20:40:47.0113153Z inflating: build/lib/libtorch.so 2024-08-06T20:40:47.0116376Z inflating: build/lib/libc10d_cuda_test.so 2024-08-06T20:40:48.1505952Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-06T20:40:48.1532105Z inflating: build/lib/libjitbackend_test.so 2024-08-06T20:40:48.1622339Z inflating: build/lib/libtorchbind_test.so 2024-08-06T20:40:48.1655129Z inflating: build/lib/libaoti_custom_ops.so 2024-08-06T20:40:48.1686859Z inflating: build/lib/libbackend_with_compiler.so 2024-08-06T20:40:48.4176433Z inflating: build/lib/libtorch_python.so 2024-08-06T20:40:48.4219129Z inflating: build/lib/libnnapi_backend.so 2024-08-06T20:40:48.4219577Z creating: build/bin/ 2024-08-06T20:40:48.4282230Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-06T20:40:48.4346301Z inflating: build/bin/c10_DeviceGuard_test 2024-08-06T20:40:48.4409341Z inflating: build/bin/c10_Device_test 2024-08-06T20:40:48.4482870Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-06T20:40:48.4549370Z inflating: build/bin/c10_Scalar_test 2024-08-06T20:40:48.4610609Z inflating: build/bin/c10_StreamGuard_test 2024-08-06T20:40:48.4674144Z inflating: build/bin/c10_SymInt_test 2024-08-06T20:40:48.4742083Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-06T20:40:48.4811627Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-06T20:40:48.4881976Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-06T20:40:48.4970318Z inflating: build/bin/c10_cow_test 2024-08-06T20:40:48.5036568Z inflating: build/bin/c10_Bitset_test 2024-08-06T20:40:48.5098062Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-06T20:40:48.5159911Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-06T20:40:48.5223134Z inflating: build/bin/c10_Half_test 2024-08-06T20:40:48.5293371Z inflating: build/bin/c10_LeftRight_test 2024-08-06T20:40:48.5362116Z inflating: build/bin/c10_Metaprogramming_test 2024-08-06T20:40:48.5424468Z inflating: build/bin/c10_Synchronized_test 2024-08-06T20:40:48.5488817Z inflating: build/bin/c10_TypeIndex_test 2024-08-06T20:40:48.5558505Z inflating: build/bin/c10_ThreadLocal_test 2024-08-06T20:40:48.5622869Z inflating: build/bin/c10_TypeList_test 2024-08-06T20:40:48.5684029Z inflating: build/bin/c10_TypeTraits_test 2024-08-06T20:40:48.5748772Z inflating: build/bin/c10_accumulate_test 2024-08-06T20:40:48.5818053Z inflating: build/bin/c10_bfloat16_test 2024-08-06T20:40:48.5881272Z inflating: build/bin/c10_bit_cast_test 2024-08-06T20:40:48.5951389Z inflating: build/bin/c10_complex_math_test 2024-08-06T20:40:48.6020970Z inflating: build/bin/c10_complex_test 2024-08-06T20:40:48.6086808Z inflating: build/bin/c10_exception_test 2024-08-06T20:40:48.6149538Z inflating: build/bin/c10_flags_test 2024-08-06T20:40:48.6212125Z inflating: build/bin/c10_generic_math_test 2024-08-06T20:40:48.6417920Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-06T20:40:48.6481285Z inflating: build/bin/c10_irange_test 2024-08-06T20:40:48.6548066Z inflating: build/bin/c10_lazy_test 2024-08-06T20:40:48.6619200Z inflating: build/bin/c10_logging_test 2024-08-06T20:40:48.6712661Z inflating: build/bin/c10_optional_test 2024-08-06T20:40:48.6790664Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-06T20:40:48.6857905Z inflating: build/bin/c10_registry_test 2024-08-06T20:40:48.7046664Z inflating: build/bin/c10_small_vector_test 2024-08-06T20:40:48.7110839Z inflating: build/bin/c10_ssize_test 2024-08-06T20:40:48.7175881Z inflating: build/bin/c10_string_util_test 2024-08-06T20:40:48.7249088Z inflating: build/bin/c10_string_view_test 2024-08-06T20:40:48.7311953Z inflating: build/bin/c10_tempfile_test 2024-08-06T20:40:48.7381963Z inflating: build/bin/c10_typeid_test 2024-08-06T20:40:48.7442719Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-06T20:40:48.7999835Z inflating: build/bin/protoc-3.13.0.0 2024-08-06T20:40:48.8556439Z inflating: build/bin/protoc 2024-08-06T20:40:48.8622556Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-06T20:40:48.8688499Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-06T20:40:48.8754435Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-06T20:40:48.8820482Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-06T20:40:48.8885538Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-06T20:40:48.8951337Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-06T20:40:48.9012689Z inflating: build/bin/c10_cuda_CUDATest 2024-08-06T20:40:48.9078539Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-06T20:40:48.9490382Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-06T20:40:48.9920213Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-06T20:40:49.0368899Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-06T20:40:49.0435591Z inflating: build/bin/HashStoreTest 2024-08-06T20:40:49.0501800Z inflating: build/bin/FileStoreTest 2024-08-06T20:40:49.0567578Z inflating: build/bin/BackoffTest 2024-08-06T20:40:49.0636960Z inflating: build/bin/TCPStoreTest 2024-08-06T20:40:49.0654152Z inflating: build/bin/ProcessGroupMPITest 2024-08-06T20:40:49.0659367Z inflating: build/bin/torch_shm_manager 2024-08-06T20:40:49.0726536Z inflating: build/bin/test_edge_op_registration 2024-08-06T20:40:49.0730415Z inflating: build/bin/example_allreduce 2024-08-06T20:40:49.0795227Z inflating: build/bin/Dimname_test 2024-08-06T20:40:49.0887444Z inflating: build/bin/Dict_test 2024-08-06T20:40:49.0967796Z inflating: build/bin/MaybeOwned_test 2024-08-06T20:40:49.1039536Z inflating: build/bin/NamedTensor_test 2024-08-06T20:40:49.1113069Z inflating: build/bin/apply_utils_test 2024-08-06T20:40:49.1186524Z inflating: build/bin/atest 2024-08-06T20:40:49.1265231Z inflating: build/bin/basic 2024-08-06T20:40:49.1328758Z inflating: build/bin/cpu_allocator_test 2024-08-06T20:40:49.1397205Z inflating: build/bin/broadcast_test 2024-08-06T20:40:49.1470089Z inflating: build/bin/cpu_generator_test 2024-08-06T20:40:49.1536820Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-06T20:40:49.1598822Z inflating: build/bin/dispatch_key_set_test 2024-08-06T20:40:49.1661621Z inflating: build/bin/dlconvertor_test 2024-08-06T20:40:49.1778184Z inflating: build/bin/cpu_rng_test 2024-08-06T20:40:49.1851052Z inflating: build/bin/extension_backend_test 2024-08-06T20:40:49.1920163Z inflating: build/bin/half_test 2024-08-06T20:40:49.2039803Z inflating: build/bin/ivalue_test 2024-08-06T20:40:49.2101864Z inflating: build/bin/lazy_tensor_test 2024-08-06T20:40:49.2169430Z inflating: build/bin/memory_format_test 2024-08-06T20:40:49.2236194Z inflating: build/bin/memory_overlapping_test 2024-08-06T20:40:49.2304335Z inflating: build/bin/math_kernel_test 2024-08-06T20:40:49.2370584Z inflating: build/bin/mobile_memory_cleanup 2024-08-06T20:40:49.2440439Z inflating: build/bin/native_test 2024-08-06T20:40:49.2503627Z inflating: build/bin/operator_name_test 2024-08-06T20:40:49.2568037Z inflating: build/bin/operators_test 2024-08-06T20:40:49.2633169Z inflating: build/bin/packedtensoraccessor_test 2024-08-06T20:40:49.2716896Z inflating: build/bin/pow_test 2024-08-06T20:40:49.2788222Z inflating: build/bin/quantized_test 2024-08-06T20:40:49.2852358Z inflating: build/bin/reportMemoryUsage_test 2024-08-06T20:40:49.2914600Z inflating: build/bin/reduce_ops_test 2024-08-06T20:40:49.2985284Z inflating: build/bin/scalar_tensor_test 2024-08-06T20:40:49.3050279Z inflating: build/bin/StorageUtils_test 2024-08-06T20:40:49.3123313Z inflating: build/bin/scalar_test 2024-08-06T20:40:49.3188802Z inflating: build/bin/stride_properties_test 2024-08-06T20:40:49.3287291Z inflating: build/bin/tensor_iterator_test 2024-08-06T20:40:49.3354842Z inflating: build/bin/test_parallel 2024-08-06T20:40:49.3358632Z inflating: build/bin/thread_init_test 2024-08-06T20:40:49.3428137Z inflating: build/bin/type_ptr_test 2024-08-06T20:40:49.3502989Z inflating: build/bin/type_test 2024-08-06T20:40:49.3568326Z inflating: build/bin/undefined_tensor_test 2024-08-06T20:40:49.3570304Z inflating: build/bin/verify_api_visibility 2024-08-06T20:40:49.3634783Z inflating: build/bin/weakref_test 2024-08-06T20:40:49.3721509Z inflating: build/bin/legacy_vmap_test 2024-08-06T20:40:49.3785550Z inflating: build/bin/wrapdim_test 2024-08-06T20:40:49.3918443Z inflating: build/bin/List_test 2024-08-06T20:40:49.3993311Z inflating: build/bin/IListRef_test 2024-08-06T20:40:49.4058158Z inflating: build/bin/xla_tensor_test 2024-08-06T20:40:49.4210357Z inflating: build/bin/kernel_function_legacy_test 2024-08-06T20:40:49.4331194Z inflating: build/bin/kernel_function_test 2024-08-06T20:40:49.4414414Z inflating: build/bin/KernelFunction_test 2024-08-06T20:40:49.4574532Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-06T20:40:49.4704074Z inflating: build/bin/kernel_lambda_test 2024-08-06T20:40:49.4780362Z inflating: build/bin/kernel_stackbased_test 2024-08-06T20:40:49.4844241Z inflating: build/bin/CppSignature_test 2024-08-06T20:40:49.4964641Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-06T20:40:49.5025888Z inflating: build/bin/op_allowlist_test 2024-08-06T20:40:49.5095441Z inflating: build/bin/backend_fallback_test 2024-08-06T20:40:49.5174980Z inflating: build/bin/inline_container_test 2024-08-06T20:40:49.5552020Z inflating: build/bin/op_registration_test 2024-08-06T20:40:49.5618372Z inflating: build/bin/cuda_apply_test 2024-08-06T20:40:49.5683342Z inflating: build/bin/cuda_allocator_test 2024-08-06T20:40:49.5751409Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-06T20:40:49.5839449Z inflating: build/bin/cuda_complex_math_test 2024-08-06T20:40:49.5913185Z inflating: build/bin/cuda_atomic_ops_test 2024-08-06T20:40:49.5986873Z inflating: build/bin/cuda_complex_test 2024-08-06T20:40:49.6048993Z inflating: build/bin/cuda_device_test 2024-08-06T20:40:49.6121189Z inflating: build/bin/cuda_cub_test 2024-08-06T20:40:49.6184542Z inflating: build/bin/cuda_dlconvertor_test 2024-08-06T20:40:49.6248363Z inflating: build/bin/cuda_integer_divider_test 2024-08-06T20:40:49.6330083Z inflating: build/bin/cuda_distributions_test 2024-08-06T20:40:49.6400971Z inflating: build/bin/cuda_generator_test 2024-08-06T20:40:49.6463192Z inflating: build/bin/cuda_half_test 2024-08-06T20:40:49.6524805Z inflating: build/bin/cuda_optional_test 2024-08-06T20:40:49.6587018Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-06T20:40:49.6662859Z inflating: build/bin/cuda_stream_test 2024-08-06T20:40:49.6728595Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-06T20:40:49.6792700Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-06T20:40:49.6854599Z inflating: build/bin/cuda_cudnn_test 2024-08-06T20:40:49.6919503Z inflating: build/bin/cuda_vectorized_test 2024-08-06T20:40:49.6937332Z inflating: build/bin/tutorial_tensorexpr 2024-08-06T20:40:49.7019354Z inflating: build/bin/ProcessGroupGlooTest 2024-08-06T20:40:49.7091276Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-06T20:40:49.7156783Z inflating: build/bin/ProcessGroupUCCTest 2024-08-06T20:40:49.7234843Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-06T20:40:49.7314406Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-06T20:40:49.7382916Z inflating: build/bin/test_dist_autograd 2024-08-06T20:40:49.7386086Z inflating: build/bin/parallel_benchmark 2024-08-06T20:40:49.7469821Z inflating: build/bin/test_mobile_nnc 2024-08-06T20:40:49.7555945Z inflating: build/bin/test_cpp_rpc 2024-08-06T20:40:49.7567344Z inflating: build/bin/aot_model_compiler_test 2024-08-06T20:40:49.8598581Z inflating: build/bin/test_tensorexpr 2024-08-06T20:40:49.9031259Z inflating: build/bin/test_lazy 2024-08-06T20:40:50.0502616Z inflating: build/bin/test_api 2024-08-06T20:40:50.1225995Z inflating: build/bin/test_jit 2024-08-06T20:40:50.1226422Z creating: .additional_ci_files/ 2024-08-06T20:40:50.1298385Z inflating: .additional_ci_files/test-times.json 2024-08-06T20:40:50.1584957Z inflating: .additional_ci_files/test-class-times.json 2024-08-06T20:40:50.1629527Z ##[group]Run rm artifacts.zip 2024-08-06T20:40:50.1629964Z rm artifacts.zip 2024-08-06T20:40:50.1638031Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:50.1638521Z env: 2024-08-06T20:40:50.1638795Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:50.1639228Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:50.1639691Z ##[endgroup] 2024-08-06T20:40:50.2805729Z ##[group]Run df -H 2024-08-06T20:40:50.2806090Z df -H 2024-08-06T20:40:50.2814096Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:50.2814563Z env: 2024-08-06T20:40:50.2814859Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:50.2815295Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:50.2815743Z ##[endgroup] 2024-08-06T20:40:50.2849182Z Filesystem Size Used Avail Use% Mounted on 2024-08-06T20:40:50.2849686Z devtmpfs 129G 0 129G 0% /dev 2024-08-06T20:40:50.2850108Z tmpfs 129G 0 129G 0% /dev/shm 2024-08-06T20:40:50.2851282Z tmpfs 129G 627k 129G 1% /run 2024-08-06T20:40:50.2851749Z tmpfs 129G 0 129G 0% /sys/fs/cgroup 2024-08-06T20:40:50.2852180Z /dev/xvda1 162G 38G 125G 23% / 2024-08-06T20:40:50.2852617Z tmpfs 26G 0 26G 0% /run/user/0 2024-08-06T20:40:50.2891788Z Prepare all required actions 2024-08-06T20:40:50.2892313Z Getting action download info 2024-08-06T20:40:50.4041302Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-06T20:40:50.4041760Z with: 2024-08-06T20:40:50.4042039Z env: 2024-08-06T20:40:50.4042313Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:50.4042748Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:50.4043208Z ##[endgroup] 2024-08-06T20:40:50.4088278Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-06T20:40:50.4088725Z with: 2024-08-06T20:40:50.4089013Z name: td_results 2024-08-06T20:40:50.4089320Z s3-bucket: gha-artifacts 2024-08-06T20:40:50.4089663Z region: us-east-1 2024-08-06T20:40:50.4089954Z env: 2024-08-06T20:40:50.4090216Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:50.4090652Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:50.4091114Z ##[endgroup] 2024-08-06T20:40:51.0102484Z (node:104247) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-06T20:40:51.0103113Z 2024-08-06T20:40:51.0103389Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-06T20:40:51.0104042Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-06T20:40:51.0104748Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-06T20:40:51.0959685Z Found 1 objects with prefix pytorch/pytorch/10273124344/td_results/ 2024-08-06T20:40:51.0960500Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-06T20:40:51.1527638Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-06T20:40:51.1538192Z Artifact download has finished successfully 2024-08-06T20:40:51.1747451Z ##[group]Run mkdir -p .additional_ci_files 2024-08-06T20:40:51.1747932Z mkdir -p .additional_ci_files 2024-08-06T20:40:51.1748459Z mv td_results.json .additional_ci_files/td_results.json 2024-08-06T20:40:51.1756979Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:51.1757801Z env: 2024-08-06T20:40:51.1758077Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:51.1758512Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:51.1758977Z ##[endgroup] 2024-08-06T20:40:51.1857855Z ##[group]Run .github/scripts/parse_ref.py 2024-08-06T20:40:51.1858466Z .github/scripts/parse_ref.py 2024-08-06T20:40:51.1866123Z shell: /usr/bin/bash -e {0} 2024-08-06T20:40:51.1866546Z env: 2024-08-06T20:40:51.1866845Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:51.1867286Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:51.1867763Z ##[endgroup] 2024-08-06T20:40:51.2162411Z Prepare all required actions 2024-08-06T20:40:51.2208539Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-06T20:40:51.2208999Z with: 2024-08-06T20:40:51.2209516Z github-token: *** 2024-08-06T20:40:51.2209833Z env: 2024-08-06T20:40:51.2210106Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:51.2210554Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:51.2211003Z ##[endgroup] 2024-08-06T20:40:51.2236193Z ##[group]Run set -eux 2024-08-06T20:40:51.2236548Z set -eux 2024-08-06T20:40:51.2237348Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-06T20:40:51.2244871Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:51.2245360Z env: 2024-08-06T20:40:51.2245659Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:51.2246079Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:51.2246734Z GITHUB_TOKEN: *** 2024-08-06T20:40:51.2247061Z ##[endgroup] 2024-08-06T20:40:51.2270083Z + python3 .github/scripts/get_workflow_job_id.py 10273124344 i-01c85e67ad0689ec5 2024-08-06T20:40:55.2549573Z setting job-id=28428108140 2024-08-06T20:40:55.2550341Z setting job-name=linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:40:55.2764724Z Prepare all required actions 2024-08-06T20:40:55.2765341Z Getting action download info 2024-08-06T20:40:55.3886739Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-06T20:40:55.3887194Z with: 2024-08-06T20:40:55.3887708Z github-token: *** 2024-08-06T20:40:55.3888996Z test-matrix: {"include": [{"config": "distributed", "shard": 1, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 2, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 3, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}]} 2024-08-06T20:40:55.3890646Z job-name: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:40:55.3891317Z env: 2024-08-06T20:40:55.3891610Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:55.3892033Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:55.3892495Z ##[endgroup] 2024-08-06T20:40:55.3943884Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-06T20:40:55.3944426Z with: 2024-08-06T20:40:55.3944714Z shell: bash 2024-08-06T20:40:55.3945004Z timeout_minutes: 10 2024-08-06T20:40:55.3945340Z max_attempts: 5 2024-08-06T20:40:55.3945658Z retry_wait_seconds: 30 2024-08-06T20:40:55.3946600Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-06T20:40:55.3947603Z polling_interval_seconds: 1 2024-08-06T20:40:55.3947980Z warning_on_retry: true 2024-08-06T20:40:55.3948331Z continue_on_error: false 2024-08-06T20:40:55.3948652Z env: 2024-08-06T20:40:55.3948948Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:55.3949374Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:55.3950009Z GITHUB_TOKEN: *** 2024-08-06T20:40:55.3950329Z ##[endgroup] 2024-08-06T20:40:55.4676460Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-06T20:40:55.7561906Z Defaulting to user installation because normal site-packages is not writeable 2024-08-06T20:40:55.7768726Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.7/site-packages (2.27.1) 2024-08-06T20:40:55.7951609Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.7/site-packages (6.0.1) 2024-08-06T20:40:55.7963183Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (3.7) 2024-08-06T20:40:55.7978940Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-06T20:40:55.7989802Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (1.26.19) 2024-08-06T20:40:55.8237779Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2.0.12) 2024-08-06T20:40:56.4678562Z Command completed after 1 attempt(s). 2024-08-06T20:40:56.4743752Z ##[group]Run set -x 2024-08-06T20:40:56.4744124Z set -x 2024-08-06T20:40:56.4744421Z  2024-08-06T20:40:56.4744927Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-06T20:40:56.4745538Z # in runner workspace 2024-08-06T20:40:56.4746023Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-06T20:40:56.4754338Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:56.4754835Z env: 2024-08-06T20:40:56.4755118Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:56.4755827Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:56.4756311Z ##[endgroup] 2024-08-06T20:40:56.4782863Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-06T20:40:56.5065482Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-06T20:40:56.5066024Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-06T20:40:56.5066451Z echo "Job name: ${JOB_NAME}" 2024-08-06T20:40:56.5066842Z  2024-08-06T20:40:56.5067329Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-06T20:40:56.5067912Z # in runner workspace 2024-08-06T20:40:56.5068458Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-06T20:40:56.5069067Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-06T20:40:56.5069492Z  --job-name "${JOB_NAME}" \ 2024-08-06T20:40:56.5070956Z  --test-matrix "{"include": [{"config": "distributed", "shard": 1, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 2, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 3, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}]}" \ 2024-08-06T20:40:56.5072402Z  --selected-test-configs "" \ 2024-08-06T20:40:56.5072843Z  --pr-number "${PR_NUMBER}" \ 2024-08-06T20:40:56.5073239Z  --tag "${TAG}" \ 2024-08-06T20:40:56.5073631Z  --event-name "${EVENT_NAME}" \ 2024-08-06T20:40:56.5074053Z  --schedule "${SCHEDULE}" \ 2024-08-06T20:40:56.5074445Z  --branch "${HEAD_BRANCH}" 2024-08-06T20:40:56.5082204Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:56.5082689Z env: 2024-08-06T20:40:56.5082987Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:56.5099359Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:56.5100134Z GITHUB_TOKEN: *** 2024-08-06T20:40:56.5100716Z JOB_NAME: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:40:56.5101381Z PR_NUMBER: 132710 2024-08-06T20:40:56.5101700Z TAG: 2024-08-06T20:40:56.5101980Z EVENT_NAME: pull_request 2024-08-06T20:40:56.5102340Z SCHEDULE: 2024-08-06T20:40:56.5102627Z HEAD_BRANCH: 2024-08-06T20:40:56.5102938Z ##[endgroup] 2024-08-06T20:40:56.5126210Z Workflow: pull 2024-08-06T20:40:56.5126826Z Job name: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:40:56.7348043Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-06T20:40:56.8646969Z ##[group]Run echo "Filtered matrix:" 2024-08-06T20:40:56.8647412Z echo "Filtered matrix:" 2024-08-06T20:40:56.8648759Z echo "{"include": [{"config": "distributed", "shard": 1, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 2, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 3, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}]}" 2024-08-06T20:40:56.8650166Z  2024-08-06T20:40:56.8650457Z echo 2024-08-06T20:40:56.8650816Z echo "Is the current job unstable? False" 2024-08-06T20:40:56.8651260Z  2024-08-06T20:40:56.8651550Z echo 2024-08-06T20:40:56.8651879Z echo "Is keep-going label set? False" 2024-08-06T20:40:56.8652298Z  2024-08-06T20:40:56.8652583Z echo 2024-08-06T20:40:56.8652885Z echo "Renabled issues? " 2024-08-06T20:40:56.8660538Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:56.8661025Z env: 2024-08-06T20:40:56.8661306Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:56.8661742Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:56.8662202Z ##[endgroup] 2024-08-06T20:40:56.8685274Z Filtered matrix: 2024-08-06T20:40:56.8686772Z {include: [{config: distributed, shard: 1, num_shards: 3, runner: linux.8xlarge.nvidia.gpu}, {config: distributed, shard: 2, num_shards: 3, runner: linux.8xlarge.nvidia.gpu}, {config: distributed, shard: 3, num_shards: 3, runner: linux.8xlarge.nvidia.gpu}]} 2024-08-06T20:40:56.8688206Z 2024-08-06T20:40:56.8688507Z Is the current job unstable? False 2024-08-06T20:40:56.8688788Z 2024-08-06T20:40:56.8688961Z Is keep-going label set? False 2024-08-06T20:40:56.8689205Z 2024-08-06T20:40:56.8689336Z Renabled issues? 2024-08-06T20:40:56.8750859Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-06T20:40:56.8751627Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-06T20:40:56.8759470Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:40:56.8759961Z env: 2024-08-06T20:40:56.8760245Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:56.8760687Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:56.8761157Z JOB_TIMEOUT: 360 2024-08-06T20:40:56.8761474Z ##[endgroup] 2024-08-06T20:40:56.8843233Z ##[group]Run set -x 2024-08-06T20:40:56.8843675Z set -x 2024-08-06T20:40:56.8843964Z  2024-08-06T20:40:56.8844298Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-06T20:40:56.8844810Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-06T20:40:56.8845301Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-06T20:40:56.8845768Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-06T20:40:56.8846169Z else 2024-08-06T20:40:56.8846488Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-06T20:40:56.8846894Z fi 2024-08-06T20:40:56.8847177Z  2024-08-06T20:40:56.8847595Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-06T20:40:56.8848261Z # TODO: Stop building test binaries as part of the build phase 2024-08-06T20:40:56.8848850Z # Used for GPU_FLAG since that doesn't play nice 2024-08-06T20:40:56.8849399Z # shellcheck disable=SC2086,SC2090 2024-08-06T20:40:56.8849827Z container_name=$(docker run \ 2024-08-06T20:40:56.8850232Z  ${GPU_FLAG:-} \ 2024-08-06T20:40:56.8850601Z  -e BUILD_ENVIRONMENT \ 2024-08-06T20:40:56.8850975Z  -e PR_NUMBER \ 2024-08-06T20:40:56.8851337Z  -e GITHUB_ACTIONS \ 2024-08-06T20:40:56.8851716Z  -e GITHUB_REPOSITORY \ 2024-08-06T20:40:56.8852092Z  -e GITHUB_WORKFLOW \ 2024-08-06T20:40:56.8852466Z  -e GITHUB_JOB \ 2024-08-06T20:40:56.8852822Z  -e GITHUB_RUN_ID \ 2024-08-06T20:40:56.8853171Z  -e GITHUB_RUN_NUMBER \ 2024-08-06T20:40:56.8853557Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-06T20:40:56.8853943Z  -e JOB_ID \ 2024-08-06T20:40:56.8854264Z  -e JOB_NAME \ 2024-08-06T20:40:56.8854607Z  -e BASE_SHA \ 2024-08-06T20:40:56.8854946Z  -e BRANCH \ 2024-08-06T20:40:56.8855257Z  -e SHA1 \ 2024-08-06T20:40:56.8855608Z  -e AWS_DEFAULT_REGION \ 2024-08-06T20:40:56.8855997Z  -e IN_WHEEL_TEST \ 2024-08-06T20:40:56.8856345Z  -e SHARD_NUMBER \ 2024-08-06T20:40:56.8856708Z  -e TEST_CONFIG \ 2024-08-06T20:40:56.8857070Z  -e NUM_TEST_SHARDS \ 2024-08-06T20:40:56.8857436Z  -e REENABLED_ISSUES \ 2024-08-06T20:40:56.8857832Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-06T20:40:56.8858249Z  -e VERBOSE_TEST_LOGS \ 2024-08-06T20:40:56.8858620Z  -e TEST_SHOWLOCALS \ 2024-08-06T20:40:56.8858997Z  -e NO_TEST_TIMEOUT \ 2024-08-06T20:40:56.8859369Z  -e NO_TD \ 2024-08-06T20:40:56.8859690Z  -e TD_DISTRIBUTED \ 2024-08-06T20:40:56.8860060Z  -e PR_LABELS \ 2024-08-06T20:40:56.8860448Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-06T20:40:56.8860860Z  -e SCCACHE_BUCKET \ 2024-08-06T20:40:56.8861245Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-06T20:40:56.8861817Z  -e XLA_CUDA \ 2024-08-06T20:40:56.8862191Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-06T20:40:56.8862661Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-06T20:40:56.8863136Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-06T20:40:56.8863589Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-06T20:40:56.8864041Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-06T20:40:56.8864444Z  -e DASHBOARD_TAG \ 2024-08-06T20:40:56.8864865Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-06T20:40:56.8865372Z  --security-opt seccomp=unconfined \ 2024-08-06T20:40:56.8865815Z  --cap-add=SYS_PTRACE \ 2024-08-06T20:40:56.8866181Z  --ipc=host \ 2024-08-06T20:40:56.8866536Z  --shm-size="${SHM_SIZE}" \ 2024-08-06T20:40:56.8866918Z  --tty \ 2024-08-06T20:40:56.8867219Z  --detach \ 2024-08-06T20:40:56.8867575Z  --name="${container_name}" \ 2024-08-06T20:40:56.8868094Z  --user jenkins \ 2024-08-06T20:40:56.8868542Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-06T20:40:56.8869053Z  -w /var/lib/jenkins/workspace \ 2024-08-06T20:40:56.8869469Z  "${DOCKER_IMAGE}" 2024-08-06T20:40:56.8869801Z ) 2024-08-06T20:40:56.8870186Z # Propagate download.pytorch.org IP to container 2024-08-06T20:40:56.8871007Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-06T20:40:56.8871855Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-06T20:40:56.8872662Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-06T20:40:56.8880338Z shell: /usr/bin/bash -e {0} 2024-08-06T20:40:56.8880685Z env: 2024-08-06T20:40:56.8880974Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:40:56.8881406Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:40:56.8881971Z BUILD_ENVIRONMENT: linux-focal-cuda11.8-py3.10-gcc9 2024-08-06T20:40:56.8882419Z PR_NUMBER: 132710 2024-08-06T20:40:56.8882759Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-06T20:40:56.8883143Z GITHUB_WORKFLOW: pull 2024-08-06T20:40:56.8883472Z GITHUB_JOB: test 2024-08-06T20:40:56.8883790Z GITHUB_RUN_ID: 10273124344 2024-08-06T20:40:56.8884135Z GITHUB_RUN_NUMBER: 233985 2024-08-06T20:40:56.8884490Z GITHUB_RUN_ATTEMPT: 1 2024-08-06T20:40:56.8884813Z JOB_ID: 28428108140 2024-08-06T20:40:56.8885382Z JOB_NAME: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:40:56.8886036Z BRANCH: pull/132710 2024-08-06T20:40:56.8886402Z SHA1: b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:40:56.8886867Z BASE_SHA: 1736af7cf736184c356be1bb00f59fb2feea6d7d 2024-08-06T20:40:56.8887310Z TEST_CONFIG: distributed 2024-08-06T20:40:56.8887656Z SHARD_NUMBER: 3 2024-08-06T20:40:56.8887947Z NUM_TEST_SHARDS: 3 2024-08-06T20:40:56.8888288Z REENABLED_ISSUES: 2024-08-06T20:40:56.8888625Z CONTINUE_THROUGH_ERROR: False 2024-08-06T20:40:56.8888979Z VERBOSE_TEST_LOGS: False 2024-08-06T20:40:56.8889333Z TEST_SHOWLOCALS: False 2024-08-06T20:40:56.8889675Z NO_TEST_TIMEOUT: False 2024-08-06T20:40:56.8889993Z NO_TD: False 2024-08-06T20:40:56.8890299Z TD_DISTRIBUTED: False 2024-08-06T20:40:56.8890713Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-06T20:40:56.8891187Z SCCACHE_S3_KEY_PREFIX: pull 2024-08-06T20:40:56.8891546Z SHM_SIZE: 2g 2024-08-06T20:40:56.8892414Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:40:56.8893380Z XLA_CUDA: 2024-08-06T20:40:56.8893848Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-06T20:40:56.8894408Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-06T20:40:56.8894827Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-06T20:40:56.8895398Z DASHBOARD_TAG: 2024-08-06T20:40:56.8895701Z HUGGING_FACE_HUB_TOKEN: 2024-08-06T20:40:56.8896051Z ##[endgroup] 2024-08-06T20:40:56.8919136Z + [[ distributed == \m\u\l\t\i\g\p\u ]] 2024-08-06T20:40:56.8919605Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *onnx* ]] 2024-08-06T20:40:56.8920076Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-06T20:40:56.8928840Z +++ nproc --ignore=2 2024-08-06T20:40:56.8943873Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=30 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10273124344 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:41:08.0176786Z + container_name=9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:41:08.0178324Z + grep download.pytorch.org /etc/hosts 2024-08-06T20:41:08.0180613Z + docker exec -i 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-06T20:41:08.0993841Z + echo DOCKER_CONTAINER_ID=9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:41:08.0997872Z ++ echo dist/torch-2.5.0a0+gitb9d86fa-cp310-cp310-linux_x86_64.whl 2024-08-06T20:41:08.1000471Z + docker exec -t 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 sh -c 'pip install dist/torch-2.5.0a0+gitb9d86fa-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-06T20:41:08.6367220Z Processing ./dist/torch-2.5.0a0+gitb9d86fa-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+gitb9d86fa) 2024-08-06T20:41:09.7680327Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (3.13.1) 2024-08-06T20:41:09.7687769Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (4.12.2) 2024-08-06T20:41:09.7693270Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (2.8.8) 2024-08-06T20:41:09.7698999Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (3.1.4) 2024-08-06T20:41:09.7704780Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (2024.6.1) 2024-08-06T20:41:09.7721499Z Requirement already satisfied: sympy>=1.13.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (1.13.1) 2024-08-06T20:41:09.7770466Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (3.3.0) 2024-08-06T20:41:09.7852240Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (1.21.2) 2024-08-06T20:41:09.7899418Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.0->torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (1.3.0) 2024-08-06T20:41:09.9176491Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (2.1.5) 2024-08-06T20:41:11.0583211Z Installing collected packages: torch 2024-08-06T20:41:23.8269303Z Successfully installed torch-2.5.0a0+gitb9d86fa 2024-08-06T20:41:23.8950698Z ++ dirname .ci/pytorch/test.sh 2024-08-06T20:41:23.8957140Z + source .ci/pytorch/common.sh 2024-08-06T20:41:23.8961084Z +++ dirname .ci/pytorch/common.sh 2024-08-06T20:41:23.8968385Z ++ source .ci/pytorch/common_utils.sh 2024-08-06T20:41:23.8970468Z +++ declare -f -t trap_add 2024-08-06T20:41:23.8977397Z ++ set -ex 2024-08-06T20:41:23.8977770Z ++ [[ linux-focal-cuda11.8-py3.10-gcc9 == *rocm* ]] 2024-08-06T20:41:23.8978263Z ++ BUILD_TEST_LIBTORCH=0 2024-08-06T20:41:23.8979716Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *rocm* ]] 2024-08-06T20:41:23.8982429Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-06T20:41:23.8993488Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-06T20:41:23.8993891Z + trap_add cleanup_workspace EXIT 2024-08-06T20:41:23.8994296Z + trap_add_cmd=cleanup_workspace 2024-08-06T20:41:23.8994666Z + shift 2024-08-06T20:41:23.8994948Z + for trap_add_name in "$@" 2024-08-06T20:41:23.9002471Z +++ trap -p EXIT 2024-08-06T20:41:23.9005563Z ++ eval 'extract_trap_cmd ' 2024-08-06T20:41:23.9005914Z +++ extract_trap_cmd 2024-08-06T20:41:23.9006242Z +++ printf '%s\n' '' 2024-08-06T20:41:23.9006576Z ++ printf '%s\n' cleanup_workspace 2024-08-06T20:41:23.9008563Z + trap -- ' 2024-08-06T20:41:23.9008882Z cleanup_workspace' EXIT 2024-08-06T20:41:23.9009291Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-06T20:41:24.4102210Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-06T20:41:24.4120361Z + echo 'Environment variables:' 2024-08-06T20:41:24.4120798Z Environment variables: 2024-08-06T20:41:24.4121140Z + env 2024-08-06T20:41:24.4129381Z INSTALLED_DB=yes 2024-08-06T20:41:24.4129854Z NV_LIBCUBLAS_VERSION=11.11.3.6-1 2024-08-06T20:41:24.4130269Z NVIDIA_VISIBLE_DEVICES=all 2024-08-06T20:41:24.4130900Z NV_NVML_DEV_VERSION=11.8.86-1 2024-08-06T20:41:24.4131446Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:41:24.4132286Z CONTINUE_THROUGH_ERROR=False 2024-08-06T20:41:24.4132713Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.16.2-1+cuda11.8 2024-08-06T20:41:24.4133342Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.16.2-1 2024-08-06T20:41:24.4133987Z BUILD_ENVIRONMENT=linux-focal-cuda11.8-py3.10-gcc9 2024-08-06T20:41:24.4134441Z HOSTNAME=9268de398b04 2024-08-06T20:41:24.4135448Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_6d740b0c-6580-4d8e-ad3f-65b3431835be 2024-08-06T20:41:24.4136526Z GITHUB_ACTION=__self 2024-08-06T20:41:24.4136887Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-06T20:41:24.4139406Z NVIDIA_REQUIRE_CUDA=cuda>=11.8 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 2024-08-06T20:41:24.4142104Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-11-8=11.11.3.6-1 2024-08-06T20:41:24.4142590Z NV_NVTX_VERSION=11.8.86-1 2024-08-06T20:41:24.4143149Z GITHUB_RUN_NUMBER=233985 2024-08-06T20:41:24.4143575Z TEST_CONFIG=distributed 2024-08-06T20:41:24.4143921Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-06T20:41:24.4144586Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-06T20:41:24.4145051Z NV_CUDA_CUDART_DEV_VERSION=11.8.89-1 2024-08-06T20:41:24.4145444Z NV_LIBCUSPARSE_VERSION=11.7.5.86-1 2024-08-06T20:41:24.4146121Z NV_LIBNPP_VERSION=11.8.0.86-1 2024-08-06T20:41:24.4146810Z GITHUB_TRIGGERING_ACTOR=drisspg 2024-08-06T20:41:24.4147537Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-06T20:41:24.4148009Z GITHUB_REF_TYPE=branch 2024-08-06T20:41:24.4148449Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-06T20:41:24.4148988Z NCCL_VERSION=2.16.2-1 2024-08-06T20:41:24.4149380Z BASE_SHA=1736af7cf736184c356be1bb00f59fb2feea6d7d 2024-08-06T20:41:24.4149946Z XLA_CUDA= 2024-08-06T20:41:24.4150362Z HUGGING_FACE_HUB_TOKEN= 2024-08-06T20:41:24.4150897Z *** 2024-08-06T20:41:24.4151317Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-06T20:41:24.4151870Z GITHUB_REPOSITORY_ID=65600975 2024-08-06T20:41:24.4152247Z GITHUB_ACTIONS=true 2024-08-06T20:41:24.4152700Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:41:24.4153284Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-11-8=11.8.87-1 2024-08-06T20:41:24.4153793Z NV_LIBNPP_PACKAGE=libnpp-11-8=11.8.0.86-1 2024-08-06T20:41:24.4154532Z SHA1=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:41:24.4154973Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-06T20:41:24.4155892Z GITHUB_SHA=bf5bb5a1585a03379137fab341e87c02c77e76cd 2024-08-06T20:41:24.4156626Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/132710/merge 2024-08-06T20:41:24.4157772Z UCC_HOME=/usr 2024-08-06T20:41:24.4158134Z NV_LIBCUBLAS_DEV_VERSION=11.11.3.6-1 2024-08-06T20:41:24.4158791Z VERBOSE_TEST_LOGS=False 2024-08-06T20:41:24.4159132Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-06T20:41:24.4159614Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-11-8 2024-08-06T20:41:24.4160276Z GITHUB_REF=refs/pull/132710/merge 2024-08-06T20:41:24.4160658Z NV_CUDA_CUDART_VERSION=11.8.89-1 2024-08-06T20:41:24.4161298Z SHARD_NUMBER=3 2024-08-06T20:41:24.4161669Z GITHUB_REF_PROTECTED=false 2024-08-06T20:41:24.4162012Z HOME=/var/lib/jenkins 2024-08-06T20:41:24.4162538Z GITHUB_API_URL=https://api.github.com 2024-08-06T20:41:24.4163096Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-06T20:41:24.4163538Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-06T20:41:24.4164297Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-06T20:41:24.4164666Z CUDA_VERSION=11.8.0 2024-08-06T20:41:24.4165079Z NV_LIBCUBLAS_PACKAGE=libcublas-11-8=11.11.3.6-1 2024-08-06T20:41:24.4165743Z NUM_TEST_SHARDS=3 2024-08-06T20:41:24.4166040Z UCX_HOME=/usr 2024-08-06T20:41:24.4166651Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-11-8=11.8.0-1 2024-08-06T20:41:24.4167720Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_6d740b0c-6580-4d8e-ad3f-65b3431835be 2024-08-06T20:41:24.4169047Z JOB_NAME=linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:41:24.4170337Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_6d740b0c-6580-4d8e-ad3f-65b3431835be 2024-08-06T20:41:24.4171627Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-06T20:41:24.4172566Z GITHUB_EVENT_NAME=pull_request 2024-08-06T20:41:24.4172924Z DASHBOARD_TAG= 2024-08-06T20:41:24.4173360Z GITHUB_RUN_ID=10273124344 2024-08-06T20:41:24.4173934Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-11-8=11.8.0.86-1 2024-08-06T20:41:24.4174403Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-11-8 2024-08-06T20:41:24.4175535Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_6d740b0c-6580-4d8e-ad3f-65b3431835be 2024-08-06T20:41:24.4176674Z GITHUB_ACTOR=drisspg 2024-08-06T20:41:24.4177009Z NV_LIBNPP_DEV_VERSION=11.8.0.86-1 2024-08-06T20:41:24.4177529Z PR_NUMBER=132710 2024-08-06T20:41:24.4177972Z GITHUB_RUN_ATTEMPT=1 2024-08-06T20:41:24.4178304Z ANACONDA_PYTHON_VERSION=3.10 2024-08-06T20:41:24.4178873Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-06T20:41:24.4179471Z TERM=xterm 2024-08-06T20:41:24.4179776Z NV_LIBCUSPARSE_DEV_VERSION=11.7.5.86-1 2024-08-06T20:41:24.4180362Z INSTALLED_VISION=yes 2024-08-06T20:41:24.4180779Z BRANCH=pull/132710 2024-08-06T20:41:24.4181106Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-06T20:41:24.4181637Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-06T20:41:24.4182323Z CUDA_PATH=/usr/local/cuda 2024-08-06T20:41:24.4183240Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-06T20:41:24.4184000Z GITHUB_SERVER_URL=https://github.com 2024-08-06T20:41:24.4184722Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-06T20:41:24.4185189Z REENABLED_ISSUES= 2024-08-06T20:41:24.4185518Z SHLVL=1 2024-08-06T20:41:24.4186008Z MAX_JOBS=30 2024-08-06T20:41:24.4186329Z NV_CUDA_LIB_VERSION=11.8.0-1 2024-08-06T20:41:24.4186692Z NVARCH=x86_64 2024-08-06T20:41:24.4187113Z GITHUB_ACTOR_ID=32754868 2024-08-06T20:41:24.4187701Z GITHUB_WORKFLOW_SHA=bf5bb5a1585a03379137fab341e87c02c77e76cd 2024-08-06T20:41:24.4188210Z GITHUB_REF_NAME=132710/merge 2024-08-06T20:41:24.4188644Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-11-8 2024-08-06T20:41:24.4189447Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-06T20:41:24.4190011Z GITHUB_JOB=test 2024-08-06T20:41:24.4190347Z NV_LIBNCCL_PACKAGE=libnccl2=2.16.2-1+cuda11.8 2024-08-06T20:41:24.4191040Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-06T20:41:24.4191846Z NO_TEST_TIMEOUT=False 2024-08-06T20:41:24.4192172Z TD_DISTRIBUTED=False 2024-08-06T20:41:24.4192537Z NV_CUDA_NSIGHT_COMPUTE_VERSION=11.8.0-1 2024-08-06T20:41:24.4192991Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-06T20:41:24.4193377Z NV_NVPROF_VERSION=11.8.87-1 2024-08-06T20:41:24.4193750Z GITHUB_RETENTION_DAYS=90 2024-08-06T20:41:24.4194111Z OPENSSL_DIR=/opt/openssl 2024-08-06T20:41:24.4194458Z GITHUB_ACTION_REPOSITORY= 2024-08-06T20:41:24.4195710Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:41:24.4196777Z GITHUB_BASE_REF=gh/drisspg/23/base 2024-08-06T20:41:24.4197322Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-06T20:41:24.4197703Z CI=true 2024-08-06T20:41:24.4198012Z NV_LIBNCCL_PACKAGE_VERSION=2.16.2-1 2024-08-06T20:41:24.4198408Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-06T20:41:24.4198797Z JOB_ID=28428108140 2024-08-06T20:41:24.4199126Z INSTALLED_PROTOBUF=yes 2024-08-06T20:41:24.4199473Z GITHUB_HEAD_REF=gh/drisspg/23/head 2024-08-06T20:41:24.4199860Z GITHUB_ACTION_REF= 2024-08-06T20:41:24.4200248Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-06T20:41:24.4200708Z TEST_SHOWLOCALS=False 2024-08-06T20:41:24.4201046Z GITHUB_WORKFLOW=pull 2024-08-06T20:41:24.4201399Z DEBIAN_FRONTEND=noninteractive 2024-08-06T20:41:24.4202167Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_6d740b0c-6580-4d8e-ad3f-65b3431835be 2024-08-06T20:41:24.4203291Z NO_TD=False 2024-08-06T20:41:24.4203611Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-06T20:41:24.4203966Z _=/usr/bin/env 2024-08-06T20:41:24.4204385Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-06T20:41:24.4343065Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-06T20:41:24.4344080Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-06T20:41:24.4345133Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-06T20:41:24.4346004Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-06T20:41:24.4346735Z + BUILD_DIR=build 2024-08-06T20:41:24.4347072Z + BUILD_RENAMED_DIR=build_renamed 2024-08-06T20:41:24.4347728Z + BUILD_BIN_DIR=build/bin 2024-08-06T20:41:24.4348072Z + SHARD_NUMBER=3 2024-08-06T20:41:24.4348389Z + NUM_TEST_SHARDS=3 2024-08-06T20:41:24.4348866Z + export VALGRIND=ON 2024-08-06T20:41:24.4349299Z + VALGRIND=ON 2024-08-06T20:41:24.4349686Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *clang9* ]] 2024-08-06T20:41:24.4350318Z + [[ 0 == \1 ]] 2024-08-06T20:41:24.4350696Z + [[ False == \1 ]] 2024-08-06T20:41:24.4351082Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *bazel* ]] 2024-08-06T20:41:24.4351765Z ++ realpath build/custom_test_artifacts 2024-08-06T20:41:24.4359304Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-06T20:41:24.4360300Z + [[ -n '' ]] 2024-08-06T20:41:24.4360696Z + echo 'Environment variables' 2024-08-06T20:41:24.4361259Z Environment variables 2024-08-06T20:41:24.4361585Z + env 2024-08-06T20:41:24.4366571Z INSTALLED_DB=yes 2024-08-06T20:41:24.4366922Z NV_LIBCUBLAS_VERSION=11.11.3.6-1 2024-08-06T20:41:24.4367402Z NVIDIA_VISIBLE_DEVICES=all 2024-08-06T20:41:24.4367978Z NV_NVML_DEV_VERSION=11.8.86-1 2024-08-06T20:41:24.4368493Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:41:24.4369350Z CONTINUE_THROUGH_ERROR=False 2024-08-06T20:41:24.4369785Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.16.2-1+cuda11.8 2024-08-06T20:41:24.4370448Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.16.2-1 2024-08-06T20:41:24.4371002Z BUILD_ENVIRONMENT=linux-focal-cuda11.8-py3.10-gcc9 2024-08-06T20:41:24.4371502Z HOSTNAME=9268de398b04 2024-08-06T20:41:24.4372441Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_6d740b0c-6580-4d8e-ad3f-65b3431835be 2024-08-06T20:41:24.4373685Z GITHUB_ACTION=__self 2024-08-06T20:41:24.4374064Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-06T20:41:24.4376558Z NVIDIA_REQUIRE_CUDA=cuda>=11.8 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 2024-08-06T20:41:24.4379144Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-11-8=11.11.3.6-1 2024-08-06T20:41:24.4379621Z NV_NVTX_VERSION=11.8.86-1 2024-08-06T20:41:24.4380242Z GITHUB_RUN_NUMBER=233985 2024-08-06T20:41:24.4380587Z TEST_CONFIG=distributed 2024-08-06T20:41:24.4380952Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-06T20:41:24.4381648Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-06T20:41:24.4382069Z NV_CUDA_CUDART_DEV_VERSION=11.8.89-1 2024-08-06T20:41:24.4382592Z NV_LIBCUSPARSE_VERSION=11.7.5.86-1 2024-08-06T20:41:24.4383178Z NV_LIBNPP_VERSION=11.8.0.86-1 2024-08-06T20:41:24.4383549Z GITHUB_TRIGGERING_ACTOR=drisspg 2024-08-06T20:41:24.4384185Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-06T20:41:24.4384717Z GITHUB_REF_TYPE=branch 2024-08-06T20:41:24.4385052Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-06T20:41:24.4385666Z NCCL_VERSION=2.16.2-1 2024-08-06T20:41:24.4386075Z BASE_SHA=1736af7cf736184c356be1bb00f59fb2feea6d7d 2024-08-06T20:41:24.4386497Z XLA_CUDA= 2024-08-06T20:41:24.4387038Z HUGGING_FACE_HUB_TOKEN= 2024-08-06T20:41:24.4387482Z *** 2024-08-06T20:41:24.4387765Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-06T20:41:24.4388403Z GITHUB_REPOSITORY_ID=65600975 2024-08-06T20:41:24.4388806Z GITHUB_ACTIONS=true 2024-08-06T20:41:24.4389186Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:41:24.4389860Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-11-8=11.8.87-1 2024-08-06T20:41:24.4390346Z NV_LIBNPP_PACKAGE=libnpp-11-8=11.8.0.86-1 2024-08-06T20:41:24.4390944Z SHA1=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:41:24.4391538Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-06T20:41:24.4392011Z GITHUB_SHA=bf5bb5a1585a03379137fab341e87c02c77e76cd 2024-08-06T20:41:24.4392948Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/132710/merge 2024-08-06T20:41:24.4393717Z UCC_HOME=/usr 2024-08-06T20:41:24.4394151Z NV_LIBCUBLAS_DEV_VERSION=11.11.3.6-1 2024-08-06T20:41:24.4394545Z VERBOSE_TEST_LOGS=False 2024-08-06T20:41:24.4395019Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-06T20:41:24.4395577Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-11-8 2024-08-06T20:41:24.4396039Z GITHUB_REF=refs/pull/132710/merge 2024-08-06T20:41:24.4396730Z NV_CUDA_CUDART_VERSION=11.8.89-1 2024-08-06T20:41:24.4397351Z SHARD_NUMBER=3 2024-08-06T20:41:24.4397778Z GITHUB_REF_PROTECTED=false 2024-08-06T20:41:24.4398304Z HOME=/var/lib/jenkins 2024-08-06T20:41:24.4398679Z GITHUB_API_URL=https://api.github.com 2024-08-06T20:41:24.4399236Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-06T20:41:24.4399970Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-06T20:41:24.4400526Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-06T20:41:24.4401058Z CUDA_VERSION=11.8.0 2024-08-06T20:41:24.4401431Z NV_LIBCUBLAS_PACKAGE=libcublas-11-8=11.11.3.6-1 2024-08-06T20:41:24.4401984Z NUM_TEST_SHARDS=3 2024-08-06T20:41:24.4402427Z UCX_HOME=/usr 2024-08-06T20:41:24.4402879Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-11-8=11.8.0-1 2024-08-06T20:41:24.4404099Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_6d740b0c-6580-4d8e-ad3f-65b3431835be 2024-08-06T20:41:24.4405424Z JOB_NAME=linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-08-06T20:41:24.4406714Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_6d740b0c-6580-4d8e-ad3f-65b3431835be 2024-08-06T20:41:24.4407982Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-06T20:41:24.4408639Z GITHUB_EVENT_NAME=pull_request 2024-08-06T20:41:24.4409399Z DASHBOARD_TAG= 2024-08-06T20:41:24.4409742Z GITHUB_RUN_ID=10273124344 2024-08-06T20:41:24.4410282Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-11-8=11.8.0.86-1 2024-08-06T20:41:24.4410908Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-11-8 2024-08-06T20:41:24.4412064Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_6d740b0c-6580-4d8e-ad3f-65b3431835be 2024-08-06T20:41:24.4412895Z GITHUB_ACTOR=drisspg 2024-08-06T20:41:24.4413427Z NV_LIBNPP_DEV_VERSION=11.8.0.86-1 2024-08-06T20:41:24.4413896Z PR_NUMBER=132710 2024-08-06T20:41:24.4414198Z GITHUB_RUN_ATTEMPT=1 2024-08-06T20:41:24.4414524Z VALGRIND=ON 2024-08-06T20:41:24.4414840Z ANACONDA_PYTHON_VERSION=3.10 2024-08-06T20:41:24.4415266Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-06T20:41:24.4415911Z TERM=xterm 2024-08-06T20:41:24.4416318Z NV_LIBCUSPARSE_DEV_VERSION=11.7.5.86-1 2024-08-06T20:41:24.4416710Z INSTALLED_VISION=yes 2024-08-06T20:41:24.4417046Z BRANCH=pull/132710 2024-08-06T20:41:24.4417402Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-06T20:41:24.4417791Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-06T20:41:24.4418216Z CUDA_PATH=/usr/local/cuda 2024-08-06T20:41:24.4418882Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-06T20:41:24.4419600Z GITHUB_SERVER_URL=https://github.com 2024-08-06T20:41:24.4420353Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-06T20:41:24.4420823Z REENABLED_ISSUES= 2024-08-06T20:41:24.4421119Z SHLVL=1 2024-08-06T20:41:24.4421399Z MAX_JOBS=30 2024-08-06T20:41:24.4421711Z NV_CUDA_LIB_VERSION=11.8.0-1 2024-08-06T20:41:24.4422055Z NVARCH=x86_64 2024-08-06T20:41:24.4422368Z GITHUB_ACTOR_ID=32754868 2024-08-06T20:41:24.4422804Z GITHUB_WORKFLOW_SHA=bf5bb5a1585a03379137fab341e87c02c77e76cd 2024-08-06T20:41:24.4423316Z GITHUB_REF_NAME=132710/merge 2024-08-06T20:41:24.4423710Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-11-8 2024-08-06T20:41:24.4424283Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-06T20:41:24.4424849Z GITHUB_JOB=test 2024-08-06T20:41:24.4425203Z NV_LIBNCCL_PACKAGE=libnccl2=2.16.2-1+cuda11.8 2024-08-06T20:41:24.4425720Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-06T20:41:24.4426222Z NO_TEST_TIMEOUT=False 2024-08-06T20:41:24.4426561Z TD_DISTRIBUTED=False 2024-08-06T20:41:24.4426907Z NV_CUDA_NSIGHT_COMPUTE_VERSION=11.8.0-1 2024-08-06T20:41:24.4427619Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-06T20:41:24.4428022Z NV_NVPROF_VERSION=11.8.87-1 2024-08-06T20:41:24.4428370Z GITHUB_RETENTION_DAYS=90 2024-08-06T20:41:24.4428725Z OPENSSL_DIR=/opt/openssl 2024-08-06T20:41:24.4429085Z GITHUB_ACTION_REPOSITORY= 2024-08-06T20:41:24.4430020Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:41:24.4431024Z GITHUB_BASE_REF=gh/drisspg/23/base 2024-08-06T20:41:24.4431435Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-06T20:41:24.4431932Z CI=true 2024-08-06T20:41:24.4432241Z NV_LIBNCCL_PACKAGE_VERSION=2.16.2-1 2024-08-06T20:41:24.4432658Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-06T20:41:24.4433009Z JOB_ID=28428108140 2024-08-06T20:41:24.4433330Z INSTALLED_PROTOBUF=yes 2024-08-06T20:41:24.4433687Z GITHUB_HEAD_REF=gh/drisspg/23/head 2024-08-06T20:41:24.4434060Z GITHUB_ACTION_REF= 2024-08-06T20:41:24.4434449Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-06T20:41:24.4434911Z TEST_SHOWLOCALS=False 2024-08-06T20:41:24.4435234Z GITHUB_WORKFLOW=pull 2024-08-06T20:41:24.4435581Z DEBIAN_FRONTEND=noninteractive 2024-08-06T20:41:24.4436357Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_6d740b0c-6580-4d8e-ad3f-65b3431835be 2024-08-06T20:41:24.4437458Z NO_TD=False 2024-08-06T20:41:24.4437791Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-06T20:41:24.4438165Z _=/usr/bin/env 2024-08-06T20:41:24.4438465Z + echo 'Testing pytorch' 2024-08-06T20:41:24.4438820Z Testing pytorch 2024-08-06T20:41:24.4439163Z + export LANG=C.UTF-8 2024-08-06T20:41:24.4439612Z + LANG=C.UTF-8 2024-08-06T20:41:24.4439944Z + PR_NUMBER=132710 2024-08-06T20:41:24.4440271Z + [[ distributed == \d\e\f\a\u\l\t ]] 2024-08-06T20:41:24.4440767Z + [[ distributed == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-06T20:41:24.4441489Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *rocm* ]] 2024-08-06T20:41:24.4441937Z + [[ distributed == \s\l\o\w ]] 2024-08-06T20:41:24.4442419Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *slow-gradcheck* ]] 2024-08-06T20:41:24.4442968Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *cuda* ]] 2024-08-06T20:41:24.4443440Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-06T20:41:24.4443899Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-06T20:41:24.4444325Z + [[ distributed == *crossref* ]] 2024-08-06T20:41:24.4444745Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *rocm* ]] 2024-08-06T20:41:24.4445246Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *xpu* ]] 2024-08-06T20:41:24.4445967Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *-bazel-* ]] 2024-08-06T20:41:24.4446468Z + pip_install --user ninja==1.10.2 2024-08-06T20:41:24.4446935Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-06T20:41:24.9539897Z Collecting ninja==1.10.2 2024-08-06T20:41:24.9725540Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-06T20:41:24.9890635Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-06T20:41:26.1748151Z Installing collected packages: ninja 2024-08-06T20:41:26.1842992Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-06T20:41:26.1844162Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-06T20:41:26.1892136Z Successfully installed ninja-1.10.2 2024-08-06T20:41:26.2571473Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:41:26.2573416Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:41:26.2574610Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *aarch64* ]] 2024-08-06T20:41:26.2575060Z + install_tlparse 2024-08-06T20:41:26.2575408Z + pip_install --user tlparse==0.3.7 2024-08-06T20:41:26.2575871Z + pip install --progress-bar off --user tlparse==0.3.7 2024-08-06T20:41:26.7730443Z Collecting tlparse==0.3.7 2024-08-06T20:41:26.8191410Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-08-06T20:41:26.8315098Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-06T20:41:28.0587260Z Installing collected packages: tlparse 2024-08-06T20:41:28.1079187Z Successfully installed tlparse-0.3.7 2024-08-06T20:41:28.1753850Z ++ python -m site --user-base 2024-08-06T20:41:28.1976430Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:41:28.1977879Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *asan* ]] 2024-08-06T20:41:28.1978412Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *-debug* ]] 2024-08-06T20:41:28.1978925Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *-bazel-* ]] 2024-08-06T20:41:28.1979643Z + echo 'We are not in debug mode: linux-focal-cuda11.8-py3.10-gcc9. Expect the assertion to pass' 2024-08-06T20:41:28.1980505Z We are not in debug mode: linux-focal-cuda11.8-py3.10-gcc9. Expect the assertion to pass 2024-08-06T20:41:28.1981753Z + cd test 2024-08-06T20:41:28.1982196Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-06T20:41:30.3081076Z + [[ distributed == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-06T20:41:30.3081964Z + [[ distributed == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-06T20:41:30.3087196Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-06T20:41:30.3087610Z + [[ distributed == *dynamo_eager* ]] 2024-08-06T20:41:30.3088017Z + [[ distributed == *aot_eager* ]] 2024-08-06T20:41:30.3088430Z + [[ distributed == *aot_inductor* ]] 2024-08-06T20:41:30.3088845Z + [[ distributed == *inductor* ]] 2024-08-06T20:41:30.3089235Z + [[ distributed == *dynamic* ]] 2024-08-06T20:41:30.3089635Z + [[ distributed == *cpu* ]] 2024-08-06T20:41:30.3090046Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-06T20:41:30.3130066Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *libtorch* ]] 2024-08-06T20:41:30.3130618Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *-bazel-* ]] 2024-08-06T20:41:30.3133937Z + cd test 2024-08-06T20:41:30.3135029Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-06T20:41:32.2161267Z PyTorch built with: 2024-08-06T20:41:32.2161651Z - GCC 9.4 2024-08-06T20:41:32.2161936Z - C++ Version: 201703 2024-08-06T20:41:32.2162712Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-06T20:41:32.2163612Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-06T20:41:32.2164161Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-06T20:41:32.2164607Z - LAPACK is enabled (usually provided by MKL) 2024-08-06T20:41:32.2165046Z - NNPACK is enabled 2024-08-06T20:41:32.2165376Z - CPU capability usage: AVX2 2024-08-06T20:41:32.2165748Z - CUDA Runtime 11.8 2024-08-06T20:41:32.2166201Z - NVCC architecture flags: -gencode;arch=compute_52,code=sm_52 2024-08-06T20:41:32.2166685Z - CuDNN 90.1 2024-08-06T20:41:32.2166985Z - Magma 2.6.1 2024-08-06T20:41:32.2172817Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-06T20:41:32.2179142Z 2024-08-06T20:41:32.4721388Z + cd test 2024-08-06T20:41:32.4722206Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-06T20:41:34.3212803Z ATen/Parallel: 2024-08-06T20:41:34.3213207Z at::get_num_threads() : 16 2024-08-06T20:41:34.3213622Z at::get_num_interop_threads() : 16 2024-08-06T20:41:34.3214043Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-06T20:41:34.3214420Z omp_get_max_threads() : 16 2024-08-06T20:41:34.3215141Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-06T20:41:34.3215892Z mkl_get_max_threads() : 16 2024-08-06T20:41:34.3216382Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-06T20:41:34.3216963Z std::thread::hardware_concurrency() : 32 2024-08-06T20:41:34.3217391Z Environment variables: 2024-08-06T20:41:34.3217730Z OMP_NUM_THREADS : [not set] 2024-08-06T20:41:34.3218104Z MKL_NUM_THREADS : [not set] 2024-08-06T20:41:34.3218486Z ATen parallel backend: OpenMP 2024-08-06T20:41:34.3219115Z 2024-08-06T20:41:34.5578108Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *aarch64* ]] 2024-08-06T20:41:34.5578671Z + [[ distributed == *backward* ]] 2024-08-06T20:41:34.5579064Z + [[ distributed == *xla* ]] 2024-08-06T20:41:34.5579454Z + [[ distributed == *executorch* ]] 2024-08-06T20:41:34.5579879Z + [[ distributed == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-06T20:41:34.5580353Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *libtorch* ]] 2024-08-06T20:41:34.5580852Z + [[ distributed == distributed ]] 2024-08-06T20:41:34.5581354Z + test_distributed 2024-08-06T20:41:34.5581695Z + echo 'Testing distributed python tests' 2024-08-06T20:41:34.5582146Z Testing distributed python tests 2024-08-06T20:41:34.5582683Z + python test/run_test.py --distributed-tests --shard 3 3 --verbose 2024-08-06T20:41:34.6849637Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-06T20:41:34.6850712Z import pkg_resources 2024-08-06T20:41:39.0825576Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-06T20:41:39.1430807Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-06T20:41:39.1878007Z Ignoring disabled issues: [''] 2024-08-06T20:41:39.2017875Z Found test times from artifacts 2024-08-06T20:41:39.2545364Z Found test times from artifacts 2024-08-06T20:41:39.2562951Z Running 25% of tests based on TD 2024-08-06T20:41:39.2689342Z Running parallel tests on 2 processes 2024-08-06T20:41:39.2691485Z Name: tests to run (est. time: 5.59min) 2024-08-06T20:41:39.2691900Z Serial tests (23): 2024-08-06T20:41:39.2692298Z distributed/_tensor/debug/test_op_coverage 1/1 2024-08-06T20:41:39.2692797Z distributed/pipelining/test_backward 1/1 2024-08-06T20:41:39.2693279Z distributed/_shard/sharded_tensor/test_logger 1/1 2024-08-06T20:41:39.2693803Z distributed/test_symmetric_memory 1/1 2024-08-06T20:41:39.2694271Z distributed/_tensor/test_dtensor_compile 1/1 2024-08-06T20:41:39.2694725Z distributed/pipelining/test_pipe 1/1 2024-08-06T20:41:39.2695173Z distributed/_tools/test_mem_tracker 1/1 2024-08-06T20:41:39.2695657Z distributed/_tensor/test_experimental_ops 1/1 2024-08-06T20:41:39.2696197Z distributed/_tensor/test_common_rules 1/1 2024-08-06T20:41:39.2696670Z distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 2024-08-06T20:41:39.2697164Z distributed/_tensor/test_xla_integration 1/1 2024-08-06T20:41:39.2697698Z distributed/_tensor/experimental/test_tp_transform 1/1 2024-08-06T20:41:39.2698220Z distributed/checkpoint/test_dedup_tensors 1/1 2024-08-06T20:41:39.2698706Z distributed/pipelining/test_unflatten 1/1 2024-08-06T20:41:39.2699187Z distributed/_tensor/test_convolution_ops 1/1 2024-08-06T20:41:39.2699644Z distributed/_tensor/test_tensor_ops 1/1 2024-08-06T20:41:39.2700359Z distributed/optim/test_named_optimizer 1/1 2024-08-06T20:41:39.2700842Z distributed/_composable/test_checkpoint 1/1 2024-08-06T20:41:39.2701315Z distributed/_tools/test_fsdp2_mem_tracker 1/1 2024-08-06T20:41:39.2701798Z distributed/_tensor/test_pointwise_ops 1/1 2024-08-06T20:41:39.2702284Z distributed/checkpoint/test_save_load_api 1/1 2024-08-06T20:41:39.2702768Z distributed/checkpoint/test_tp_checkpoint 1/1 2024-08-06T20:41:39.2703289Z distributed/_tensor/experimental/test_local_map 1/1 2024-08-06T20:41:39.2703768Z Parallel tests (0): 2024-08-06T20:41:39.2704152Z Name: excluded (est. time: 81.36min) 2024-08-06T20:41:39.2704531Z Serial tests (54): 2024-08-06T20:41:39.2704901Z distributed/test_distributed_spawn 3/10 2024-08-06T20:41:39.2705361Z distributed/test_distributed_spawn 6/10 2024-08-06T20:41:39.2705800Z distributed/test_distributed_spawn 9/10 2024-08-06T20:41:39.2706240Z distributed/test_c10d_nccl 1/2 2024-08-06T20:41:39.2706645Z distributed/test_c10d_nccl 2/2 2024-08-06T20:41:39.2707205Z distributed/fsdp/test_fsdp_state_dict 1/2 2024-08-06T20:41:39.2707713Z distributed/rpc/cuda/test_tensorpipe_agent 2/2 2024-08-06T20:41:39.2708273Z distributed/_composable/fsdp/test_fully_shard_training 1/1 2024-08-06T20:41:39.2708768Z distributed/test_c10d_ucc 1/1 2024-08-06T20:41:39.2709193Z distributed/checkpoint/test_state_dict 1/1 2024-08-06T20:41:39.2709636Z distributed/fsdp/test_wrap 1/1 2024-08-06T20:41:39.2710107Z distributed/_composable/test_replicate_with_compiler 1/1 2024-08-06T20:41:39.2710653Z distributed/fsdp/test_fsdp_unshard_params 1/1 2024-08-06T20:41:39.2711133Z distributed/test_dynamo_distributed 1/1 2024-08-06T20:41:39.2711561Z distributed/test_functional_api 1/1 2024-08-06T20:41:39.2712096Z distributed/algorithms/quantization/test_quantization 1/1 2024-08-06T20:41:39.2712648Z distributed/fsdp/test_fsdp_hybrid_shard 1/1 2024-08-06T20:41:39.2713115Z distributed/test_c10d_object_collectives 1/1 2024-08-06T20:41:39.2713609Z distributed/_composable/test_compose 1/1 2024-08-06T20:41:39.2714058Z distributed/algorithms/test_join 1/1 2024-08-06T20:41:39.2714523Z distributed/elastic/multiprocessing/api_test 1/1 2024-08-06T20:41:39.2715010Z distributed/test_c10d_spawn_gloo 1/1 2024-08-06T20:41:39.2715461Z distributed/checkpoint/test_checkpoint 1/1 2024-08-06T20:41:39.2715944Z distributed/fsdp/test_hsdp_dtensor_state_dict 1/1 2024-08-06T20:41:39.2716525Z distributed/_composable/fully_shard/test_fully_shard_init 1/1 2024-08-06T20:41:39.2717058Z distributed/test_c10d_spawn_ucc 1/1 2024-08-06T20:41:39.2717917Z distributed/_composable/fully_shard/test_fully_shard_model_checkpoint 1/1 2024-08-06T20:41:39.2718560Z distributed/checkpoint/test_state_dict_utils 1/1 2024-08-06T20:41:39.2719116Z distributed/_shard/sharded_tensor/ops/test_binary_cmp 1/1 2024-08-06T20:41:39.2719657Z distributed/checkpoint/test_hsdp_checkpoint 1/1 2024-08-06T20:41:39.2720173Z distributed/fsdp/test_fsdp_tp_integration 1/1 2024-08-06T20:41:39.2720761Z distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 2024-08-06T20:41:39.2721397Z distributed/_composable/fsdp/test_fully_shard_extensions 1/1 2024-08-06T20:41:39.2722016Z distributed/_composable/fsdp/test_fully_shard_overlap 1/1 2024-08-06T20:41:39.2722549Z distributed/checkpoint/test_fsspec 1/1 2024-08-06T20:41:39.2722978Z distributed/test_c10d_ops_nccl 1/1 2024-08-06T20:41:39.2723487Z distributed/_shard/sharded_tensor/ops/test_embedding 1/1 2024-08-06T20:41:39.2724118Z distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 2024-08-06T20:41:39.2724706Z distributed/fsdp/test_fsdp_backward_prefetch 1/1 2024-08-06T20:41:39.2725276Z distributed/_composable/fsdp/test_fully_shard_memory 1/1 2024-08-06T20:41:39.2725819Z distributed/elastic/timer/local_timer_test 1/1 2024-08-06T20:41:39.2726316Z distributed/fsdp/test_fsdp_multiple_forward 1/1 2024-08-06T20:41:39.2726811Z distributed/fsdp/test_fsdp_traversal 1/1 2024-08-06T20:41:39.2727453Z distributed/checkpoint/e2e/test_fine_tuning 1/1 2024-08-06T20:41:39.2727973Z distributed/tensor/parallel/test_ddp_2d_parallel 1/1 2024-08-06T20:41:39.2728475Z distributed/test_data_parallel 1/1 2024-08-06T20:41:39.2728921Z distributed/checkpoint/test_planner 1/1 2024-08-06T20:41:39.2729366Z distributed/elastic/utils/util_test 1/1 2024-08-06T20:41:39.2729820Z distributed/elastic/events/lib_test 1/1 2024-08-06T20:41:39.2730272Z distributed/checkpoint/test_utils 1/1 2024-08-06T20:41:39.2730771Z distributed/_composable/fsdp/test_fully_shard_state 1/1 2024-08-06T20:41:39.2731396Z distributed/_composable/fully_shard/test_fully_shard_compile 1/1 2024-08-06T20:41:39.2732024Z distributed/optim/test_apply_optimizer_in_backward 1/1 2024-08-06T20:41:39.2732534Z distributed/elastic/timer/api_test 1/1 2024-08-06T20:41:39.2732956Z Parallel tests (0): 2024-08-06T20:41:39.2733442Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-06T20:41:39.2762772Z Running distributed/_tensor/debug/test_op_coverage 1/1 ... [2024-08-06 20:41:39.275937] 2024-08-06T20:41:39.2767237Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/debug/test_op_coverage.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:41:39.276360] 2024-08-06T20:41:43.2995920Z 2024-08-06T20:41:43.2997977Z distributed/_tensor/debug/test_op_coverage 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.debug.test_op_coverage_1.1_4192fbd92cf7321e_.log 2024-08-06T20:41:43.2999636Z Running 1 items in this shard: test/distributed/_tensor/debug/test_op_coverage.py::TestOpCoverage::test_trace_with_inductor_decomp 2024-08-06T20:41:43.3000327Z 2024-08-06T20:41:43.3004436Z Running distributed/pipelining/test_backward 1/1 ... [2024-08-06 20:41:43.300107] 2024-08-06T20:41:43.3008664Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/pipelining/test_backward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:41:43.300505] 2024-08-06T20:41:47.6735715Z 2024-08-06T20:41:47.6736957Z distributed/pipelining/test_backward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_backward_1.1_1c5595844dd514c8_.log 2024-08-06T20:41:47.6738491Z Running 1 items in this shard: test/distributed/pipelining/test_backward.py::StageBackwardTests::test_stage_backward 2024-08-06T20:41:47.6739135Z 2024-08-06T20:41:47.6741851Z Running distributed/_shard/sharded_tensor/test_logger 1/1 ... [2024-08-06 20:41:47.673845] 2024-08-06T20:41:47.6746221Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_logger.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:41:47.674240] 2024-08-06T20:41:51.7969284Z 2024-08-06T20:41:51.7970586Z distributed/_shard/sharded_tensor/test_logger 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_logger_1.1_491fee39dd251ac0_.log 2024-08-06T20:41:51.7972267Z Running 1 items in this shard: test/distributed/_shard/sharded_tensor/test_logger.py::ShardingSpecLoggerTest::test_get_or_create_logger 2024-08-06T20:41:51.7973006Z 2024-08-06T20:41:51.7975113Z Running distributed/test_symmetric_memory 1/1 ... [2024-08-06 20:41:51.797193] 2024-08-06T20:41:52.1256075Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/test_symmetric_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:41:52.125085] 2024-08-06T20:41:55.5966898Z 2024-08-06T20:41:55.5968003Z distributed/test_symmetric_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_symmetric_memory_1.1_6b6e00a53d54af57_.log 2024-08-06T20:41:55.5979942Z Running 20 items in this shard: test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_cuda_nvlink_connectivity_detection, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_empty_strided_p2p, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_empty_strided_p2p_persistent, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_matmul_gather_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_matmul_gather_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_scaled_matmul_gather_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_scaled_matmul_gather_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_matmul_reduce_scatter_scatter_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_matmul_reduce_scatter_scatter_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_scaled_matmul_reduce_scatter_scatter_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_scaled_matmul_reduce_scatter_scatter_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_all_gather_symm_mem_input_False, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_all_gather_symm_mem_input_True, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_avg_symm_mem_input_False, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_avg_symm_mem_input_True, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_sum_symm_mem_input_False, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_sum_symm_mem_input_True, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_optimal_layout_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_optimal_layout_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_optimal_layout_dim_2 2024-08-06T20:41:55.5991041Z 2024-08-06T20:41:55.5991408Z Running distributed/_tensor/test_dtensor_compile 1/1 ... [2024-08-06 20:41:55.596971] 2024-08-06T20:41:55.5992955Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_dtensor_compile.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:41:55.597363] 2024-08-06T20:42:27.1334291Z 2024-08-06T20:42:27.1335583Z distributed/_tensor/test_dtensor_compile 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_dtensor_compile_1.1_71e65084c74df593_.log 2024-08-06T20:42:27.1350099Z Running 25 items in this shard: test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_device_mesh_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_attribute_access_on_intermediate, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_constructor_w_dynamo_disable, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_constructor_w_graph_break, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_contiguous_dtensor_noncontiguous_local_as_tangent, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_different_gradient_placement, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_dynamo_device_mesh_attrs, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_noncontiguous_output, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_partial_placement_graph_output, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_partial_placement_redistribute_unbalanced_correct_strides, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_from_local, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_from_local_redistribute, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_recompile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_to_local_kwargs, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_to_local_kwargs_forward_hook, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_fakify_dtensor, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_graph_input_is_async, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_placement_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_tp_compile_comm_reordering, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_2d_fsdp_tp_ac_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_2d_fsdp_tp_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_compile_dtensor_redistribute_backward, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_tp_compile_fullgraph_is_seq_parallel_False, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_tp_compile_fullgraph_is_seq_parallel_True 2024-08-06T20:42:27.1368393Z 2024-08-06T20:42:27.1368754Z Running distributed/pipelining/test_pipe 1/1 ... [2024-08-06 20:42:27.133683] 2024-08-06T20:42:27.1370289Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/pipelining/test_pipe.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:42:27.134109] 2024-08-06T20:42:32.2585108Z 2024-08-06T20:42:32.2586344Z distributed/pipelining/test_pipe 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_pipe_1.1_eda731756ed21888_.log 2024-08-06T20:42:32.2588655Z Running 3 items in this shard: test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass0, test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass1, test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass2 2024-08-06T20:42:32.2590286Z 2024-08-06T20:42:32.2590710Z Running distributed/_tools/test_mem_tracker 1/1 ... [2024-08-06 20:42:32.258789] 2024-08-06T20:42:32.2595194Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tools/test_mem_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:42:32.259197] 2024-08-06T20:42:37.6337407Z 2024-08-06T20:42:37.6338578Z distributed/_tools/test_mem_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_mem_tracker_1.1_d6424aef5ada9f3a_.log 2024-08-06T20:42:37.6340980Z Running 3 items in this shard: test/distributed/_tools/test_mem_tracker.py::TestMemTracker::test_cuda_tracker_equivalence, test/distributed/_tools/test_mem_tracker.py::TestMemTracker::test_tracker_attribution, test/distributed/_tools/test_mem_tracker.py::TestMemTracker::test_tracker_with_activation_checkpointing 2024-08-06T20:42:37.6343043Z 2024-08-06T20:42:37.6343697Z Running distributed/_tensor/test_experimental_ops 1/1 ... [2024-08-06 20:42:37.634023] 2024-08-06T20:42:37.6347946Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_experimental_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:42:37.634428] 2024-08-06T20:42:54.7855156Z 2024-08-06T20:42:54.7856656Z distributed/_tensor/test_experimental_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_experimental_ops_1.1_54cb6d9c18d9ebbb_.log 2024-08-06T20:42:54.7859007Z Running 3 items in this shard: test/distributed/_tensor/test_experimental_ops.py::DistOtherOpsTest::test_bernoulli, test/distributed/_tensor/test_experimental_ops.py::DistOtherOpsTest::test_nll, test/distributed/_tensor/test_experimental_ops.py::DistOtherOpsTest::test_slice 2024-08-06T20:42:54.7861437Z 2024-08-06T20:42:54.7862098Z Running distributed/_tensor/test_common_rules 1/1 ... [2024-08-06 20:42:54.785772] 2024-08-06T20:42:54.7865416Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_common_rules.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:42:54.786169] 2024-08-06T20:43:36.0925728Z 2024-08-06T20:43:36.0926955Z distributed/_tensor/test_common_rules 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_common_rules_1.1_b3b1cf05ed7c4f93_.log 2024-08-06T20:43:36.0932685Z Running 10 items in this shard: test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_basic_propagation, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_errors, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_linearity, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_merge_sharding, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_pointwise_propagation, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_enforce_sharding_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_rules_broadcasting, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_rules_suggestion 2024-08-06T20:43:36.0937549Z 2024-08-06T20:43:36.0937937Z Running distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 ... [2024-08-06 20:43:36.092983] 2024-08-06T20:43:36.0939562Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/checkpoint/fsdp/test_fsdp_dsd.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:43:36.093459] 2024-08-06T20:43:54.7462016Z 2024-08-06T20:43:54.7463654Z distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.fsdp.test_fsdp_dsd_1.1_e5cc9a0b80d84b8f_.log 2024-08-06T20:43:54.7466867Z Running 3 items in this shard: test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_1d_fsdp_cpu_offload_full_model_state_dict, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_1d_fsdp_get_model_state_dict, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_save_with_fsdp1_and_load_with_fsdp2 2024-08-06T20:43:54.7469202Z 2024-08-06T20:43:54.7469563Z Running distributed/_tensor/test_xla_integration 1/1 ... [2024-08-06 20:43:54.746594] 2024-08-06T20:43:54.7474577Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_xla_integration.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:43:54.747052] 2024-08-06T20:43:58.5190261Z 2024-08-06T20:43:58.5191670Z distributed/_tensor/test_xla_integration 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_xla_integration_1.1_d012f1d3ce1f60c0_.log 2024-08-06T20:43:58.5194453Z Running 3 items in this shard: test/distributed/_tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_1d_replicate, test/distributed/_tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_1d_shard, test/distributed/_tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_2d 2024-08-06T20:43:58.5196927Z 2024-08-06T20:43:58.5197617Z Running distributed/_tensor/experimental/test_tp_transform 1/1 ... [2024-08-06 20:43:58.519420] 2024-08-06T20:43:58.5203199Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/experimental/test_tp_transform.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:43:58.519877] 2024-08-06T20:44:18.6242220Z 2024-08-06T20:44:18.6244637Z distributed/_tensor/experimental/test_tp_transform 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_tp_transform_1.1_e3559fa9c020dd98_.log 2024-08-06T20:44:18.6250556Z Running 3 items in this shard: test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_e2e, test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_no_bias, test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_with_uncovered_op 2024-08-06T20:44:18.6254386Z 2024-08-06T20:44:18.6255162Z Running distributed/checkpoint/test_dedup_tensors 1/1 ... [2024-08-06 20:44:18.624469] 2024-08-06T20:44:18.6257874Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/checkpoint/test_dedup_tensors.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:44:18.624867] 2024-08-06T20:44:22.3465328Z 2024-08-06T20:44:22.3466898Z distributed/checkpoint/test_dedup_tensors 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_dedup_tensors_1.1_1d38d54779eecec0_.log 2024-08-06T20:44:22.3468490Z Running 1 items in this shard: test/distributed/checkpoint/test_dedup_tensors.py::TestDedupTensor::test_dedup_shards 2024-08-06T20:44:22.3469135Z 2024-08-06T20:44:22.3471831Z Running distributed/pipelining/test_unflatten 1/1 ... [2024-08-06 20:44:22.346822] 2024-08-06T20:44:22.3476064Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/pipelining/test_unflatten.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:44:22.347216] 2024-08-06T20:44:27.1709558Z 2024-08-06T20:44:27.1710845Z distributed/pipelining/test_unflatten 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_unflatten_1.1_2aac9a5505d75ba4_.log 2024-08-06T20:44:27.1712349Z Running 1 items in this shard: test/distributed/pipelining/test_unflatten.py::UnflattenTests::test_unflatten 2024-08-06T20:44:27.1712976Z 2024-08-06T20:44:27.1715420Z Running distributed/_tensor/test_convolution_ops 1/1 ... [2024-08-06 20:44:27.171181] 2024-08-06T20:44:27.1719764Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_convolution_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:44:27.171581] 2024-08-06T20:45:05.3214614Z 2024-08-06T20:45:05.3217892Z distributed/_tensor/test_convolution_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_convolution_ops_1.1_17b0a03ee402d14e_.log 2024-08-06T20:45:05.3220414Z Running 2 items in this shard: test/distributed/_tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_depthwise_convolution, test/distributed/_tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_downsampling_convolution 2024-08-06T20:45:05.3221772Z 2024-08-06T20:45:05.3222115Z Running distributed/_tensor/test_tensor_ops 1/1 ... [2024-08-06 20:45:05.321737] 2024-08-06T20:45:05.3225527Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_tensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:45:05.322143] 2024-08-06T20:47:01.8294741Z 2024-08-06T20:47:01.8299092Z distributed/_tensor/test_tensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_tensor_ops_1.1_db63d9a6ed07ea22_.log 2024-08-06T20:47:01.8322363Z Running 25 items in this shard: test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_aten_contiguous, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_clone, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_contiguous, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_detach, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_dtensor_dtype_conversion, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_empty_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_equal, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_fill_inplace, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_fill_inplace_partial_sum, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_full_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_gather, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_index, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_inplace_op, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_new_empty_strided, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_new_full, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_ones_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_ones_like_partial_sum, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_op_out_variant, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_scatter, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_slice, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_stack, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_where_type_promotion, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_zero_inplace, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_zeros_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_zeros_like_partial_sum 2024-08-06T20:47:01.8343934Z 2024-08-06T20:47:01.8344646Z Running distributed/optim/test_named_optimizer 1/1 ... [2024-08-06 20:47:01.829724] 2024-08-06T20:47:01.8347869Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/optim/test_named_optimizer.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:47:01.830135] 2024-08-06T20:47:04.7268246Z 2024-08-06T20:47:04.7269869Z distributed/optim/test_named_optimizer 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.optim.test_named_optimizer_1.1_c9b19f6fc8f539b9_.log 2024-08-06T20:47:04.7271218Z 2024-08-06T20:47:04.7273646Z Running distributed/_composable/test_checkpoint 1/1 ... [2024-08-06 20:47:04.727068] 2024-08-06T20:47:04.7278352Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_composable/test_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:47:04.727475] 2024-08-06T20:47:08.8001274Z 2024-08-06T20:47:08.8002642Z distributed/_composable/test_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_checkpoint_1.1_8c6ad52dbc2ad9fd_.log 2024-08-06T20:47:08.8006668Z Running 6 items in this shard: test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_checkpoint_kwargs, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_clears_state_on_error_in_forward, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_multi_args, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_random_cpu, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_tensor_only_cpu, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_tensor_only_gpu 2024-08-06T20:47:08.8009526Z 2024-08-06T20:47:08.8009888Z Running distributed/_tools/test_fsdp2_mem_tracker 1/1 ... [2024-08-06 20:47:08.800428] 2024-08-06T20:47:08.8011693Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tools/test_fsdp2_mem_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:47:08.800825] 2024-08-06T20:47:51.3005102Z 2024-08-06T20:47:51.3008340Z distributed/_tools/test_fsdp2_mem_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_fsdp2_mem_tracker_1.1_0ff3e0138145a17c_.log 2024-08-06T20:47:51.3011413Z Running 3 items in this shard: test/distributed/_tools/test_fsdp2_mem_tracker.py::TestTrackerFullyShard1DTrainingCore::test_tracker_multi_group_eager, test/distributed/_tools/test_fsdp2_mem_tracker.py::TestTrackerFullyShard1DTrainingCore::test_tracker_non_root_forward_backward, test/distributed/_tools/test_fsdp2_mem_tracker.py::TestTrackerFullyShard1DTrainingCompose::test_tracker_with_activation_checkpointing 2024-08-06T20:47:51.3013599Z 2024-08-06T20:47:51.3013955Z Running distributed/_tensor/test_pointwise_ops 1/1 ... [2024-08-06 20:47:51.300759] 2024-08-06T20:47:51.3015507Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_pointwise_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:47:51.301154] 2024-08-06T20:47:55.3741054Z 2024-08-06T20:47:55.3742489Z distributed/_tensor/test_pointwise_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_pointwise_ops_1.1_f16e419599270bf9_.log 2024-08-06T20:47:55.3746661Z Running 7 items in this shard: test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_activations, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_dropout, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_dropout_backward, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_dropout_errors, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_mul_out, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_partial_add, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_partial_mul 2024-08-06T20:47:55.3749943Z 2024-08-06T20:47:55.3750330Z Running distributed/checkpoint/test_save_load_api 1/1 ... [2024-08-06 20:47:55.374336] 2024-08-06T20:47:55.3752452Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/checkpoint/test_save_load_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:47:55.374732] 2024-08-06T20:48:02.7037111Z 2024-08-06T20:48:02.7038965Z distributed/checkpoint/test_save_load_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_save_load_api_1.1_6ac2886dd0dcddf5_.log 2024-08-06T20:48:02.7040558Z Running 1 items in this shard: test/distributed/checkpoint/test_save_load_api.py::TestSaveAndLoadAPI::test_auto_detect 2024-08-06T20:48:02.7041214Z 2024-08-06T20:48:02.7044951Z Running distributed/checkpoint/test_tp_checkpoint 1/1 ... [2024-08-06 20:48:02.704186] 2024-08-06T20:48:02.7050604Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/checkpoint/test_tp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:48:02.704674] 2024-08-06T20:48:15.6955477Z 2024-08-06T20:48:15.6958522Z distributed/checkpoint/test_tp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_tp_checkpoint_1.1_71e3820a06e43f4b_.log 2024-08-06T20:48:15.6962900Z Running 2 items in this shard: test/distributed/checkpoint/test_tp_checkpoint.py::TestTpCheckpoint::test_tp_checkpoint, test/distributed/checkpoint/test_tp_checkpoint.py::TestTpCheckpoint::test_tp_checkpoint_load_on_meta_device 2024-08-06T20:48:15.6965454Z 2024-08-06T20:48:15.6966255Z Running distributed/_tensor/experimental/test_local_map 1/1 ... [2024-08-06 20:48:15.696063] 2024-08-06T20:48:15.6972319Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/experimental/test_local_map.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:48:15.696702] 2024-08-06T20:48:37.4582251Z 2024-08-06T20:48:37.4585600Z distributed/_tensor/experimental/test_local_map 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_local_map_1.1_c4d93377e21cd17f_.log 2024-08-06T20:48:37.4588740Z Running 4 items in this shard: test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_correctness, test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_in_placements, test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_out_placements, test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_redistribute 2024-08-06T20:48:37.4590895Z 2024-08-06T20:48:38.4019151Z 2024-08-06T20:48:38.4019602Z real 7m3.844s 2024-08-06T20:48:38.4020293Z user 14m33.458s 2024-08-06T20:48:38.4020657Z sys 5m44.644s 2024-08-06T20:48:38.4021154Z + assert_git_not_dirty 2024-08-06T20:48:38.4021828Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *rocm* ]] 2024-08-06T20:48:38.4022865Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *xla* ]] 2024-08-06T20:48:38.4024489Z ++ git status --porcelain 2024-08-06T20:48:38.4025415Z ++ grep -v '?? third_party' 2024-08-06T20:48:41.2758922Z ++ true 2024-08-06T20:48:41.2760183Z + git_status= 2024-08-06T20:48:41.2760495Z + [[ -n '' ]] 2024-08-06T20:48:41.2760882Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *cuda* ]] 2024-08-06T20:48:41.2761417Z + [[ 3 == 1 ]] 2024-08-06T20:48:41.2761893Z + [[ 3 == 1 ]] 2024-08-06T20:48:41.2762399Z + cleanup_workspace 2024-08-06T20:48:41.2763030Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-06T20:48:41.2763997Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-06T20:48:41.2765097Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-06T20:48:41.2765688Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-06T20:48:41.2766345Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-06T20:48:41.2767097Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-06T20:48:41.2767701Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-06T20:48:41.8113607Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-06T20:48:41.8114112Z cat test/**/*_toprint.log || true 2024-08-06T20:48:41.8122373Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:48:41.8122858Z env: 2024-08-06T20:48:41.8123135Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:41.8123569Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:41.8124258Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:41.8124856Z ##[endgroup] 2024-08-06T20:48:41.8195193Z cat: test/**/*_toprint.log: No such file or directory 2024-08-06T20:48:41.8235937Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-06T20:48:41.8236433Z kill "$MONITOR_SCRIPT_PID" 2024-08-06T20:48:41.8244302Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:48:41.8244790Z env: 2024-08-06T20:48:41.8245084Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:41.8245503Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:41.8246195Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:41.8246826Z MONITOR_SCRIPT_PID: 104206 2024-08-06T20:48:41.8247171Z ##[endgroup] 2024-08-06T20:48:41.8413708Z Prepare all required actions 2024-08-06T20:48:41.8414247Z Getting action download info 2024-08-06T20:48:42.0015408Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-06T20:48:42.1506107Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-06T20:48:42.1506569Z with: 2024-08-06T20:48:42.1507040Z file-suffix: test-distributed-3-3-linux.8xlarge.nvidia.gpu_28428108140 2024-08-06T20:48:42.1507622Z s3-bucket: gha-artifacts 2024-08-06T20:48:42.1507954Z env: 2024-08-06T20:48:42.1508241Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:42.1508654Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:42.1509343Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:42.1509956Z ##[endgroup] 2024-08-06T20:48:42.1549456Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-06T20:48:42.1550124Z # Remove any previous test jsons if they exist 2024-08-06T20:48:42.1550618Z rm -f test-jsons-*.zip 2024-08-06T20:48:42.1551124Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-06T20:48:42.1559215Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:48:42.1559707Z env: 2024-08-06T20:48:42.1560003Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:42.1560467Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:42.1561179Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:42.1562016Z FILE_SUFFIX: test-distributed-3-3-linux.8xlarge.nvidia.gpu_28428108140 2024-08-06T20:48:42.1562611Z ##[endgroup] 2024-08-06T20:48:42.1804049Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-06T20:48:42.1839709Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-06T20:48:42.1840370Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-06T20:48:42.1847844Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-06T20:48:42.1850309Z adding: test/test-reports/td_exclusions-eea9378772e777cee319.json (deflated 86%) 2024-08-06T20:48:42.1851002Z adding: test/.pytorch-slow-tests.json (deflated 66%) 2024-08-06T20:48:42.1866786Z adding: test/.pytorch-disabled-tests.json (deflated 89%) 2024-08-06T20:48:42.1904495Z ##[group]Run # Remove any previous test reports if they exist 2024-08-06T20:48:42.1905085Z # Remove any previous test reports if they exist 2024-08-06T20:48:42.1905577Z rm -f test-reports-*.zip 2024-08-06T20:48:42.1906126Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-06T20:48:42.1913505Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:48:42.1913987Z env: 2024-08-06T20:48:42.1914283Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:42.1914705Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:42.1915406Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:42.1916201Z FILE_SUFFIX: test-distributed-3-3-linux.8xlarge.nvidia.gpu_28428108140 2024-08-06T20:48:42.1916744Z ##[endgroup] 2024-08-06T20:48:42.2130486Z adding: test/test-reports/python-pytest/distributed._tensor.debug.test_op_coverage/distributed._tensor.debug.test_op_coverage-4acd84478393dc8a.xml (deflated 35%) 2024-08-06T20:48:42.2134196Z adding: test/test-reports/python-pytest/distributed.pipelining.test_backward/distributed.pipelining.test_backward-fc62d39f78646008.xml (deflated 37%) 2024-08-06T20:48:42.2138010Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_logger/distributed._shard.sharded_tensor.test_logger-c4c1e670d4d477f4.xml (deflated 36%) 2024-08-06T20:48:42.2141660Z adding: test/test-reports/python-pytest/distributed.test_symmetric_memory/distributed.test_symmetric_memory-c77766a61735044d.xml (deflated 93%) 2024-08-06T20:48:42.2144192Z adding: test/test-reports/python-pytest/distributed._tensor.test_dtensor_compile/distributed._tensor.test_dtensor_compile-2edc18130eee0b93.xml (deflated 84%) 2024-08-06T20:48:42.2146041Z adding: test/test-reports/python-pytest/distributed.pipelining.test_pipe/distributed.pipelining.test_pipe-c03a9a7226ba4b98.xml (deflated 65%) 2024-08-06T20:48:42.2147567Z adding: test/test-reports/python-pytest/distributed._tools.test_mem_tracker/distributed._tools.test_mem_tracker-e79e05a891503f17.xml (deflated 59%) 2024-08-06T20:48:42.2149104Z adding: test/test-reports/python-pytest/distributed._tensor.test_experimental_ops/distributed._tensor.test_experimental_ops-5f1b7b81146bd81b.xml (deflated 56%) 2024-08-06T20:48:42.2150671Z adding: test/test-reports/python-pytest/distributed._tensor.test_common_rules/distributed._tensor.test_common_rules-8b8eeb73ea7e1ef1.xml (deflated 77%) 2024-08-06T20:48:42.2152269Z adding: test/test-reports/python-pytest/distributed.checkpoint.fsdp.test_fsdp_dsd/distributed.checkpoint.fsdp.test_fsdp_dsd-86f7ef4d68089d20.xml (deflated 58%) 2024-08-06T20:48:42.2153901Z adding: test/test-reports/python-pytest/distributed._tensor.test_xla_integration/distributed._tensor.test_xla_integration-9765b9f437dff4f2.xml (deflated 74%) 2024-08-06T20:48:42.2155588Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_tp_transform/distributed._tensor.experimental.test_tp_transform-85b72f8968af4bf3.xml (deflated 59%) 2024-08-06T20:48:42.2157616Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_dedup_tensors/distributed.checkpoint.test_dedup_tensors-ddd2cf8107084a05.xml (deflated 38%) 2024-08-06T20:48:42.2159275Z adding: test/test-reports/python-pytest/distributed.pipelining.test_unflatten/distributed.pipelining.test_unflatten-ae3f63ac20c065d8.xml (deflated 41%) 2024-08-06T20:48:42.2160881Z adding: test/test-reports/python-pytest/distributed._tensor.test_convolution_ops/distributed._tensor.test_convolution_ops-ccd376c72c792f39.xml (deflated 50%) 2024-08-06T20:48:42.2162401Z adding: test/test-reports/python-pytest/distributed._tensor.test_tensor_ops/distributed._tensor.test_tensor_ops-55587fb90ae5794d.xml (deflated 84%) 2024-08-06T20:48:42.2163945Z adding: test/test-reports/python-pytest/distributed._composable.test_checkpoint/distributed._composable.test_checkpoint-890b15fd2954b149.xml (deflated 72%) 2024-08-06T20:48:42.2184543Z adding: test/test-reports/python-pytest/distributed._tools.test_fsdp2_mem_tracker/distributed._tools.test_fsdp2_mem_tracker-bf8d1f677936fb8d.xml (deflated 55%) 2024-08-06T20:48:42.2186225Z adding: test/test-reports/python-pytest/distributed._tensor.test_pointwise_ops/distributed._tensor.test_pointwise_ops-4511aa0d37b2fe08.xml (deflated 70%) 2024-08-06T20:48:42.2187836Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_save_load_api/distributed.checkpoint.test_save_load_api-703fa22553f4321f.xml (deflated 50%) 2024-08-06T20:48:42.2189507Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_tp_checkpoint/distributed.checkpoint.test_tp_checkpoint-f6c2410bedfc4627.xml (deflated 51%) 2024-08-06T20:48:42.2191227Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_local_map/distributed._tensor.experimental.test_local_map-98054a9531bea52c.xml (deflated 64%) 2024-08-06T20:48:42.2222968Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-06T20:48:42.2223571Z # Remove any previous usage logs if they exist 2024-08-06T20:48:42.2224046Z rm -f logs-*.zip 2024-08-06T20:48:42.2224619Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-06T20:48:42.2225300Z # so check to see if the file exists first 2024-08-06T20:48:42.2225768Z if [ -f 'usage_log.txt' ]; then 2024-08-06T20:48:42.2226232Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-06T20:48:42.2226690Z fi 2024-08-06T20:48:42.2227033Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-06T20:48:42.2227686Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-06T20:48:42.2228162Z fi 2024-08-06T20:48:42.2235217Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:48:42.2235685Z env: 2024-08-06T20:48:42.2235976Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:42.2236411Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:42.2237387Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:42.2238216Z FILE_SUFFIX: test-distributed-3-3-linux.8xlarge.nvidia.gpu_28428108140 2024-08-06T20:48:42.2238781Z ##[endgroup] 2024-08-06T20:48:42.2283187Z adding: usage_log.txt (deflated 92%) 2024-08-06T20:48:42.2506906Z adding: test/test-reports/distributed._tensor.debug.test_op_coverage_1.1_4192fbd92cf7321e_.log (deflated 55%) 2024-08-06T20:48:42.2509091Z adding: test/test-reports/distributed.pipelining.test_backward_1.1_1c5595844dd514c8_.log (deflated 54%) 2024-08-06T20:48:42.2510766Z adding: test/test-reports/distributed._shard.sharded_tensor.test_logger_1.1_491fee39dd251ac0_.log (deflated 55%) 2024-08-06T20:48:42.2512202Z adding: test/test-reports/distributed.test_symmetric_memory_1.1_6b6e00a53d54af57_.log (deflated 86%) 2024-08-06T20:48:42.2513342Z adding: test/test-reports/distributed._tensor.test_dtensor_compile_1.1_71e65084c74df593_.log (deflated 84%) 2024-08-06T20:48:42.2514647Z adding: test/test-reports/distributed.pipelining.test_pipe_1.1_eda731756ed21888_.log (deflated 62%) 2024-08-06T20:48:42.2515904Z adding: test/test-reports/distributed._tools.test_mem_tracker_1.1_d6424aef5ada9f3a_.log (deflated 62%) 2024-08-06T20:48:42.2517355Z adding: test/test-reports/distributed._tensor.test_experimental_ops_1.1_54cb6d9c18d9ebbb_.log (deflated 63%) 2024-08-06T20:48:42.2518447Z adding: test/test-reports/distributed._tensor.test_common_rules_1.1_b3b1cf05ed7c4f93_.log (deflated 76%) 2024-08-06T20:48:42.2519763Z adding: test/test-reports/distributed.checkpoint.fsdp.test_fsdp_dsd_1.1_e5cc9a0b80d84b8f_.log (deflated 89%) 2024-08-06T20:48:42.2521729Z adding: test/test-reports/distributed._tensor.test_xla_integration_1.1_d012f1d3ce1f60c0_.log (deflated 66%) 2024-08-06T20:48:42.2524136Z adding: test/test-reports/distributed._tensor.experimental.test_tp_transform_1.1_e3559fa9c020dd98_.log (deflated 64%) 2024-08-06T20:48:42.2526539Z adding: test/test-reports/distributed.checkpoint.test_dedup_tensors_1.1_1d38d54779eecec0_.log (deflated 52%) 2024-08-06T20:48:42.2529099Z adding: test/test-reports/distributed.pipelining.test_unflatten_1.1_2aac9a5505d75ba4_.log (deflated 54%) 2024-08-06T20:48:42.2531381Z adding: test/test-reports/distributed._tensor.test_convolution_ops_1.1_17b0a03ee402d14e_.log (deflated 59%) 2024-08-06T20:48:42.2533571Z adding: test/test-reports/distributed._tensor.test_tensor_ops_1.1_db63d9a6ed07ea22_.log (deflated 79%) 2024-08-06T20:48:42.2535696Z adding: test/test-reports/distributed.optim.test_named_optimizer_1.1_c9b19f6fc8f539b9_.log (deflated 35%) 2024-08-06T20:48:42.2537865Z adding: test/test-reports/distributed._composable.test_checkpoint_1.1_8c6ad52dbc2ad9fd_.log (deflated 70%) 2024-08-06T20:48:42.2539497Z adding: test/test-reports/distributed._tools.test_fsdp2_mem_tracker_1.1_0ff3e0138145a17c_.log (deflated 65%) 2024-08-06T20:48:42.2540560Z adding: test/test-reports/distributed._tensor.test_pointwise_ops_1.1_f16e419599270bf9_.log (deflated 69%) 2024-08-06T20:48:42.2541660Z adding: test/test-reports/distributed.checkpoint.test_save_load_api_1.1_6ac2886dd0dcddf5_.log (deflated 51%) 2024-08-06T20:48:42.2542753Z adding: test/test-reports/distributed.checkpoint.test_tp_checkpoint_1.1_71e3820a06e43f4b_.log (deflated 60%) 2024-08-06T20:48:42.2543872Z adding: test/test-reports/distributed._tensor.experimental.test_local_map_1.1_c4d93377e21cd17f_.log (deflated 67%) 2024-08-06T20:48:42.2577505Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-06T20:48:42.2578153Z # Remove any previous debugging artifacts if they exist 2024-08-06T20:48:42.2578810Z rm -f debug-*.zip 2024-08-06T20:48:42.2579197Z if [ -d 'test/debug' ]; then 2024-08-06T20:48:42.2579636Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-06T20:48:42.2580074Z fi 2024-08-06T20:48:42.2587270Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:48:42.2587751Z env: 2024-08-06T20:48:42.2588039Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:42.2588452Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:42.2589141Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:42.2589925Z FILE_SUFFIX: test-distributed-3-3-linux.8xlarge.nvidia.gpu_28428108140 2024-08-06T20:48:42.2590462Z ##[endgroup] 2024-08-06T20:48:42.2682650Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-06T20:48:42.2683081Z with: 2024-08-06T20:48:42.2683363Z s3-bucket: gha-artifacts 2024-08-06T20:48:42.2683782Z s3-prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:48:42.2684241Z retention-days: 14 2024-08-06T20:48:42.2684567Z if-no-files-found: warn 2024-08-06T20:48:42.2684931Z path: test-jsons-*.zip 2024-08-06T20:48:42.2685269Z name: artifact 2024-08-06T20:48:42.2685562Z region: us-east-1 2024-08-06T20:48:42.2685870Z env: 2024-08-06T20:48:42.2686150Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:42.2686570Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:42.2687254Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:42.2687862Z ##[endgroup] 2024-08-06T20:48:42.7107078Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-06T20:48:42.7107940Z With the provided path, there will be 1 file uploaded 2024-08-06T20:48:42.7108511Z Uploading to s3 prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:48:42.7155021Z Starting upload of test-jsons-test-distributed-3-3-linux.8xlarge.nvidia.gpu_28428108140.zip 2024-08-06T20:48:42.9291011Z Finished upload of test-jsons-test-distributed-3-3-linux.8xlarge.nvidia.gpu_28428108140.zip 2024-08-06T20:48:42.9467627Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-06T20:48:42.9468071Z with: 2024-08-06T20:48:42.9468357Z s3-bucket: gha-artifacts 2024-08-06T20:48:42.9468774Z s3-prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:48:42.9469237Z retention-days: 14 2024-08-06T20:48:42.9469788Z if-no-files-found: error 2024-08-06T20:48:42.9470158Z path: test-reports-*.zip 2024-08-06T20:48:42.9470513Z name: artifact 2024-08-06T20:48:42.9470804Z region: us-east-1 2024-08-06T20:48:42.9471111Z env: 2024-08-06T20:48:42.9471397Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:42.9471811Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:42.9472502Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:42.9473110Z ##[endgroup] 2024-08-06T20:48:43.3898860Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-06T20:48:43.3899826Z With the provided path, there will be 1 file uploaded 2024-08-06T20:48:43.3900415Z Uploading to s3 prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:48:43.3947195Z Starting upload of test-reports-test-distributed-3-3-linux.8xlarge.nvidia.gpu_28428108140.zip 2024-08-06T20:48:43.5437925Z Finished upload of test-reports-test-distributed-3-3-linux.8xlarge.nvidia.gpu_28428108140.zip 2024-08-06T20:48:43.5609747Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-06T20:48:43.5610191Z with: 2024-08-06T20:48:43.5610472Z s3-bucket: gha-artifacts 2024-08-06T20:48:43.5610891Z s3-prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:48:43.5611341Z retention-days: 14 2024-08-06T20:48:43.5611666Z if-no-files-found: ignore 2024-08-06T20:48:43.5612021Z path: logs-*.zip 2024-08-06T20:48:43.5612331Z name: artifact 2024-08-06T20:48:43.5612618Z region: us-east-1 2024-08-06T20:48:43.5612929Z env: 2024-08-06T20:48:43.5613211Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:43.5613624Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:43.5614487Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:43.5615120Z ##[endgroup] 2024-08-06T20:48:43.9992732Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-06T20:48:43.9993728Z With the provided path, there will be 1 file uploaded 2024-08-06T20:48:43.9994330Z Uploading to s3 prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:48:44.0041336Z Starting upload of logs-test-distributed-3-3-linux.8xlarge.nvidia.gpu_28428108140.zip 2024-08-06T20:48:44.1359417Z Finished upload of logs-test-distributed-3-3-linux.8xlarge.nvidia.gpu_28428108140.zip 2024-08-06T20:48:44.1530541Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-06T20:48:44.1530979Z with: 2024-08-06T20:48:44.1531260Z s3-bucket: gha-artifacts 2024-08-06T20:48:44.1531682Z s3-prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T20:48:44.1532134Z retention-days: 14 2024-08-06T20:48:44.1532461Z if-no-files-found: ignore 2024-08-06T20:48:44.1532845Z path: debug-*.zip 2024-08-06T20:48:44.1533160Z name: artifact 2024-08-06T20:48:44.1533451Z region: us-east-1 2024-08-06T20:48:44.1533757Z env: 2024-08-06T20:48:44.1534042Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:44.1534461Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:44.1535147Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:44.1535780Z ##[endgroup] 2024-08-06T20:48:44.5805885Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-06T20:48:44.6049003Z ##[group]Run # shellcheck disable=SC2156 2024-08-06T20:48:44.6049562Z # shellcheck disable=SC2156 2024-08-06T20:48:44.6050291Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-06T20:48:44.6059162Z shell: /usr/bin/bash -e {0} 2024-08-06T20:48:44.6059513Z env: 2024-08-06T20:48:44.6059805Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:44.6060240Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:44.6060912Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:44.6061521Z ##[endgroup] 2024-08-06T20:48:44.9029430Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-06T20:48:44.9030158Z with: 2024-08-06T20:48:44.9030420Z env: 2024-08-06T20:48:44.9030717Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:44.9031162Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:44.9031833Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:44.9032460Z ##[endgroup] 2024-08-06T20:48:44.9060877Z ##[group]Run set -eou pipefail 2024-08-06T20:48:44.9061289Z set -eou pipefail 2024-08-06T20:48:44.9061626Z  2024-08-06T20:48:44.9062092Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-06T20:48:44.9062671Z for _ in $(seq 1440); do 2024-08-06T20:48:44.9063086Z  # Break if no ssh session exists anymore 2024-08-06T20:48:44.9063534Z  if [ "$(who)" = "" ]; then 2024-08-06T20:48:44.9063924Z  break 2024-08-06T20:48:44.9064279Z  fi 2024-08-06T20:48:44.9064571Z  echo "." 2024-08-06T20:48:44.9064912Z  sleep 5 2024-08-06T20:48:44.9065231Z done 2024-08-06T20:48:44.9072564Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:48:44.9073044Z env: 2024-08-06T20:48:44.9073339Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:44.9073754Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:44.9074440Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:44.9075049Z ##[endgroup] 2024-08-06T20:48:44.9098762Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-06T20:48:44.9150513Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-06T20:48:44.9151220Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-06T20:48:44.9151745Z # shellcheck disable=SC2046 2024-08-06T20:48:44.9152194Z docker stop $(docker ps -q) || true 2024-08-06T20:48:44.9152653Z # Prune all of the docker images 2024-08-06T20:48:44.9153076Z docker system prune -af 2024-08-06T20:48:44.9160631Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:48:44.9161120Z env: 2024-08-06T20:48:44.9161396Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:48:44.9161833Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-06T20:48:44.9162517Z DOCKER_CONTAINER_ID: 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:44.9163115Z ##[endgroup] 2024-08-06T20:48:45.2658863Z 9268de398b04 2024-08-06T20:48:45.8295111Z Deleted Containers: 2024-08-06T20:48:45.8295612Z 9268de398b04d43cff2483fe5039b0516b7cd264a4d627039392342b8a61d200 2024-08-06T20:48:45.8296024Z 2024-08-06T20:48:52.4434336Z Deleted Images: 2024-08-06T20:48:52.4435424Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:48:52.4437462Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9@sha256:66181a134b3ebc35388dabcc46dc11e767152fd72514dad76711590f50b650f7 2024-08-06T20:48:52.4439354Z deleted: sha256:8e4e19ae0377c273bc603b24bc49d406d6f755072d4534d040c86afbe761a129 2024-08-06T20:48:52.4440185Z deleted: sha256:bee5df81b6fe1ee0aae7e536a9bda16351d0515be5b789ad50bf0ac60fd41e33 2024-08-06T20:48:52.4441000Z deleted: sha256:8ee1ee099b65b2d73d66dc3330f82f8be2dd6cbdccf3d718b448a5e3419c214e 2024-08-06T20:48:52.4441810Z deleted: sha256:aed7424cdc4428e32e398a33e98e6e252075e6280e7068e20df7626451fa312a 2024-08-06T20:48:52.4442604Z deleted: sha256:61b7d2994ed16481c3de07839b5a3bb779dc2f3f716ad49acb91537042c3b7d1 2024-08-06T20:48:52.4443385Z deleted: sha256:79999dfec9b5df258486b014aa99c30979900c4131effd2a7bc18251cd5fc372 2024-08-06T20:48:52.4444191Z deleted: sha256:9020e3bab07b431af26c788ac2d12cbca9a76ffb1c923ea45a38e872942e1f34 2024-08-06T20:48:52.4445260Z deleted: sha256:387e2f2f359df5f69c500ae9589c224d8283ae5418520509b3c9ef0a45d116fa 2024-08-06T20:48:52.4446081Z deleted: sha256:5ddfc73c14e2e0a762c51ddecfb2710258366e1d048a8bf56d70268486907b99 2024-08-06T20:48:52.4447073Z deleted: sha256:4dffb251634d1c8105e68310ef1b2ffb923dac4c4139f494be0938ad032a1bca 2024-08-06T20:48:52.4447872Z deleted: sha256:e0df7617281f0e4396253d2f0c40d1418ad604dc0912cae13f153fe22e316fa0 2024-08-06T20:48:52.4448657Z deleted: sha256:d239b3ccf05c2850364c3f73acbc6e0017a25f8faa4c3abaa8524e63dc9f9c29 2024-08-06T20:48:52.4449479Z deleted: sha256:918d0912faf4251f3be58f9a5aaa80f59a180ace4771b9ce903cedf94a57723a 2024-08-06T20:48:52.4450268Z deleted: sha256:09d5696392736fa88c52a0656d1d831d2bcb4015630810412565a5a086eb98ff 2024-08-06T20:48:52.4451050Z deleted: sha256:67542af70388f69b345fc7f0965e1fce3c552ee987689f804c8406536f16534f 2024-08-06T20:48:52.4451833Z deleted: sha256:350337bafbf2976e8d4e0b7ce3cc66b2cb43ebb8622fa6e538b02f561f929393 2024-08-06T20:48:52.4452667Z deleted: sha256:00fe9fe12fefe10d5dbfb2f855aa385e840f360deeccbc9ee92cae51c6eea4ba 2024-08-06T20:48:52.4453487Z deleted: sha256:9895af0586ff7d94936462a92d7cec0d8ec3b01df27b664f7e4ac277ce7bbe9e 2024-08-06T20:48:52.4454285Z deleted: sha256:7dd7dbc3ec792cf9d043ff867afe169d15fe414138097ad9247a5f388d3eeccf 2024-08-06T20:48:52.4455094Z deleted: sha256:e0a3ace1b87c67d71f5db76d211740431660a7fbcd80415333c9e7b7b9a2e89b 2024-08-06T20:48:52.4455892Z deleted: sha256:b9c56b5040d5d5998961b788563b85e5be12f678bc6b3976d089a81f1c93b7e8 2024-08-06T20:48:52.4456683Z deleted: sha256:9232fe8591973938b7bdb0a7a31dcdbc79f9667d6e011a2facf4e17fb77d4922 2024-08-06T20:48:52.4457480Z deleted: sha256:52f5fbe778d735123327825c6a16dcec9bc84c2623087bb4c7c094ac99764ae7 2024-08-06T20:48:52.4458286Z deleted: sha256:4b7c4121b76b04d5c7e4a0b77add435c0caa99cccff506ea704eba27fd2b2166 2024-08-06T20:48:52.4459084Z deleted: sha256:795dd765ace5b617808fc40cd576b449ae01b8671290e6df6b49d86ea79f4145 2024-08-06T20:48:52.4459893Z deleted: sha256:51cc651f1ffbeef49ad010ab2d4cf2d5ce5f8f5767c2133346a959c773d837bb 2024-08-06T20:48:52.4460714Z deleted: sha256:4c3019549b40f7a268d344eae162d7a7cc78855880db865944da136e01c190e9 2024-08-06T20:48:52.4461530Z deleted: sha256:855b789ec056edaab8cc8dad50a82e9dbfb9b20bcf39c70d9c3bd40080b2b65f 2024-08-06T20:48:52.4462330Z deleted: sha256:cef853cb7e0f5d6351692a4b9a0413a2496ea3c7bfe5d8063528ac2d6c17696f 2024-08-06T20:48:52.4463128Z deleted: sha256:4f7b163ef5ff693ce9f7380bdf7b526131f2567489342f0630949c9341e1c63f 2024-08-06T20:48:52.4463922Z deleted: sha256:09f12e5e102232b4ec9fd5168f4aa4b20ee0dfc755b053ac57690657e72e3fe3 2024-08-06T20:48:52.4464709Z deleted: sha256:c9962be393e03353af0d6a26f5478a94cacb5e528ee3f97609fac594c9a5d387 2024-08-06T20:48:52.4465524Z deleted: sha256:ac7aa4ec25b0d9a035c438effaa309071729de3f3a0f96e8e481b9839e7460b8 2024-08-06T20:48:52.4466327Z deleted: sha256:63ff2106018988888d022ee38d6ce1805aa29d8368a18dec1ab8eb587799ab44 2024-08-06T20:48:52.4467102Z deleted: sha256:3298c1acdeb701f11351366cfe680a84b8290666fdcd76af5ac79f226422fc0d 2024-08-06T20:48:52.4467918Z deleted: sha256:f076cc29eff5dae969bc7c05003d680b6e683eaef971c0a3f67a8a4d12693ee4 2024-08-06T20:48:52.4468761Z deleted: sha256:2604655e98c557e6de2e650d21cd0306c01f12204f3538386a9f374d003a4d70 2024-08-06T20:48:52.4469579Z deleted: sha256:fb942194d6d4fc6f9c8a4c8cba17330563877c92369892edc9b005b8b491f204 2024-08-06T20:48:52.4470370Z deleted: sha256:d349cc387708a540eee839ccd78494458dc5f1ccf989db608d25c5f798ec8c76 2024-08-06T20:48:52.4471168Z deleted: sha256:269864865f9b12ee399ca9ee3e15a16cedb86fd4889e01a83567afb357a774e6 2024-08-06T20:48:52.4471964Z deleted: sha256:236a91ce743cd500427338a78915ff834ae80331fcc06afccfca82a35bffb63c 2024-08-06T20:48:52.4472761Z deleted: sha256:de968bbc1d8bc2ad420d6a1c78a9a6cf8a763bb855ca3aefe7de6c5c52201f7a 2024-08-06T20:48:52.4473591Z deleted: sha256:0f15ecc513ffdac89965bd76cd9f8c7f4e37bbe7be0bfa086353133e2dd4f086 2024-08-06T20:48:52.4474420Z deleted: sha256:754bb9b5a5af40b6ad88f0ab14be37b9e512e25edccbfc9fd5e0c5a88ca09c0f 2024-08-06T20:48:52.4475320Z deleted: sha256:427104437dfa1883feec2c441eb4e4f825481883c6d3d31759361cbc924a27fe 2024-08-06T20:48:52.4476137Z deleted: sha256:0b404434cf2c3087ef28ea3cdc5813cc555f6e261c2514a7ab5bf5d484b5b575 2024-08-06T20:48:52.4477053Z deleted: sha256:bfb91e062bab9dafb63d0d4ab366026692fbd63e642da9ab28f04c8bba9b56a3 2024-08-06T20:48:52.4478149Z deleted: sha256:54617b36a396e9e82f2092257d2f621c90de460e0e54b1cf8df065573fbaa0ba 2024-08-06T20:48:52.4478940Z deleted: sha256:d8bdc337db1750bb3ebc7c8e4e1bd48e722218f261cb35ec2056ceaa6d3c4e9c 2024-08-06T20:48:52.4479775Z deleted: sha256:25cbacf4be7b2bbcbeaf7375d0eff0eaa33e94c5bef20ff0993bf074fdaa54eb 2024-08-06T20:48:52.4480600Z deleted: sha256:43744f05dcf6dc44c2f5e06533b13e085319a78b6b5e17eba5e7bd4419f5c905 2024-08-06T20:48:52.4481385Z deleted: sha256:2ea7d74f63d16a8d6e14a87e0e916f67f99536b21ac53e80b27786dcd1ce36fd 2024-08-06T20:48:52.4482190Z deleted: sha256:dbfb6031f18e50c89ff1a268c1e80e2fa88e727c48656718b326b60e8ef0d383 2024-08-06T20:48:52.4483008Z deleted: sha256:3eb3bb1755fcc3d4aba9abe2d2270ed2a55ec6254b3e87ca7688370e67e707a3 2024-08-06T20:48:52.4483801Z deleted: sha256:04605aba2d51924a85499584edda24c28390eb777c1c88eff2f4191b48e3da8c 2024-08-06T20:48:52.4484608Z deleted: sha256:117ff9e5eddba4582a39d416380eadf1c245237612c5ae0d6781d36c36bcb87f 2024-08-06T20:48:52.4485416Z deleted: sha256:ebf86c84eafa12a81b8eb66541a56c8056f4f9bc30e4be761771e53c6e1342aa 2024-08-06T20:48:52.4486211Z deleted: sha256:23d7f7e0c9d7e60acb59e60da9b4ee4e1840c1c406bf2082822c49430975529d 2024-08-06T20:48:52.4487019Z deleted: sha256:3e7e5583645195af5eeb60fe49024a4aa2f0eb3be28ff95c9859bb6233c29c4b 2024-08-06T20:48:52.4487818Z deleted: sha256:45b8f0dd78e72f0156b1fbc4dcb14200c61954f37a75129c753275e5a45ccd9b 2024-08-06T20:48:52.4488609Z deleted: sha256:f6b7a042384e2f2bb3401da2b9f4e9b9f942adb70be7e7f89c46fa81a42774d8 2024-08-06T20:48:52.4489424Z deleted: sha256:eb547a7b8fe732bffe8cc9a66aa19f2b41f6e0616a8a27040307c022d00fbc0d 2024-08-06T20:48:52.4490227Z deleted: sha256:20f729283bb4f2063f3a8baa0b2afc17911da345bf367e1918d8cc09ddd6d964 2024-08-06T20:48:52.4491040Z deleted: sha256:8ce8da9ab0319625a66c35d24ddadf8d2688d16c5024ba2345099431e0aa7ff8 2024-08-06T20:48:52.4491838Z deleted: sha256:e7a91181c74f879360530f0bbe957b8ff96a1ec19adde8a41d9e3fa2038d0d16 2024-08-06T20:48:52.4492673Z deleted: sha256:212b7a0fad7208624a2bc56768d8c489d2690142d0db0fa20f69eb648234e8d1 2024-08-06T20:48:52.4493475Z deleted: sha256:72315e37f51ba8bcc88fc5b56f92e5bb6264456d9a755afa01fd19f9128c1df7 2024-08-06T20:48:52.4494276Z deleted: sha256:bfc5ab07cfab1698147152988eabdc7ebbdeb35b6d9aefd9360dd64104cd3888 2024-08-06T20:48:52.4495084Z deleted: sha256:89edf8c906678c217b9a88160e1561991f53909c3fd27a4a5e1f0ca0f7ff7888 2024-08-06T20:48:52.4495891Z deleted: sha256:d5ba67f8f9b0c2b6da06d2d5d9d5fedc9f17cbe1a5e13a1a4033a4619f4528ba 2024-08-06T20:48:52.4496679Z deleted: sha256:5d61311e1a293c0c9642e34871719525dfeec7028e4b45e08344a469e3fe24bf 2024-08-06T20:48:52.4497473Z deleted: sha256:b89bb70f745a94c0edd5e974aee294e160735f76fd666a1a06a0b9b1be3caf63 2024-08-06T20:48:52.4498291Z deleted: sha256:9e5477f1f21a38c3dfe21a56a7d77d1b1a42453aed3e11729afcbb9d5731770b 2024-08-06T20:48:52.4499090Z deleted: sha256:b76f4af91cffe2bb7b3f09960f5bdbb26f741419defda65c451c4407fc5b0a92 2024-08-06T20:48:52.4499907Z deleted: sha256:3f8748a00ebd394c0e9d9d481972d764dff58896841ef0150b3f7efaac0f4a4a 2024-08-06T20:48:52.4500700Z deleted: sha256:13989c771e3ba4d40f618344e55e5cb7062a35041b5efd69e2f740ba819f46ee 2024-08-06T20:48:52.4501506Z deleted: sha256:9ebb27837f6c67ad8c17e9af4e63c008faae03fa226c3641dd8bbbfeefb106c4 2024-08-06T20:48:52.4502315Z deleted: sha256:8c16907d735950f8093582bc3cf392b70e002456bf9c4224de5a5b86b78b3ea2 2024-08-06T20:48:52.4503091Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-08-06T20:48:52.4503591Z 2024-08-06T20:48:52.4503752Z Total reclaimed space: 26.03GB 2024-08-06T20:48:52.4622220Z Post job cleanup. 2024-08-06T20:48:52.4679394Z Post job cleanup. 2024-08-06T20:48:52.5836606Z [command]/usr/bin/git version 2024-08-06T20:48:52.5889722Z git version 2.40.1 2024-08-06T20:48:52.5950850Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/367f3a3d-188e-4f10-8efb-2917152b78ae' before making global git config changes 2024-08-06T20:48:52.5952290Z Adding repository directory to the temporary git global config as a safe directory 2024-08-06T20:48:52.5958700Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:48:52.5999530Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-06T20:48:52.6035147Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-06T20:48:52.6346571Z Entering 'android/libs/fbjni' 2024-08-06T20:48:52.6396642Z Entering 'third_party/FP16' 2024-08-06T20:48:52.6446559Z Entering 'third_party/FXdiv' 2024-08-06T20:48:52.6497956Z Entering 'third_party/NNPACK' 2024-08-06T20:48:52.6549144Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:48:52.6600863Z Entering 'third_party/XNNPACK' 2024-08-06T20:48:52.6669325Z Entering 'third_party/benchmark' 2024-08-06T20:48:52.6723882Z Entering 'third_party/cpp-httplib' 2024-08-06T20:48:52.6772224Z Entering 'third_party/cpuinfo' 2024-08-06T20:48:52.6823863Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:48:52.6874211Z Entering 'third_party/cutlass' 2024-08-06T20:48:52.6932360Z Entering 'third_party/eigen' 2024-08-06T20:48:52.6984270Z Entering 'third_party/fbgemm' 2024-08-06T20:48:52.7036000Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:48:52.7086450Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:48:52.7135947Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:48:52.7192064Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:48:52.7243892Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:48:52.7294430Z Entering 'third_party/flatbuffers' 2024-08-06T20:48:52.7347959Z Entering 'third_party/fmt' 2024-08-06T20:48:52.7398992Z Entering 'third_party/foxi' 2024-08-06T20:48:52.7448213Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:48:52.7498560Z Entering 'third_party/gloo' 2024-08-06T20:48:52.7548454Z Entering 'third_party/googletest' 2024-08-06T20:48:52.7599294Z Entering 'third_party/ideep' 2024-08-06T20:48:52.7647558Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:48:52.7704352Z Entering 'third_party/ittapi' 2024-08-06T20:48:52.7755971Z Entering 'third_party/kineto' 2024-08-06T20:48:52.7807749Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:48:52.7857838Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:48:52.7908374Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:48:52.7958333Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:48:52.8009564Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:48:52.8059948Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:48:52.8111327Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:48:52.8161190Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:48:52.8211815Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:48:52.8262740Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:48:52.8314120Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:48:52.8364742Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:48:52.8414514Z Entering 'third_party/mimalloc' 2024-08-06T20:48:52.8464802Z Entering 'third_party/nccl/nccl' 2024-08-06T20:48:52.8514958Z Entering 'third_party/nlohmann' 2024-08-06T20:48:52.8566783Z Entering 'third_party/onnx' 2024-08-06T20:48:52.8635105Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:48:52.8685859Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:48:52.8739051Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:48:52.8791151Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:48:52.8840899Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:48:52.8890493Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:48:52.8940107Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:48:52.8990592Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:48:52.9039538Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:48:52.9087265Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:48:52.9135005Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:48:52.9186589Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:48:52.9237691Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:48:52.9310730Z Entering 'third_party/pocketfft' 2024-08-06T20:48:52.9362633Z Entering 'third_party/protobuf' 2024-08-06T20:48:52.9417562Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:48:52.9466966Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:48:52.9521131Z Entering 'third_party/psimd' 2024-08-06T20:48:52.9571792Z Entering 'third_party/pthreadpool' 2024-08-06T20:48:52.9621650Z Entering 'third_party/pybind11' 2024-08-06T20:48:52.9671766Z Entering 'third_party/python-peachpy' 2024-08-06T20:48:52.9722406Z Entering 'third_party/sleef' 2024-08-06T20:48:52.9773407Z Entering 'third_party/tensorpipe' 2024-08-06T20:48:52.9823161Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:48:52.9872588Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:48:52.9921920Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:48:52.9971733Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:48:53.0020137Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:48:53.0091032Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-06T20:48:53.0118792Z http.https://github.com/.extraheader 2024-08-06T20:48:53.0130214Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-06T20:48:53.0167447Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-06T20:48:53.0465951Z Entering 'android/libs/fbjni' 2024-08-06T20:48:53.0498533Z http.https://github.com/.extraheader 2024-08-06T20:48:53.0532115Z Entering 'third_party/FP16' 2024-08-06T20:48:53.0564295Z http.https://github.com/.extraheader 2024-08-06T20:48:53.0597193Z Entering 'third_party/FXdiv' 2024-08-06T20:48:53.0630170Z http.https://github.com/.extraheader 2024-08-06T20:48:53.0663193Z Entering 'third_party/NNPACK' 2024-08-06T20:48:53.0696326Z http.https://github.com/.extraheader 2024-08-06T20:48:53.0729646Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:48:53.0762130Z http.https://github.com/.extraheader 2024-08-06T20:48:53.0795246Z Entering 'third_party/XNNPACK' 2024-08-06T20:48:53.0827290Z http.https://github.com/.extraheader 2024-08-06T20:48:53.0879352Z Entering 'third_party/benchmark' 2024-08-06T20:48:53.0910860Z http.https://github.com/.extraheader 2024-08-06T20:48:53.0944754Z Entering 'third_party/cpp-httplib' 2024-08-06T20:48:53.0976639Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1009550Z Entering 'third_party/cpuinfo' 2024-08-06T20:48:53.1041783Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1075199Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:48:53.1107149Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1140042Z Entering 'third_party/cutlass' 2024-08-06T20:48:53.1172461Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1214456Z Entering 'third_party/eigen' 2024-08-06T20:48:53.1246735Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1283817Z Entering 'third_party/fbgemm' 2024-08-06T20:48:53.1315735Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1350482Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:48:53.1382708Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1416392Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:48:53.1448302Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1482659Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:48:53.1513737Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1553233Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:48:53.1586591Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1620142Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:48:53.1651869Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1686675Z Entering 'third_party/flatbuffers' 2024-08-06T20:48:53.1718961Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1754178Z Entering 'third_party/fmt' 2024-08-06T20:48:53.1786972Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1820740Z Entering 'third_party/foxi' 2024-08-06T20:48:53.1852508Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1885968Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:48:53.1917776Z http.https://github.com/.extraheader 2024-08-06T20:48:53.1950314Z Entering 'third_party/gloo' 2024-08-06T20:48:53.1982666Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2015795Z Entering 'third_party/googletest' 2024-08-06T20:48:53.2047828Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2081043Z Entering 'third_party/ideep' 2024-08-06T20:48:53.2112564Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2144686Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:48:53.2176434Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2217285Z Entering 'third_party/ittapi' 2024-08-06T20:48:53.2250043Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2283185Z Entering 'third_party/kineto' 2024-08-06T20:48:53.2315079Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2348442Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:48:53.2381255Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2414651Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:48:53.2446510Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2482477Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:48:53.2513455Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2547576Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:48:53.2579449Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2612795Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:48:53.2645290Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2677768Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:48:53.2709306Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2744227Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:48:53.2777191Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2811196Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:48:53.2843310Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2876012Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:48:53.2908055Z http.https://github.com/.extraheader 2024-08-06T20:48:53.2942403Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:48:53.2973836Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3010942Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:48:53.3042464Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3074793Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:48:53.3107282Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3141897Z Entering 'third_party/mimalloc' 2024-08-06T20:48:53.3174545Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3208210Z Entering 'third_party/nccl/nccl' 2024-08-06T20:48:53.3241583Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3273881Z Entering 'third_party/nlohmann' 2024-08-06T20:48:53.3306145Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3341323Z Entering 'third_party/onnx' 2024-08-06T20:48:53.3373721Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3424578Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:48:53.3456858Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3491274Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:48:53.3523404Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3558639Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:48:53.3590302Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3625373Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:48:53.3657782Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3690431Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:48:53.3721873Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3754306Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:48:53.3785906Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3818654Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:48:53.3849591Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3884158Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:48:53.3915302Z http.https://github.com/.extraheader 2024-08-06T20:48:53.3947691Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:48:53.3979053Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4011589Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:48:53.4043847Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4075969Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:48:53.4107609Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4142723Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:48:53.4175089Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4209867Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:48:53.4241665Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4297336Z Entering 'third_party/pocketfft' 2024-08-06T20:48:53.4330239Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4363001Z Entering 'third_party/protobuf' 2024-08-06T20:48:53.4394736Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4431620Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:48:53.4463206Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4496660Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:48:53.4529069Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4564208Z Entering 'third_party/psimd' 2024-08-06T20:48:53.4595598Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4628216Z Entering 'third_party/pthreadpool' 2024-08-06T20:48:53.4660701Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4693397Z Entering 'third_party/pybind11' 2024-08-06T20:48:53.4725595Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4758977Z Entering 'third_party/python-peachpy' 2024-08-06T20:48:53.4790941Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4823977Z Entering 'third_party/sleef' 2024-08-06T20:48:53.4856049Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4889786Z Entering 'third_party/tensorpipe' 2024-08-06T20:48:53.4921743Z http.https://github.com/.extraheader 2024-08-06T20:48:53.4954784Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:48:53.4987308Z http.https://github.com/.extraheader 2024-08-06T20:48:53.5020440Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:48:53.5052056Z http.https://github.com/.extraheader 2024-08-06T20:48:53.5084354Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:48:53.5115265Z http.https://github.com/.extraheader 2024-08-06T20:48:53.5148425Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:48:53.5180268Z http.https://github.com/.extraheader 2024-08-06T20:48:53.5212148Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:48:53.5244671Z http.https://github.com/.extraheader 2024-08-06T20:48:53.5399391Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-06T20:48:53.5426891Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-06T20:48:53.5433870Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:48:53.5434366Z ##[endgroup] 2024-08-06T20:48:59.2394905Z Cleaning up orphan processes