2024-06-26T06:31:06.5945738Z Current runner version: '2.317.0' 2024-06-26T06:31:06.5954420Z Runner name: 'i-011558b4ba4829615' 2024-06-26T06:31:06.5955484Z Runner group name: 'Default' 2024-06-26T06:31:06.5956565Z Machine name: 'ip-10-0-13-167' 2024-06-26T06:31:06.5962491Z ##[group]GITHUB_TOKEN Permissions 2024-06-26T06:31:06.5965023Z Actions: read 2024-06-26T06:31:06.5965796Z Attestations: read 2024-06-26T06:31:06.5966433Z Checks: read 2024-06-26T06:31:06.5966970Z Contents: read 2024-06-26T06:31:06.5967684Z Deployments: read 2024-06-26T06:31:06.5968315Z Discussions: read 2024-06-26T06:31:06.5968893Z Issues: read 2024-06-26T06:31:06.5969518Z Metadata: read 2024-06-26T06:31:06.5970127Z Packages: read 2024-06-26T06:31:06.5970686Z Pages: read 2024-06-26T06:31:06.5971370Z PullRequests: read 2024-06-26T06:31:06.5972448Z RepositoryProjects: read 2024-06-26T06:31:06.5973127Z SecurityEvents: read 2024-06-26T06:31:06.5973840Z Statuses: read 2024-06-26T06:31:06.5974466Z ##[endgroup] 2024-06-26T06:31:06.5979352Z Secret source: Actions 2024-06-26T06:31:06.5980389Z Prepare workflow directory 2024-06-26T06:31:07.0661114Z Prepare all required actions 2024-06-26T06:31:07.0913790Z Getting action download info 2024-06-26T06:31:07.2658603Z Download action repository 'pytorch/test-infra@main' (SHA:43a2ce341cc31288e9a38b65ce600a7f43021bd5) 2024-06-26T06:31:07.7230385Z Download action repository 'pytorch/pytorch@main' (SHA:000f2d637bf383567bd62289f12c868c3afea02d) 2024-06-26T06:31:11.5773411Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-06-26T06:31:11.7135240Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-06-26T06:31:12.0845247Z Getting action download info 2024-06-26T06:31:12.1768872Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-06-26T06:31:12.3752250Z Getting action download info 2024-06-26T06:31:12.5344098Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-06-26T06:31:12.6904774Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/129470/merge (4b51b1a62a63a1add1b3a0f7882f5c7dc66b8f8d) 2024-06-26T06:31:12.6908524Z ##[group] Inputs 2024-06-26T06:31:12.6909195Z build-environment: linux-focal-cuda11.8-py3.10-gcc9 2024-06-26T06:31:12.6911392Z test-matrix: {"include": [{"config": "distributed", "shard": 1, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 2, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 3, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}]} 2024-06-26T06:31:12.6914255Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:31:12.6915681Z sync-tag: 2024-06-26T06:31:12.6916958Z timeout-minutes: 360 2024-06-26T06:31:12.6917486Z use-gha: 2024-06-26T06:31:12.6917930Z dashboard-tag: 2024-06-26T06:31:12.6918379Z s3-bucket: gha-artifacts 2024-06-26T06:31:12.6918895Z aws-role-to-assume: 2024-06-26T06:31:12.6919397Z ##[endgroup] 2024-06-26T06:31:12.6920775Z Complete job name: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-06-26T06:31:12.7866607Z A job started hook has been configured by the self-hosted runner administrator 2024-06-26T06:31:12.8138433Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T06:31:12.8151603Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:31:12.8152355Z ##[endgroup] 2024-06-26T06:31:14.1418502Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-06-26T06:31:14.1419238Z with: 2024-06-26T06:31:14.1420165Z github-secret: *** 2024-06-26T06:31:14.1421428Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-06-26T06:31:14.1422951Z activate-with-label: false 2024-06-26T06:31:14.1423404Z label: with-ssh 2024-06-26T06:31:14.1423798Z remove-existing-keys: true 2024-06-26T06:31:14.1424236Z fail-silently: true 2024-06-26T06:31:14.1424631Z env: 2024-06-26T06:31:14.1424966Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:14.1425385Z ##[endgroup] 2024-06-26T06:31:14.2653857Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-06-26T06:31:14.5639062Z Grabbing public ssh keys from https://github.com/leslie-fang-intel.keys 2024-06-26T06:31:14.6299143Z No SSH keys found for user leslie-fang-intel 2024-06-26T06:31:14.6300091Z Grabbing public ssh keys from https://github.com/leslie-fang-intel.keys 2024-06-26T06:31:14.7019536Z No SSH keys found for user leslie-fang-intel 2024-06-26T06:31:14.7163460Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-06-26T06:31:14.7164245Z with: 2024-06-26T06:31:14.7164659Z submodules: recursive 2024-06-26T06:31:14.7165172Z fetch-depth: 0 2024-06-26T06:31:14.7165579Z env: 2024-06-26T06:31:14.7165988Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:14.7166491Z ##[endgroup] 2024-06-26T06:31:14.7437371Z ##[group]Run retry () { 2024-06-26T06:31:14.7437914Z retry () { 2024-06-26T06:31:14.7438614Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-06-26T06:31:14.7439387Z } 2024-06-26T06:31:14.7439819Z echo "${GITHUB_WORKSPACE}" 2024-06-26T06:31:14.7440418Z if [ -z "${NO_SUDO}" ]; then 2024-06-26T06:31:14.7441092Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T06:31:14.7441715Z else 2024-06-26T06:31:14.7442212Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T06:31:14.7442814Z fi 2024-06-26T06:31:14.7443308Z mkdir "${GITHUB_WORKSPACE}" 2024-06-26T06:31:14.7452406Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:31:14.7453123Z env: 2024-06-26T06:31:14.7453556Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:14.7454034Z NO_SUDO: 2024-06-26T06:31:14.7454440Z ##[endgroup] 2024-06-26T06:31:14.7482630Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T06:31:18.1054394Z ##[group]Run malfet/checkout@silent-checkout 2024-06-26T06:31:18.1054944Z with: 2024-06-26T06:31:18.1055332Z ref: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T06:31:18.1055880Z fetch-depth: 0 2024-06-26T06:31:18.1056266Z submodules: recursive 2024-06-26T06:31:18.1056672Z quiet-checkout: true 2024-06-26T06:31:18.1057102Z repository: pytorch/pytorch 2024-06-26T06:31:18.1057717Z token: *** 2024-06-26T06:31:18.1058057Z ssh-strict: true 2024-06-26T06:31:18.1058459Z persist-credentials: true 2024-06-26T06:31:18.1058895Z clean: true 2024-06-26T06:31:18.1059285Z sparse-checkout-cone-mode: true 2024-06-26T06:31:18.1059764Z lfs: false 2024-06-26T06:31:18.1060132Z set-safe-directory: true 2024-06-26T06:31:18.1060531Z env: 2024-06-26T06:31:18.1060861Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:18.1061297Z ##[endgroup] 2024-06-26T06:31:18.2379721Z Syncing repository: pytorch/pytorch 2024-06-26T06:31:18.2381910Z ##[group]Getting Git version info 2024-06-26T06:31:18.2382833Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T06:31:18.2384052Z [command]/usr/bin/git version 2024-06-26T06:31:18.2384408Z git version 2.40.1 2024-06-26T06:31:18.2388954Z ##[endgroup] 2024-06-26T06:31:18.2411440Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/8e6c2f5a-007e-4fcf-b0d0-24e0dd0e8b00' before making global git config changes 2024-06-26T06:31:18.2413625Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T06:31:18.2418966Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T06:31:18.2462352Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T06:31:18.2468435Z ##[group]Initializing the repository 2024-06-26T06:31:18.2472491Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T06:31:18.2504561Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-06-26T06:31:18.2505617Z hint: is subject to change. To configure the initial branch name to use in all 2024-06-26T06:31:18.2506722Z hint: of your new repositories, which will suppress this warning, call: 2024-06-26T06:31:18.2507625Z hint: 2024-06-26T06:31:18.2508264Z hint: git config --global init.defaultBranch 2024-06-26T06:31:18.2508779Z hint: 2024-06-26T06:31:18.2509495Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-06-26T06:31:18.2510630Z hint: 'development'. The just-created branch can be renamed via this command: 2024-06-26T06:31:18.2511647Z hint: 2024-06-26T06:31:18.2512152Z hint: git branch -m 2024-06-26T06:31:18.2513310Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-06-26T06:31:18.2517994Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-06-26T06:31:18.2550531Z ##[endgroup] 2024-06-26T06:31:18.2551204Z ##[group]Disabling automatic garbage collection 2024-06-26T06:31:18.2554611Z [command]/usr/bin/git config --local gc.auto 0 2024-06-26T06:31:18.2587109Z ##[endgroup] 2024-06-26T06:31:18.2587690Z ##[group]Setting up auth 2024-06-26T06:31:18.2596730Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T06:31:18.2631501Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T06:31:18.2920592Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T06:31:18.2949328Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T06:31:18.3232391Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T06:31:18.3281991Z ##[endgroup] 2024-06-26T06:31:18.3282770Z ##[group]Fetching the repository 2024-06-26T06:31:18.3290966Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-06-26T06:31:20.6307722Z remote: Enumerating objects: 987625 2024-06-26T06:31:20.6308379Z remote: Enumerating objects: 990184, done. 2024-06-26T06:31:20.6314888Z remote: Counting objects: 0% (1/2559) 2024-06-26T06:31:20.6315594Z remote: Counting objects: 1% (26/2559) 2024-06-26T06:31:20.6316195Z remote: Counting objects: 2% (52/2559) 2024-06-26T06:31:20.6316835Z remote: Counting objects: 3% (77/2559) 2024-06-26T06:31:20.6317450Z remote: Counting objects: 4% (103/2559) 2024-06-26T06:31:20.6318178Z remote: Counting objects: 5% (128/2559) 2024-06-26T06:31:20.6318809Z remote: Counting objects: 6% (154/2559) 2024-06-26T06:31:20.6319373Z remote: Counting objects: 7% (180/2559) 2024-06-26T06:31:20.6320001Z remote: Counting objects: 8% (205/2559) 2024-06-26T06:31:20.6320623Z remote: Counting objects: 9% (231/2559) 2024-06-26T06:31:20.6321295Z remote: Counting objects: 10% (256/2559) 2024-06-26T06:31:20.6321881Z remote: Counting objects: 11% (282/2559) 2024-06-26T06:31:20.6322554Z remote: Counting objects: 12% (308/2559) 2024-06-26T06:31:20.6323105Z remote: Counting objects: 13% (333/2559) 2024-06-26T06:31:20.6323742Z remote: Counting objects: 14% (359/2559) 2024-06-26T06:31:20.6324440Z remote: Counting objects: 15% (384/2559) 2024-06-26T06:31:20.6325041Z remote: Counting objects: 16% (410/2559) 2024-06-26T06:31:20.6325612Z remote: Counting objects: 17% (436/2559) 2024-06-26T06:31:20.6326600Z remote: Counting objects: 18% (461/2559) 2024-06-26T06:31:20.6327288Z remote: Counting objects: 19% (487/2559) 2024-06-26T06:31:20.6327894Z remote: Counting objects: 20% (512/2559) 2024-06-26T06:31:20.6328454Z remote: Counting objects: 21% (538/2559) 2024-06-26T06:31:20.6329107Z remote: Counting objects: 22% (563/2559) 2024-06-26T06:31:20.6329722Z remote: Counting objects: 23% (589/2559) 2024-06-26T06:31:20.6330330Z remote: Counting objects: 24% (615/2559) 2024-06-26T06:31:20.6330956Z remote: Counting objects: 25% (640/2559) 2024-06-26T06:31:20.6331621Z remote: Counting objects: 26% (666/2559) 2024-06-26T06:31:20.6332533Z remote: Counting objects: 27% (691/2559) 2024-06-26T06:31:20.6333305Z remote: Counting objects: 28% (717/2559) 2024-06-26T06:31:20.6333923Z remote: Counting objects: 29% (743/2559) 2024-06-26T06:31:20.6334569Z remote: Counting objects: 30% (768/2559) 2024-06-26T06:31:20.6335271Z remote: Counting objects: 31% (794/2559) 2024-06-26T06:31:20.6335881Z remote: Counting objects: 32% (819/2559) 2024-06-26T06:31:20.6336434Z remote: Counting objects: 33% (845/2559) 2024-06-26T06:31:20.6337084Z remote: Counting objects: 34% (871/2559) 2024-06-26T06:31:20.6337752Z remote: Counting objects: 35% (896/2559) 2024-06-26T06:31:20.6338330Z remote: Counting objects: 36% (922/2559) 2024-06-26T06:31:20.6338977Z remote: Counting objects: 37% (947/2559) 2024-06-26T06:31:20.6339645Z remote: Counting objects: 38% (973/2559) 2024-06-26T06:31:20.6340178Z remote: Counting objects: 39% (999/2559) 2024-06-26T06:31:20.6340845Z remote: Counting objects: 40% (1024/2559) 2024-06-26T06:31:20.6341493Z remote: Counting objects: 41% (1050/2559) 2024-06-26T06:31:20.6342135Z remote: Counting objects: 42% (1075/2559) 2024-06-26T06:31:20.6342851Z remote: Counting objects: 43% (1101/2559) 2024-06-26T06:31:20.6343516Z remote: Counting objects: 44% (1126/2559) 2024-06-26T06:31:20.6344074Z remote: Counting objects: 45% (1152/2559) 2024-06-26T06:31:20.6344721Z remote: Counting objects: 46% (1178/2559) 2024-06-26T06:31:20.6345372Z remote: Counting objects: 47% (1203/2559) 2024-06-26T06:31:20.6346010Z remote: Counting objects: 48% (1229/2559) 2024-06-26T06:31:20.6346696Z remote: Counting objects: 49% (1254/2559) 2024-06-26T06:31:20.6347290Z remote: Counting objects: 50% (1280/2559) 2024-06-26T06:31:20.6347936Z remote: Counting objects: 51% (1306/2559) 2024-06-26T06:31:20.6348628Z remote: Counting objects: 52% (1331/2559) 2024-06-26T06:31:20.6349211Z remote: Counting objects: 53% (1357/2559) 2024-06-26T06:31:20.6349900Z remote: Counting objects: 54% (1382/2559) 2024-06-26T06:31:20.6350508Z remote: Counting objects: 55% (1408/2559) 2024-06-26T06:31:20.6351148Z remote: Counting objects: 56% (1434/2559) 2024-06-26T06:31:20.6351798Z remote: Counting objects: 57% (1459/2559) 2024-06-26T06:31:20.6352424Z remote: Counting objects: 58% (1485/2559) 2024-06-26T06:31:20.6353068Z remote: Counting objects: 59% (1510/2559) 2024-06-26T06:31:20.6353732Z remote: Counting objects: 60% (1536/2559) 2024-06-26T06:31:20.6354342Z remote: Counting objects: 61% (1561/2559) 2024-06-26T06:31:20.6355026Z remote: Counting objects: 62% (1587/2559) 2024-06-26T06:31:20.6355673Z remote: Counting objects: 63% (1613/2559) 2024-06-26T06:31:20.6356300Z remote: Counting objects: 64% (1638/2559) 2024-06-26T06:31:20.6356941Z remote: Counting objects: 65% (1664/2559) 2024-06-26T06:31:20.6357575Z remote: Counting objects: 66% (1689/2559) 2024-06-26T06:31:20.6358251Z remote: Counting objects: 67% (1715/2559) 2024-06-26T06:31:20.6358865Z remote: Counting objects: 68% (1741/2559) 2024-06-26T06:31:20.6359657Z remote: Counting objects: 69% (1766/2559) 2024-06-26T06:31:20.6360301Z remote: Counting objects: 70% (1792/2559) 2024-06-26T06:31:20.6360941Z remote: Counting objects: 71% (1817/2559) 2024-06-26T06:31:20.6361564Z remote: Counting objects: 72% (1843/2559) 2024-06-26T06:31:20.6362209Z remote: Counting objects: 73% (1869/2559) 2024-06-26T06:31:20.6362854Z remote: Counting objects: 74% (1894/2559) 2024-06-26T06:31:20.6363498Z remote: Counting objects: 75% (1920/2559) 2024-06-26T06:31:20.6364122Z remote: Counting objects: 76% (1945/2559) 2024-06-26T06:31:20.6364761Z remote: Counting objects: 77% (1971/2559) 2024-06-26T06:31:20.6365405Z remote: Counting objects: 78% (1997/2559) 2024-06-26T06:31:20.6366121Z remote: Counting objects: 79% (2022/2559) 2024-06-26T06:31:20.6366778Z remote: Counting objects: 80% (2048/2559) 2024-06-26T06:31:20.6367422Z remote: Counting objects: 81% (2073/2559) 2024-06-26T06:31:20.6368056Z remote: Counting objects: 82% (2099/2559) 2024-06-26T06:31:20.6368696Z remote: Counting objects: 83% (2124/2559) 2024-06-26T06:31:20.6369340Z remote: Counting objects: 84% (2150/2559) 2024-06-26T06:31:20.6369960Z remote: Counting objects: 85% (2176/2559) 2024-06-26T06:31:20.6370597Z remote: Counting objects: 86% (2201/2559) 2024-06-26T06:31:20.6371236Z remote: Counting objects: 87% (2227/2559) 2024-06-26T06:31:20.6372049Z remote: Counting objects: 88% (2252/2559) 2024-06-26T06:31:20.6372694Z remote: Counting objects: 89% (2278/2559) 2024-06-26T06:31:20.6373333Z remote: Counting objects: 90% (2304/2559) 2024-06-26T06:31:20.6373956Z remote: Counting objects: 91% (2329/2559) 2024-06-26T06:31:20.6374609Z remote: Counting objects: 92% (2355/2559) 2024-06-26T06:31:20.6375253Z remote: Counting objects: 93% (2380/2559) 2024-06-26T06:31:20.6375894Z remote: Counting objects: 94% (2406/2559) 2024-06-26T06:31:20.6376532Z remote: Counting objects: 95% (2432/2559) 2024-06-26T06:31:20.6377171Z remote: Counting objects: 96% (2457/2559) 2024-06-26T06:31:20.6377811Z remote: Counting objects: 97% (2483/2559) 2024-06-26T06:31:20.6378435Z remote: Counting objects: 98% (2508/2559) 2024-06-26T06:31:20.6379076Z remote: Counting objects: 99% (2534/2559) 2024-06-26T06:31:20.6379713Z remote: Counting objects: 100% (2559/2559) 2024-06-26T06:31:20.6380384Z remote: Counting objects: 100% (2559/2559), done. 2024-06-26T06:31:20.6520561Z remote: Compressing objects: 0% (1/1092) 2024-06-26T06:31:20.6910672Z remote: Compressing objects: 1% (11/1092) 2024-06-26T06:31:20.7269196Z remote: Compressing objects: 2% (22/1092) 2024-06-26T06:31:20.7673748Z remote: Compressing objects: 3% (33/1092) 2024-06-26T06:31:20.7869718Z remote: Compressing objects: 4% (44/1092) 2024-06-26T06:31:20.8122933Z remote: Compressing objects: 5% (55/1092) 2024-06-26T06:31:20.8305506Z remote: Compressing objects: 6% (66/1092) 2024-06-26T06:31:20.8589201Z remote: Compressing objects: 7% (77/1092) 2024-06-26T06:31:20.9031270Z remote: Compressing objects: 8% (88/1092) 2024-06-26T06:31:20.9717807Z remote: Compressing objects: 9% (99/1092) 2024-06-26T06:31:21.0644662Z remote: Compressing objects: 10% (110/1092) 2024-06-26T06:31:21.1800954Z remote: Compressing objects: 11% (121/1092) 2024-06-26T06:31:21.2196288Z remote: Compressing objects: 12% (132/1092) 2024-06-26T06:31:21.2573955Z remote: Compressing objects: 13% (142/1092) 2024-06-26T06:31:21.2861429Z remote: Compressing objects: 14% (153/1092) 2024-06-26T06:31:21.2998389Z remote: Compressing objects: 15% (164/1092) 2024-06-26T06:31:21.3167181Z remote: Compressing objects: 16% (175/1092) 2024-06-26T06:31:21.3249005Z remote: Compressing objects: 17% (186/1092) 2024-06-26T06:31:21.3288077Z remote: Compressing objects: 18% (197/1092) 2024-06-26T06:31:21.3302184Z remote: Compressing objects: 19% (208/1092) 2024-06-26T06:31:21.3335708Z remote: Compressing objects: 20% (219/1092) 2024-06-26T06:31:21.3347255Z remote: Compressing objects: 21% (230/1092) 2024-06-26T06:31:21.3363046Z remote: Compressing objects: 22% (241/1092) 2024-06-26T06:31:21.3385470Z remote: Compressing objects: 23% (252/1092) 2024-06-26T06:31:21.3390070Z remote: Compressing objects: 24% (263/1092) 2024-06-26T06:31:21.3406903Z remote: Compressing objects: 25% (273/1092) 2024-06-26T06:31:21.3415770Z remote: Compressing objects: 26% (284/1092) 2024-06-26T06:31:21.3418747Z remote: Compressing objects: 27% (295/1092) 2024-06-26T06:31:21.3424087Z remote: Compressing objects: 28% (306/1092) 2024-06-26T06:31:21.3431472Z remote: Compressing objects: 29% (317/1092) 2024-06-26T06:31:21.3435767Z remote: Compressing objects: 30% (328/1092) 2024-06-26T06:31:21.3439422Z remote: Compressing objects: 31% (339/1092) 2024-06-26T06:31:21.3454385Z remote: Compressing objects: 32% (350/1092) 2024-06-26T06:31:21.3463998Z remote: Compressing objects: 33% (361/1092) 2024-06-26T06:31:21.3466608Z remote: Compressing objects: 34% (372/1092) 2024-06-26T06:31:21.3492370Z remote: Compressing objects: 35% (383/1092) 2024-06-26T06:31:21.3508278Z remote: Compressing objects: 36% (394/1092) 2024-06-26T06:31:21.3514563Z remote: Compressing objects: 37% (405/1092) 2024-06-26T06:31:21.3539215Z remote: Compressing objects: 38% (415/1092) 2024-06-26T06:31:21.3554872Z remote: Compressing objects: 39% (426/1092) 2024-06-26T06:31:21.3576737Z remote: Compressing objects: 40% (437/1092) 2024-06-26T06:31:21.3593737Z remote: Compressing objects: 41% (448/1092) 2024-06-26T06:31:21.3613532Z remote: Compressing objects: 42% (459/1092) 2024-06-26T06:31:21.3639225Z remote: Compressing objects: 43% (470/1092) 2024-06-26T06:31:21.3652456Z remote: Compressing objects: 44% (481/1092) 2024-06-26T06:31:21.3662057Z remote: Compressing objects: 45% (492/1092) 2024-06-26T06:31:21.3665130Z remote: Compressing objects: 46% (503/1092) 2024-06-26T06:31:21.3669946Z remote: Compressing objects: 47% (514/1092) 2024-06-26T06:31:21.3676535Z remote: Compressing objects: 48% (525/1092) 2024-06-26T06:31:21.3717442Z remote: Compressing objects: 49% (536/1092) 2024-06-26T06:31:21.3739364Z remote: Compressing objects: 50% (546/1092) 2024-06-26T06:31:21.3742986Z remote: Compressing objects: 51% (557/1092) 2024-06-26T06:31:21.3754949Z remote: Compressing objects: 52% (568/1092) 2024-06-26T06:31:21.3763307Z remote: Compressing objects: 53% (579/1092) 2024-06-26T06:31:21.3782559Z remote: Compressing objects: 54% (590/1092) 2024-06-26T06:31:21.3796063Z remote: Compressing objects: 55% (601/1092) 2024-06-26T06:31:21.3814205Z remote: Compressing objects: 56% (612/1092) 2024-06-26T06:31:21.3833669Z remote: Compressing objects: 57% (623/1092) 2024-06-26T06:31:21.3842490Z remote: Compressing objects: 58% (634/1092) 2024-06-26T06:31:21.3857705Z remote: Compressing objects: 59% (645/1092) 2024-06-26T06:31:21.3869077Z remote: Compressing objects: 60% (656/1092) 2024-06-26T06:31:21.3883901Z remote: Compressing objects: 61% (667/1092) 2024-06-26T06:31:21.3894079Z remote: Compressing objects: 62% (678/1092) 2024-06-26T06:31:21.3908287Z remote: Compressing objects: 63% (688/1092) 2024-06-26T06:31:21.3921913Z remote: Compressing objects: 64% (699/1092) 2024-06-26T06:31:21.3936927Z remote: Compressing objects: 65% (710/1092) 2024-06-26T06:31:21.3944092Z remote: Compressing objects: 66% (721/1092) 2024-06-26T06:31:21.3961771Z remote: Compressing objects: 67% (732/1092) 2024-06-26T06:31:21.3966106Z remote: Compressing objects: 68% (743/1092) 2024-06-26T06:31:21.3969899Z remote: Compressing objects: 69% (754/1092) 2024-06-26T06:31:21.3978443Z remote: Compressing objects: 70% (765/1092) 2024-06-26T06:31:21.3985484Z remote: Compressing objects: 71% (776/1092) 2024-06-26T06:31:21.3993672Z remote: Compressing objects: 72% (787/1092) 2024-06-26T06:31:21.3998253Z remote: Compressing objects: 73% (798/1092) 2024-06-26T06:31:21.4001436Z remote: Compressing objects: 74% (809/1092) 2024-06-26T06:31:21.4004436Z remote: Compressing objects: 75% (819/1092) 2024-06-26T06:31:21.4007134Z remote: Compressing objects: 76% (830/1092) 2024-06-26T06:31:21.4009274Z remote: Compressing objects: 77% (841/1092) 2024-06-26T06:31:21.4011592Z remote: Compressing objects: 78% (852/1092) 2024-06-26T06:31:21.4013162Z remote: Compressing objects: 79% (863/1092) 2024-06-26T06:31:21.4015495Z remote: Compressing objects: 80% (874/1092) 2024-06-26T06:31:21.4024303Z remote: Compressing objects: 81% (885/1092) 2024-06-26T06:31:21.4036985Z remote: Compressing objects: 82% (896/1092) 2024-06-26T06:31:21.4042397Z remote: Compressing objects: 83% (907/1092) 2024-06-26T06:31:21.4046159Z remote: Compressing objects: 84% (918/1092) 2024-06-26T06:31:21.4048708Z remote: Compressing objects: 85% (929/1092) 2024-06-26T06:31:21.4050535Z remote: Compressing objects: 86% (940/1092) 2024-06-26T06:31:21.4054860Z remote: Compressing objects: 87% (951/1092) 2024-06-26T06:31:21.4070221Z remote: Compressing objects: 88% (961/1092) 2024-06-26T06:31:21.4076023Z remote: Compressing objects: 89% (972/1092) 2024-06-26T06:31:21.4076737Z remote: Compressing objects: 90% (983/1092) 2024-06-26T06:31:21.4077373Z remote: Compressing objects: 91% (994/1092) 2024-06-26T06:31:21.4078051Z remote: Compressing objects: 92% (1005/1092) 2024-06-26T06:31:21.4078747Z remote: Compressing objects: 93% (1016/1092) 2024-06-26T06:31:21.4082804Z remote: Compressing objects: 94% (1027/1092) 2024-06-26T06:31:21.4083499Z remote: Compressing objects: 95% (1038/1092) 2024-06-26T06:31:21.4084184Z remote: Compressing objects: 96% (1049/1092) 2024-06-26T06:31:21.4085035Z remote: Compressing objects: 97% (1060/1092) 2024-06-26T06:31:21.4085698Z remote: Compressing objects: 98% (1071/1092) 2024-06-26T06:31:21.4086381Z remote: Compressing objects: 99% (1082/1092) 2024-06-26T06:31:21.4087098Z remote: Compressing objects: 100% (1092/1092) 2024-06-26T06:31:21.4087792Z remote: Compressing objects: 100% (1092/1092), done. 2024-06-26T06:31:48.2794690Z remote: Total 990184 (delta 2041), reused 1828 (delta 1466), pack-reused 987625 2024-06-26T06:32:10.7457268Z [command]/usr/bin/git rev-parse --verify --quiet b8c4c54d347aa776934c60784e35936878ef18dc^{object} 2024-06-26T06:32:10.7486077Z b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T06:32:10.7491373Z ##[endgroup] 2024-06-26T06:32:10.7492500Z ##[group]Determining the checkout info 2024-06-26T06:32:10.7496021Z ##[endgroup] 2024-06-26T06:32:10.7498880Z ##[group]Checking out the ref 2024-06-26T06:32:10.7499811Z [command]/usr/bin/git checkout --quiet --force b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T06:32:12.5550823Z ##[endgroup] 2024-06-26T06:32:12.5551568Z ##[group]Setting up auth for fetching submodules 2024-06-26T06:32:12.5557271Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T06:32:12.5611270Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-06-26T06:32:12.5643828Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-06-26T06:32:12.5676269Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-06-26T06:32:12.5706196Z ##[endgroup] 2024-06-26T06:32:12.5708302Z ##[group]Fetching submodules 2024-06-26T06:32:12.5711265Z [command]/usr/bin/git submodule sync --recursive 2024-06-26T06:32:12.6039657Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-06-26T06:32:12.6344569Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-06-26T06:32:12.6346641Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-06-26T06:32:12.6350142Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-06-26T06:32:12.6353301Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-06-26T06:32:12.6357126Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-06-26T06:32:12.6359827Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-06-26T06:32:12.6363765Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-06-26T06:32:12.6367720Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-06-26T06:32:12.6371543Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-06-26T06:32:12.6376604Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-06-26T06:32:12.6380296Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-06-26T06:32:12.6384597Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-06-26T06:32:12.6389011Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-06-26T06:32:12.6393818Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-06-26T06:32:12.6398278Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-06-26T06:32:12.6403138Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-06-26T06:32:12.6408352Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-06-26T06:32:12.6413898Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-06-26T06:32:12.6419031Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-06-26T06:32:12.6424221Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-06-26T06:32:12.6429790Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-06-26T06:32:12.6435463Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-06-26T06:32:12.6441928Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-06-26T06:32:12.6447780Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-06-26T06:32:12.6454528Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-06-26T06:32:12.6460422Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-06-26T06:32:12.6467151Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-06-26T06:32:12.6473240Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-06-26T06:32:12.6479800Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-06-26T06:32:12.6486455Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-06-26T06:32:12.6493545Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-06-26T06:32:12.6500869Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-06-26T06:32:12.6507846Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-06-26T06:32:12.6514805Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-06-26T06:32:12.6522376Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-06-26T06:32:12.6549791Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-06-26T06:32:12.9488716Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-06-26T06:32:13.1186746Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-06-26T06:32:13.2864869Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-06-26T06:32:13.6035975Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-06-26T06:32:15.8191337Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-06-26T06:32:26.6383807Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-06-26T06:32:27.0544607Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-06-26T06:32:27.5600580Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-06-26T06:32:28.1813086Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-06-26T06:32:29.5880955Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-06-26T06:32:31.7613650Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-06-26T06:32:38.8880622Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-06-26T06:32:40.4429249Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-06-26T06:32:42.2929325Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-06-26T06:32:43.6954558Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-06-26T06:32:43.9288887Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-06-26T06:32:44.3661643Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-06-26T06:32:44.9357694Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-06-26T06:32:46.7349460Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-06-26T06:32:47.1165584Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-06-26T06:32:47.3605771Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-06-26T06:32:49.1724115Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-06-26T06:32:49.9135234Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-06-26T06:32:50.6438153Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-06-26T06:32:57.4933097Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-06-26T06:33:00.0592983Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-06-26T06:33:06.6410463Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-06-26T06:33:06.8631271Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-06-26T06:33:16.4911400Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-06-26T06:33:16.6450071Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-06-26T06:33:16.8246022Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-06-26T06:33:17.9799409Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-06-26T06:33:18.2806038Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-06-26T06:33:18.9273875Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-06-26T06:33:19.3688219Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-06-26T06:33:19.3813378Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-06-26T06:33:19.3909918Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-06-26T06:33:19.4189761Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-06-26T06:33:19.4634897Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-06-26T06:33:20.6525884Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-06-26T06:33:20.6786512Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-06-26T06:33:20.7259612Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-06-26T06:33:20.8472604Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-06-26T06:33:20.8878153Z Submodule path 'third_party/cudnn_frontend': checked out 'aa3abd4bc689d6412979c7f55f9cd132848c9c6a' 2024-06-26T06:33:21.5445980Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-06-26T06:33:21.8609517Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-06-26T06:33:21.9523665Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-06-26T06:33:21.9541610Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-06-26T06:33:21.9544374Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T06:33:21.9547909Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-06-26T06:33:21.9551349Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-06-26T06:33:21.9555133Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T06:33:21.9582056Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-06-26T06:33:22.9117086Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-06-26T06:33:23.5884830Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-06-26T06:33:25.7238888Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-06-26T06:33:26.8233201Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-06-26T06:33:27.1883796Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-06-26T06:33:27.3057037Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-06-26T06:33:27.8272654Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-06-26T06:33:27.9028849Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-06-26T06:33:27.9163265Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-06-26T06:33:28.0517715Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-06-26T06:33:28.0923209Z Submodule path 'third_party/fmt': checked out 'e69e5f977d458f2650bb346dadf2ad30c5320281' 2024-06-26T06:33:28.1022837Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-06-26T06:33:28.1486330Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-06-26T06:33:28.1770024Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-06-26T06:33:28.2332627Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T06:33:28.2472399Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-06-26T06:33:28.2489499Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-06-26T06:33:28.2515232Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-06-26T06:33:42.2652545Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-06-26T06:33:42.2852776Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-06-26T06:33:42.3985878Z Submodule path 'third_party/kineto': checked out '8681ff11e1fa54da39023076c5c43eddd87b7a8a' 2024-06-26T06:33:42.4003906Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T06:33:42.4006497Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T06:33:42.4010200Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T06:33:42.4036998Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-06-26T06:33:42.9597068Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-06-26T06:33:44.2736756Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-06-26T06:33:45.4149565Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-06-26T06:33:45.4166059Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T06:33:45.4168903Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T06:33:45.4172815Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T06:33:45.4177241Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T06:33:45.4180381Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T06:33:45.4184693Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T06:33:45.4187890Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T06:33:45.4192122Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T06:33:45.4220135Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-06-26T06:33:46.7216160Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-06-26T06:33:47.0799575Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-06-26T06:33:48.4332215Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-06-26T06:33:48.7190977Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-06-26T06:33:49.2089410Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-06-26T06:33:50.3244088Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-06-26T06:33:57.2766571Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-06-26T06:33:57.7016544Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-06-26T06:33:57.7228280Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-06-26T06:33:57.7667277Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-06-26T06:33:57.7810965Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-06-26T06:33:57.7828091Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T06:33:57.7855107Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-06-26T06:33:58.0904401Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-06-26T06:33:58.1113795Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-06-26T06:33:58.1589787Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-06-26T06:33:58.2865589Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-06-26T06:33:58.3046053Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-06-26T06:33:58.3472485Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out 'a33701196adfad74917046096bf5a2aa0ab0bb50' 2024-06-26T06:33:58.4165635Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-06-26T06:33:58.4602198Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-06-26T06:33:58.4881061Z Submodule path 'third_party/nccl/nccl': checked out '48bb7fec7953112ff37499a272317f6663f8f600' 2024-06-26T06:33:58.6169538Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-06-26T06:33:59.0556220Z Submodule path 'third_party/onnx': checked out '990217f043af7222348ca8f0301e17fa7b841781' 2024-06-26T06:33:59.0592705Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-06-26T06:33:59.0595526Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-06-26T06:33:59.0623375Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-06-26T06:33:59.4544829Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-06-26T06:34:00.5668685Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-06-26T06:34:00.6070236Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-06-26T06:34:00.6870463Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-06-26T06:34:00.6890619Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T06:34:00.6895443Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T06:34:00.6897540Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T06:34:00.6900104Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T06:34:00.6904298Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T06:34:00.6907337Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T06:34:00.6911337Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T06:34:00.6914849Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T06:34:00.6941711Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-06-26T06:34:01.1436099Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-06-26T06:34:02.2446151Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-06-26T06:34:02.5620129Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-06-26T06:34:09.4888458Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-06-26T06:34:09.7507382Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-06-26T06:34:09.9801149Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-06-26T06:34:10.3229689Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-06-26T06:34:18.6929180Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-06-26T06:34:18.7412727Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-06-26T06:34:18.7587417Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-06-26T06:34:18.8831188Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-06-26T06:34:18.8971521Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-06-26T06:34:18.9131453Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-06-26T06:34:18.9304449Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-06-26T06:34:18.9320273Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T06:34:18.9323196Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T06:34:18.9350039Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-06-26T06:34:20.9159428Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-06-26T06:34:22.2902562Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-06-26T06:34:22.3456112Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T06:34:22.9093592Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-06-26T06:34:22.9217897Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-06-26T06:34:23.2465290Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-06-26T06:34:23.2488172Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-06-26T06:34:23.2491011Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-06-26T06:34:23.2518448Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-06-26T06:34:23.6754185Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-06-26T06:34:24.7510443Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-06-26T06:34:24.8379219Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-06-26T06:34:24.8472300Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-06-26T06:34:24.8606659Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-06-26T06:34:24.9026227Z Submodule path 'third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b' 2024-06-26T06:34:24.9361871Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-06-26T06:34:24.9868944Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-06-26T06:34:25.0171048Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-06-26T06:34:25.0188909Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-06-26T06:34:25.0191776Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-06-26T06:34:25.0195415Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-06-26T06:34:25.0198975Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T06:34:25.0224631Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-06-26T06:34:26.0805906Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-06-26T06:34:26.3300202Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-06-26T06:34:27.5895566Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-06-26T06:34:28.7191283Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-06-26T06:34:28.7360486Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-06-26T06:34:28.8205029Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-06-26T06:34:28.8539500Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-06-26T06:34:28.8555842Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T06:34:28.8582354Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-06-26T06:34:29.0375920Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-06-26T06:34:29.0412988Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-06-26T06:34:29.0729159Z Entering 'android/libs/fbjni' 2024-06-26T06:34:29.0770865Z Entering 'third_party/FP16' 2024-06-26T06:34:29.0812829Z Entering 'third_party/FXdiv' 2024-06-26T06:34:29.0856531Z Entering 'third_party/NNPACK' 2024-06-26T06:34:29.0897506Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T06:34:29.0939168Z Entering 'third_party/XNNPACK' 2024-06-26T06:34:29.0997688Z Entering 'third_party/benchmark' 2024-06-26T06:34:29.1040740Z Entering 'third_party/cpp-httplib' 2024-06-26T06:34:29.1082188Z Entering 'third_party/cpuinfo' 2024-06-26T06:34:29.1123317Z Entering 'third_party/cudnn_frontend' 2024-06-26T06:34:29.1165471Z Entering 'third_party/cutlass' 2024-06-26T06:34:29.1215997Z Entering 'third_party/eigen' 2024-06-26T06:34:29.1258887Z Entering 'third_party/fbgemm' 2024-06-26T06:34:29.1300578Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T06:34:29.1342055Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T06:34:29.1382162Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T06:34:29.1429282Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T06:34:29.1469833Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T06:34:29.1511295Z Entering 'third_party/flatbuffers' 2024-06-26T06:34:29.1555275Z Entering 'third_party/fmt' 2024-06-26T06:34:29.1596542Z Entering 'third_party/foxi' 2024-06-26T06:34:29.1637742Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T06:34:29.1678843Z Entering 'third_party/gloo' 2024-06-26T06:34:29.1720383Z Entering 'third_party/googletest' 2024-06-26T06:34:29.1760963Z Entering 'third_party/ideep' 2024-06-26T06:34:29.1801918Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T06:34:29.1849478Z Entering 'third_party/ittapi' 2024-06-26T06:34:29.1890090Z Entering 'third_party/kineto' 2024-06-26T06:34:29.1931689Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T06:34:29.1973305Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T06:34:29.2015645Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T06:34:29.2057769Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T06:34:29.2099254Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T06:34:29.2140737Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T06:34:29.2183816Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T06:34:29.2225117Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T06:34:29.2265810Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T06:34:29.2307909Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T06:34:29.2349950Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T06:34:29.2391318Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T06:34:29.2432865Z Entering 'third_party/mimalloc' 2024-06-26T06:34:29.2474942Z Entering 'third_party/nccl/nccl' 2024-06-26T06:34:29.2515968Z Entering 'third_party/nlohmann' 2024-06-26T06:34:29.2559214Z Entering 'third_party/onnx' 2024-06-26T06:34:29.2618489Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T06:34:29.2659104Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T06:34:29.2701845Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T06:34:29.2746750Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T06:34:29.2787211Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T06:34:29.2827430Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T06:34:29.2867031Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T06:34:29.2909840Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T06:34:29.2950107Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T06:34:29.2989965Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T06:34:29.3029311Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T06:34:29.3071410Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T06:34:29.3113830Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T06:34:29.3177730Z Entering 'third_party/pocketfft' 2024-06-26T06:34:29.3218488Z Entering 'third_party/protobuf' 2024-06-26T06:34:29.3262643Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T06:34:29.3302379Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T06:34:29.3344491Z Entering 'third_party/psimd' 2024-06-26T06:34:29.3385299Z Entering 'third_party/pthreadpool' 2024-06-26T06:34:29.3426275Z Entering 'third_party/pybind11' 2024-06-26T06:34:29.3467105Z Entering 'third_party/python-peachpy' 2024-06-26T06:34:29.3508612Z Entering 'third_party/sleef' 2024-06-26T06:34:29.3549660Z Entering 'third_party/tensorpipe' 2024-06-26T06:34:29.3591911Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T06:34:29.3632781Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T06:34:29.3673663Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T06:34:29.3714389Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T06:34:29.3753749Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T06:34:29.3809759Z ##[endgroup] 2024-06-26T06:34:29.3812814Z ##[group]Persisting credentials for submodules 2024-06-26T06:34:29.3817534Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-06-26T06:34:29.4128389Z Entering 'android/libs/fbjni' 2024-06-26T06:34:29.4180003Z Entering 'third_party/FP16' 2024-06-26T06:34:29.4230486Z Entering 'third_party/FXdiv' 2024-06-26T06:34:29.4281748Z Entering 'third_party/NNPACK' 2024-06-26T06:34:29.4333137Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T06:34:29.4384335Z Entering 'third_party/XNNPACK' 2024-06-26T06:34:29.4453867Z Entering 'third_party/benchmark' 2024-06-26T06:34:29.4504663Z Entering 'third_party/cpp-httplib' 2024-06-26T06:34:29.4555746Z Entering 'third_party/cpuinfo' 2024-06-26T06:34:29.4607473Z Entering 'third_party/cudnn_frontend' 2024-06-26T06:34:29.4659913Z Entering 'third_party/cutlass' 2024-06-26T06:34:29.4719560Z Entering 'third_party/eigen' 2024-06-26T06:34:29.4772388Z Entering 'third_party/fbgemm' 2024-06-26T06:34:29.4822963Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T06:34:29.4873381Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T06:34:29.4923483Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T06:34:29.4980833Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T06:34:29.5031264Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T06:34:29.5082182Z Entering 'third_party/flatbuffers' 2024-06-26T06:34:29.5135856Z Entering 'third_party/fmt' 2024-06-26T06:34:29.5186314Z Entering 'third_party/foxi' 2024-06-26T06:34:29.5237187Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T06:34:29.5288159Z Entering 'third_party/gloo' 2024-06-26T06:34:29.5339921Z Entering 'third_party/googletest' 2024-06-26T06:34:29.5391127Z Entering 'third_party/ideep' 2024-06-26T06:34:29.5440383Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T06:34:29.5498882Z Entering 'third_party/ittapi' 2024-06-26T06:34:29.5549660Z Entering 'third_party/kineto' 2024-06-26T06:34:29.5601313Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T06:34:29.5652021Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T06:34:29.5705281Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T06:34:29.5756042Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T06:34:29.5807445Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T06:34:29.5857970Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T06:34:29.5911374Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T06:34:29.5962252Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T06:34:29.6012706Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T06:34:29.6066289Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T06:34:29.6118135Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T06:34:29.6168409Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T06:34:29.6219824Z Entering 'third_party/mimalloc' 2024-06-26T06:34:29.6271318Z Entering 'third_party/nccl/nccl' 2024-06-26T06:34:29.6321781Z Entering 'third_party/nlohmann' 2024-06-26T06:34:29.6374867Z Entering 'third_party/onnx' 2024-06-26T06:34:29.6443694Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T06:34:29.6494411Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T06:34:29.6547764Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T06:34:29.6600220Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T06:34:29.6650185Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T06:34:29.6701812Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T06:34:29.6752009Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T06:34:29.6803576Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T06:34:29.6853784Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T06:34:29.6904235Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T06:34:29.6952886Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T06:34:29.7005570Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T06:34:29.7057774Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T06:34:29.7129715Z Entering 'third_party/pocketfft' 2024-06-26T06:34:29.7180270Z Entering 'third_party/protobuf' 2024-06-26T06:34:29.7234099Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T06:34:29.7282765Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T06:34:29.7335398Z Entering 'third_party/psimd' 2024-06-26T06:34:29.7386468Z Entering 'third_party/pthreadpool' 2024-06-26T06:34:29.7436699Z Entering 'third_party/pybind11' 2024-06-26T06:34:29.7487488Z Entering 'third_party/python-peachpy' 2024-06-26T06:34:29.7538026Z Entering 'third_party/sleef' 2024-06-26T06:34:29.7588370Z Entering 'third_party/tensorpipe' 2024-06-26T06:34:29.7640626Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T06:34:29.7691284Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T06:34:29.7742740Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T06:34:29.7793294Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T06:34:29.7842206Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T06:34:29.7910340Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-06-26T06:34:29.8218848Z Entering 'android/libs/fbjni' 2024-06-26T06:34:29.8266976Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-06-26T06:34:29.8285446Z Entering 'third_party/FP16' 2024-06-26T06:34:29.8333558Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-06-26T06:34:29.8351947Z Entering 'third_party/FXdiv' 2024-06-26T06:34:29.8399812Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-06-26T06:34:29.8418371Z Entering 'third_party/NNPACK' 2024-06-26T06:34:29.8465819Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-06-26T06:34:29.8484096Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T06:34:29.8533747Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-06-26T06:34:29.8552389Z Entering 'third_party/XNNPACK' 2024-06-26T06:34:29.8600489Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-06-26T06:34:29.8637198Z Entering 'third_party/benchmark' 2024-06-26T06:34:29.8685311Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-06-26T06:34:29.8704085Z Entering 'third_party/cpp-httplib' 2024-06-26T06:34:29.8753525Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-06-26T06:34:29.8771346Z Entering 'third_party/cpuinfo' 2024-06-26T06:34:29.8820053Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T06:34:29.8839065Z Entering 'third_party/cudnn_frontend' 2024-06-26T06:34:29.8886986Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-06-26T06:34:29.8906628Z Entering 'third_party/cutlass' 2024-06-26T06:34:29.8954931Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-06-26T06:34:29.8981917Z Entering 'third_party/eigen' 2024-06-26T06:34:29.9029974Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-06-26T06:34:29.9050264Z Entering 'third_party/fbgemm' 2024-06-26T06:34:29.9100620Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-06-26T06:34:29.9118789Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T06:34:29.9166928Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-06-26T06:34:29.9184988Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T06:34:29.9232320Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T06:34:29.9250470Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T06:34:29.9299384Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-06-26T06:34:29.9323877Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T06:34:29.9370991Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-06-26T06:34:29.9389150Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T06:34:29.9436450Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-06-26T06:34:29.9455622Z Entering 'third_party/flatbuffers' 2024-06-26T06:34:29.9503003Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-06-26T06:34:29.9524931Z Entering 'third_party/fmt' 2024-06-26T06:34:29.9573377Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-06-26T06:34:29.9591982Z Entering 'third_party/foxi' 2024-06-26T06:34:29.9639483Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-06-26T06:34:29.9658264Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T06:34:29.9707390Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-06-26T06:34:29.9725666Z Entering 'third_party/gloo' 2024-06-26T06:34:29.9773197Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-06-26T06:34:29.9792075Z Entering 'third_party/googletest' 2024-06-26T06:34:29.9839357Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-06-26T06:34:29.9858053Z Entering 'third_party/ideep' 2024-06-26T06:34:29.9906613Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-06-26T06:34:29.9923317Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T06:34:29.9970030Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-06-26T06:34:29.9995603Z Entering 'third_party/ittapi' 2024-06-26T06:34:30.0043101Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-06-26T06:34:30.0061654Z Entering 'third_party/kineto' 2024-06-26T06:34:30.0110369Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-06-26T06:34:30.0128055Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T06:34:30.0176659Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-06-26T06:34:30.0194190Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T06:34:30.0242549Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-06-26T06:34:30.0263455Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T06:34:30.0311813Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-06-26T06:34:30.0329552Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T06:34:30.0378130Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-06-26T06:34:30.0396656Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T06:34:30.0445250Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-06-26T06:34:30.0462044Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T06:34:30.0509619Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-06-26T06:34:30.0529174Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T06:34:30.0578741Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-06-26T06:34:30.0596424Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T06:34:30.0645057Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-06-26T06:34:30.0664874Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T06:34:30.0712061Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-06-26T06:34:30.0731161Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T06:34:30.0780340Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-06-26T06:34:30.0799670Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T06:34:30.0848071Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-06-26T06:34:30.0866428Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T06:34:30.0913430Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-06-26T06:34:30.0932918Z Entering 'third_party/mimalloc' 2024-06-26T06:34:30.0981329Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-06-26T06:34:30.1000184Z Entering 'third_party/nccl/nccl' 2024-06-26T06:34:30.1048123Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-06-26T06:34:30.1066988Z Entering 'third_party/nlohmann' 2024-06-26T06:34:30.1115046Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-06-26T06:34:30.1135438Z Entering 'third_party/onnx' 2024-06-26T06:34:30.1183000Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-06-26T06:34:30.1220862Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T06:34:30.1268565Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-06-26T06:34:30.1286906Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T06:34:30.1335022Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-06-26T06:34:30.1355892Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T06:34:30.1404223Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-06-26T06:34:30.1424919Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T06:34:30.1471826Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-06-26T06:34:30.1489539Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T06:34:30.1536931Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-06-26T06:34:30.1555322Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T06:34:30.1602205Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-06-26T06:34:30.1619885Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T06:34:30.1666915Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-06-26T06:34:30.1686151Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T06:34:30.1733488Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-06-26T06:34:30.1752495Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T06:34:30.1799099Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-06-26T06:34:30.1817670Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T06:34:30.1864315Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-06-26T06:34:30.1880935Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T06:34:30.1928167Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-06-26T06:34:30.1948786Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T06:34:30.1995970Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-06-26T06:34:30.2015589Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T06:34:30.2062342Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-06-26T06:34:30.2102458Z Entering 'third_party/pocketfft' 2024-06-26T06:34:30.2150632Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-06-26T06:34:30.2169009Z Entering 'third_party/protobuf' 2024-06-26T06:34:30.2217540Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-06-26T06:34:30.2238571Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T06:34:30.2287863Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-06-26T06:34:30.2305871Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T06:34:30.2352534Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-06-26T06:34:30.2372202Z Entering 'third_party/psimd' 2024-06-26T06:34:30.2421243Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-06-26T06:34:30.2439059Z Entering 'third_party/pthreadpool' 2024-06-26T06:34:30.2487496Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-06-26T06:34:30.2505639Z Entering 'third_party/pybind11' 2024-06-26T06:34:30.2553208Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-06-26T06:34:30.2572050Z Entering 'third_party/python-peachpy' 2024-06-26T06:34:30.2619619Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-06-26T06:34:30.2637141Z Entering 'third_party/sleef' 2024-06-26T06:34:30.2684583Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-06-26T06:34:30.2703595Z Entering 'third_party/tensorpipe' 2024-06-26T06:34:30.2751438Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-06-26T06:34:30.2769027Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T06:34:30.2816965Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-06-26T06:34:30.2834309Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T06:34:30.2882692Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-06-26T06:34:30.2901768Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T06:34:30.2949415Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-06-26T06:34:30.2967700Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T06:34:30.3015349Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-06-26T06:34:30.3031901Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T06:34:30.3080162Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-06-26T06:34:30.4105626Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-06-26T06:34:30.4416294Z Entering 'android/libs/fbjni' 2024-06-26T06:34:30.4458362Z Entering 'third_party/FP16' 2024-06-26T06:34:30.4500544Z Entering 'third_party/FXdiv' 2024-06-26T06:34:30.4542739Z Entering 'third_party/NNPACK' 2024-06-26T06:34:30.4585731Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T06:34:30.4627144Z Entering 'third_party/XNNPACK' 2024-06-26T06:34:30.4688171Z Entering 'third_party/benchmark' 2024-06-26T06:34:30.4729517Z Entering 'third_party/cpp-httplib' 2024-06-26T06:34:30.4770344Z Entering 'third_party/cpuinfo' 2024-06-26T06:34:30.4812860Z Entering 'third_party/cudnn_frontend' 2024-06-26T06:34:30.4855182Z Entering 'third_party/cutlass' 2024-06-26T06:34:30.4905482Z Entering 'third_party/eigen' 2024-06-26T06:34:30.4949403Z Entering 'third_party/fbgemm' 2024-06-26T06:34:30.4991494Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T06:34:30.5032194Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T06:34:30.5074012Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T06:34:30.5122009Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T06:34:30.5162339Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T06:34:30.5205102Z Entering 'third_party/flatbuffers' 2024-06-26T06:34:30.5249560Z Entering 'third_party/fmt' 2024-06-26T06:34:30.5290986Z Entering 'third_party/foxi' 2024-06-26T06:34:30.5332979Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T06:34:30.5375321Z Entering 'third_party/gloo' 2024-06-26T06:34:30.5417755Z Entering 'third_party/googletest' 2024-06-26T06:34:30.5460230Z Entering 'third_party/ideep' 2024-06-26T06:34:30.5501307Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T06:34:30.5548970Z Entering 'third_party/ittapi' 2024-06-26T06:34:30.5590308Z Entering 'third_party/kineto' 2024-06-26T06:34:30.5633414Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T06:34:30.5674974Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T06:34:30.5718309Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T06:34:30.5758924Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T06:34:30.5800781Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T06:34:30.5840672Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T06:34:30.5882678Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T06:34:30.5923833Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T06:34:30.5966323Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T06:34:30.6009200Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T06:34:30.6052398Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T06:34:30.6093449Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T06:34:30.6135583Z Entering 'third_party/mimalloc' 2024-06-26T06:34:30.6177397Z Entering 'third_party/nccl/nccl' 2024-06-26T06:34:30.6218640Z Entering 'third_party/nlohmann' 2024-06-26T06:34:30.6261096Z Entering 'third_party/onnx' 2024-06-26T06:34:30.6319509Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T06:34:30.6362354Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T06:34:30.6405562Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T06:34:30.6449611Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T06:34:30.6490139Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T06:34:30.6529892Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T06:34:30.6570355Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T06:34:30.6611879Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T06:34:30.6651507Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T06:34:30.6691188Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T06:34:30.6730489Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T06:34:30.6775245Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T06:34:30.6817132Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T06:34:30.6879276Z Entering 'third_party/pocketfft' 2024-06-26T06:34:30.6920089Z Entering 'third_party/protobuf' 2024-06-26T06:34:30.6964618Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T06:34:30.7005698Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T06:34:30.7047486Z Entering 'third_party/psimd' 2024-06-26T06:34:30.7088818Z Entering 'third_party/pthreadpool' 2024-06-26T06:34:30.7130263Z Entering 'third_party/pybind11' 2024-06-26T06:34:30.7171686Z Entering 'third_party/python-peachpy' 2024-06-26T06:34:30.7213799Z Entering 'third_party/sleef' 2024-06-26T06:34:30.7254936Z Entering 'third_party/tensorpipe' 2024-06-26T06:34:30.7296854Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T06:34:30.7337426Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T06:34:30.7378254Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T06:34:30.7418309Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T06:34:30.7460092Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T06:34:30.7518459Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-06-26T06:34:30.7831369Z Entering 'android/libs/fbjni' 2024-06-26T06:34:30.7877141Z Entering 'third_party/FP16' 2024-06-26T06:34:30.7920627Z Entering 'third_party/FXdiv' 2024-06-26T06:34:30.7962950Z Entering 'third_party/NNPACK' 2024-06-26T06:34:30.8005032Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T06:34:30.8047672Z Entering 'third_party/XNNPACK' 2024-06-26T06:34:30.8109480Z Entering 'third_party/benchmark' 2024-06-26T06:34:30.8151500Z Entering 'third_party/cpp-httplib' 2024-06-26T06:34:30.8194120Z Entering 'third_party/cpuinfo' 2024-06-26T06:34:30.8237175Z Entering 'third_party/cudnn_frontend' 2024-06-26T06:34:30.8279465Z Entering 'third_party/cutlass' 2024-06-26T06:34:30.8329231Z Entering 'third_party/eigen' 2024-06-26T06:34:30.8373548Z Entering 'third_party/fbgemm' 2024-06-26T06:34:30.8415381Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T06:34:30.8457906Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T06:34:30.8501089Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T06:34:30.8549242Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T06:34:30.8590933Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T06:34:30.8632633Z Entering 'third_party/flatbuffers' 2024-06-26T06:34:30.8678145Z Entering 'third_party/fmt' 2024-06-26T06:34:30.8720437Z Entering 'third_party/foxi' 2024-06-26T06:34:30.8762809Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T06:34:30.8805135Z Entering 'third_party/gloo' 2024-06-26T06:34:30.8846586Z Entering 'third_party/googletest' 2024-06-26T06:34:30.8889460Z Entering 'third_party/ideep' 2024-06-26T06:34:30.8931489Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T06:34:30.8980983Z Entering 'third_party/ittapi' 2024-06-26T06:34:30.9023137Z Entering 'third_party/kineto' 2024-06-26T06:34:30.9065588Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T06:34:30.9106638Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T06:34:30.9148926Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T06:34:30.9189868Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T06:34:30.9231692Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T06:34:30.9271949Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T06:34:30.9313960Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T06:34:30.9355185Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T06:34:30.9395879Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T06:34:30.9438941Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T06:34:30.9480378Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T06:34:30.9521229Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T06:34:30.9562779Z Entering 'third_party/mimalloc' 2024-06-26T06:34:30.9604669Z Entering 'third_party/nccl/nccl' 2024-06-26T06:34:30.9648052Z Entering 'third_party/nlohmann' 2024-06-26T06:34:30.9690738Z Entering 'third_party/onnx' 2024-06-26T06:34:30.9751391Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T06:34:30.9792928Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T06:34:30.9836734Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T06:34:30.9881744Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T06:34:30.9923481Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T06:34:30.9966205Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T06:34:31.0006746Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T06:34:31.0049733Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T06:34:31.0089333Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T06:34:31.0129170Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T06:34:31.0168500Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T06:34:31.0212278Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T06:34:31.0254937Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T06:34:31.0317730Z Entering 'third_party/pocketfft' 2024-06-26T06:34:31.0358543Z Entering 'third_party/protobuf' 2024-06-26T06:34:31.0404144Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T06:34:31.0444703Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T06:34:31.0487801Z Entering 'third_party/psimd' 2024-06-26T06:34:31.0529599Z Entering 'third_party/pthreadpool' 2024-06-26T06:34:31.0571604Z Entering 'third_party/pybind11' 2024-06-26T06:34:31.0613651Z Entering 'third_party/python-peachpy' 2024-06-26T06:34:31.0654852Z Entering 'third_party/sleef' 2024-06-26T06:34:31.0696285Z Entering 'third_party/tensorpipe' 2024-06-26T06:34:31.0736916Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T06:34:31.0777379Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T06:34:31.0817158Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T06:34:31.0858114Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T06:34:31.0897589Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T06:34:31.0952193Z ##[endgroup] 2024-06-26T06:34:31.0994780Z [command]/usr/bin/git log -1 --format='%H' 2024-06-26T06:34:31.1023063Z 'b8c4c54d347aa776934c60784e35936878ef18dc' 2024-06-26T06:34:31.1253196Z Prepare all required actions 2024-06-26T06:34:31.1253801Z Getting action download info 2024-06-26T06:34:31.2671764Z ##[group]Run ./.github/actions/setup-linux 2024-06-26T06:34:31.2672293Z env: 2024-06-26T06:34:31.2672631Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:31.2673046Z ##[endgroup] 2024-06-26T06:34:31.2755019Z ##[group]Run set -euo pipefail 2024-06-26T06:34:31.2755557Z set -euo pipefail 2024-06-26T06:34:31.2756030Z function get_ec2_metadata() { 2024-06-26T06:34:31.2756699Z  # Pulled from instance metadata endpoint for EC2 2024-06-26T06:34:31.2758012Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-06-26T06:34:31.2758983Z  category=$1 2024-06-26T06:34:31.2759646Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-06-26T06:34:31.2760446Z  runner_name_str=i-011558b4ba4829615 2024-06-26T06:34:31.2761065Z  if [[ -f /.inarc ]]; then 2024-06-26T06:34:31.2761687Z  echo "ARC Runner, no info on ec2 metadata" 2024-06-26T06:34:31.2762388Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-06-26T06:34:31.2763264Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-06-26T06:34:31.2764030Z  else 2024-06-26T06:34:31.2764696Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-06-26T06:34:31.2765359Z  fi 2024-06-26T06:34:31.2765712Z } 2024-06-26T06:34:31.2766167Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-06-26T06:34:31.2766877Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-06-26T06:34:31.2767756Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-06-26T06:34:31.2768491Z echo "system info $(uname -a)" 2024-06-26T06:34:31.2778381Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:34:31.2779402Z env: 2024-06-26T06:34:31.2779763Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:31.2780199Z ##[endgroup] 2024-06-26T06:34:31.2882508Z ami-id: ami-0ce0c36d7a00b20e2 2024-06-26T06:34:31.2950040Z instance-id: i-011558b4ba4829615 2024-06-26T06:34:31.3017907Z instance-type: g3.8xlarge 2024-06-26T06:34:31.3026695Z system info Linux ip-10-0-13-167.ec2.internal 4.14.336-257.562.amzn2.x86_64 #1 SMP Sat Feb 24 09:50:35 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-06-26T06:34:31.3078221Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T06:34:31.3079379Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T06:34:31.3087527Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:34:31.3088158Z env: 2024-06-26T06:34:31.3088494Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:31.3088913Z ##[endgroup] 2024-06-26T06:34:31.3168889Z ##[group]Run if systemctl is-active --quiet docker; then 2024-06-26T06:34:31.3169642Z if systemctl is-active --quiet docker; then 2024-06-26T06:34:31.3170315Z  echo "Docker daemon is running..."; 2024-06-26T06:34:31.3170874Z else 2024-06-26T06:34:31.3171467Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-06-26T06:34:31.3172489Z fi 2024-06-26T06:34:31.3180012Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:34:31.3180614Z env: 2024-06-26T06:34:31.3180951Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:31.3181381Z ##[endgroup] 2024-06-26T06:34:31.3226736Z Docker daemon is running... 2024-06-26T06:34:31.3283786Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T06:34:31.3284495Z with: 2024-06-26T06:34:31.3284823Z shell: bash 2024-06-26T06:34:31.3285173Z timeout_minutes: 5 2024-06-26T06:34:31.3285572Z max_attempts: 3 2024-06-26T06:34:31.3285954Z retry_wait_seconds: 30 2024-06-26T06:34:31.3287910Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-06-26T06:34:31.3289784Z polling_interval_seconds: 1 2024-06-26T06:34:31.3290250Z warning_on_retry: true 2024-06-26T06:34:31.3290685Z continue_on_error: false 2024-06-26T06:34:31.3291082Z env: 2024-06-26T06:34:31.3291417Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:31.3292149Z AWS_RETRY_MODE: standard 2024-06-26T06:34:31.3292550Z AWS_MAX_ATTEMPTS: 5 2024-06-26T06:34:31.3292972Z AWS_DEFAULT_REGION: us-east-1 2024-06-26T06:34:31.3293599Z ##[endgroup] 2024-06-26T06:34:32.3449121Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T06:34:32.3450131Z Configure a credential helper to remove this warning. See 2024-06-26T06:34:32.3451298Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T06:34:32.3452051Z 2024-06-26T06:34:32.3452248Z Login Succeeded 2024-06-26T06:34:32.4002366Z Command completed after 1 attempt(s). 2024-06-26T06:34:32.4058426Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T06:34:32.4059336Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T06:34:32.4060163Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T06:34:32.4068594Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:34:32.4069229Z env: 2024-06-26T06:34:32.4069565Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:32.4069981Z ##[endgroup] 2024-06-26T06:34:32.4155235Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T06:34:32.4156207Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T06:34:32.4156949Z # shellcheck disable=SC2046 2024-06-26T06:34:32.4157519Z docker stop $(docker ps -q) || true 2024-06-26T06:34:32.4158108Z # Prune all of the docker images 2024-06-26T06:34:32.4158672Z docker system prune -af 2024-06-26T06:34:32.4166465Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:34:32.4167069Z env: 2024-06-26T06:34:32.4167408Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:32.4167841Z ##[endgroup] 2024-06-26T06:34:32.4506065Z "docker stop" requires at least 1 argument. 2024-06-26T06:34:32.4506853Z See 'docker stop --help'. 2024-06-26T06:34:32.4507158Z 2024-06-26T06:34:32.4507439Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-06-26T06:34:32.4507914Z 2024-06-26T06:34:32.4508168Z Stop one or more running containers 2024-06-26T06:34:32.4690786Z Total reclaimed space: 0B 2024-06-26T06:34:32.4731334Z ##[group]Run set +e 2024-06-26T06:34:32.4732277Z set +e 2024-06-26T06:34:32.4732644Z set -x 2024-06-26T06:34:32.4732998Z  2024-06-26T06:34:32.4733388Z PT_DOMAIN=download.pytorch.org 2024-06-26T06:34:32.4734377Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-06-26T06:34:32.4735753Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-06-26T06:34:32.4736703Z # one is returned at random 2024-06-26T06:34:32.4737357Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-06-26T06:34:32.4737991Z  2024-06-26T06:34:32.4738368Z if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T06:34:32.4739121Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-06-26T06:34:32.4740077Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-06-26T06:34:32.4740782Z  2024-06-26T06:34:32.4741175Z  if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T06:34:32.4741844Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-06-26T06:34:32.4742479Z  exit 1 2024-06-26T06:34:32.4742855Z  fi 2024-06-26T06:34:32.4743181Z fi 2024-06-26T06:34:32.4743701Z  2024-06-26T06:34:32.4744167Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-06-26T06:34:32.4744792Z  # Clean up any old records first 2024-06-26T06:34:32.4745421Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-06-26T06:34:32.4745982Z fi 2024-06-26T06:34:32.4746297Z  2024-06-26T06:34:32.4746817Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-06-26T06:34:32.4747498Z cat /etc/hosts 2024-06-26T06:34:32.4755547Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:34:32.4756335Z env: 2024-06-26T06:34:32.4756673Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:32.4757084Z ##[endgroup] 2024-06-26T06:34:32.4783950Z + PT_DOMAIN=download.pytorch.org 2024-06-26T06:34:32.4789425Z ++ dig -4 +short download.pytorch.org 2024-06-26T06:34:32.4789985Z ++ tail -n1 2024-06-26T06:34:32.4955606Z + RESOLVED_IP=18.160.18.37 2024-06-26T06:34:32.4956110Z + '[' -z 18.160.18.37 ']' 2024-06-26T06:34:32.4956730Z + grep -r download.pytorch.org /etc/hosts 2024-06-26T06:34:32.4965236Z 18.67.76.37 download.pytorch.org 2024-06-26T06:34:32.4966114Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-06-26T06:34:32.5111189Z + echo '18.160.18.37 download.pytorch.org' 2024-06-26T06:34:32.5111762Z + sudo tee -a /etc/hosts 2024-06-26T06:34:32.5429818Z 18.160.18.37 download.pytorch.org 2024-06-26T06:34:32.5446126Z + cat /etc/hosts 2024-06-26T06:34:32.5452966Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-06-26T06:34:32.5468008Z ::1 localhost6 localhost6.localdomain6 2024-06-26T06:34:32.5468708Z 18.160.18.37 download.pytorch.org 2024-06-26T06:34:32.5671149Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-06-26T06:34:32.5671942Z with: 2024-06-26T06:34:32.5673155Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:32.5674560Z docker-build-dir: .ci/docker 2024-06-26T06:34:32.5675028Z working-directory: . 2024-06-26T06:34:32.5675595Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T06:34:32.5676264Z force-push: false 2024-06-26T06:34:32.5676630Z env: 2024-06-26T06:34:32.5676941Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:32.5677368Z ##[endgroup] 2024-06-26T06:34:32.5703745Z ##[group]Run set -ex 2024-06-26T06:34:32.5704182Z set -ex 2024-06-26T06:34:32.5704544Z  2024-06-26T06:34:32.5705231Z # If the docker build directory or the build script doesn't exist, the action will 2024-06-26T06:34:32.5706558Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-06-26T06:34:32.5707573Z # job could then download the pre-built image as usual 2024-06-26T06:34:32.5708511Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-06-26T06:34:32.5709356Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-06-26T06:34:32.5710153Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T06:34:32.5710840Z  2024-06-26T06:34:32.5711478Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-06-26T06:34:32.5712264Z  exit 0 2024-06-26T06:34:32.5712616Z else 2024-06-26T06:34:32.5713061Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-06-26T06:34:32.5713620Z fi 2024-06-26T06:34:32.5713933Z  2024-06-26T06:34:32.5714510Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-06-26T06:34:32.5715612Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-06-26T06:34:32.5716583Z  # use it as it is, but first let's extract the tag 2024-06-26T06:34:32.5717471Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-06-26T06:34:32.5718387Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T06:34:32.5719269Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T06:34:32.5719957Z else 2024-06-26T06:34:32.5720494Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-06-26T06:34:32.5721327Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T06:34:32.5722449Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T06:34:32.5723585Z fi 2024-06-26T06:34:32.5731076Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:34:32.5731991Z env: 2024-06-26T06:34:32.5732274Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:32.5732714Z REPO_NAME: pytorch 2024-06-26T06:34:32.5733970Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:32.5735352Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T06:34:32.5735967Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T06:34:32.5736611Z ##[endgroup] 2024-06-26T06:34:32.5761600Z + [[ ! -d .ci/docker ]] 2024-06-26T06:34:32.5762203Z + [[ ! -f .ci/docker/build.sh ]] 2024-06-26T06:34:32.5763000Z + echo skip=false 2024-06-26T06:34:32.5765151Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-06-26T06:34:32.5769748Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:32.5771148Z ++ awk -F '[:,]' '{print $2}' 2024-06-26T06:34:32.5780151Z + DOCKER_TAG=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:32.5781063Z + echo docker-tag=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:32.5784138Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:32.5818794Z ##[group]Run set +e 2024-06-26T06:34:32.5819232Z set +e 2024-06-26T06:34:32.5819595Z set -x 2024-06-26T06:34:32.5819930Z  2024-06-26T06:34:32.5820261Z login() { 2024-06-26T06:34:32.5821087Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T06:34:32.5822020Z } 2024-06-26T06:34:32.5822333Z  2024-06-26T06:34:32.5822676Z retry () { 2024-06-26T06:34:32.5823140Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T06:34:32.5823691Z } 2024-06-26T06:34:32.5824024Z  2024-06-26T06:34:32.5824389Z retry login "${DOCKER_REGISTRY}" 2024-06-26T06:34:32.5824899Z  2024-06-26T06:34:32.5825481Z # Check if image already exists, if it does then skip building it 2024-06-26T06:34:32.5826374Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-06-26T06:34:32.5826985Z  exit 0 2024-06-26T06:34:32.5827352Z fi 2024-06-26T06:34:32.5827679Z  2024-06-26T06:34:32.5828280Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-06-26T06:34:32.5829329Z # be empty. The default action would be to continue rebuild the image 2024-06-26T06:34:32.5830258Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-06-26T06:34:32.5831099Z  # if we're on the base branch then use the parent commit 2024-06-26T06:34:32.5831844Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-06-26T06:34:32.5832383Z else 2024-06-26T06:34:32.5832944Z  # otherwise we're on a PR, so use the most recent base commit 2024-06-26T06:34:32.5833808Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-06-26T06:34:32.5834434Z fi 2024-06-26T06:34:32.5834758Z  2024-06-26T06:34:32.5835140Z if [[ -z "${MERGE_BASE}" ]]; then 2024-06-26T06:34:32.5835767Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T06:34:32.5836333Z  2024-06-26T06:34:32.5837133Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-06-26T06:34:32.5838098Z  exit 0 2024-06-26T06:34:32.5838463Z fi 2024-06-26T06:34:32.5838779Z  2024-06-26T06:34:32.5839469Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-06-26T06:34:32.5840713Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-06-26T06:34:32.5841721Z  exit 1 2024-06-26T06:34:32.5842088Z fi 2024-06-26T06:34:32.5842423Z  2024-06-26T06:34:32.5843008Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-06-26T06:34:32.5844223Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-06-26T06:34:32.5845326Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-06-26T06:34:32.5846551Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-06-26T06:34:32.5847935Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-06-26T06:34:32.5848766Z fi 2024-06-26T06:34:32.5849105Z  2024-06-26T06:34:32.5849634Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T06:34:32.5857533Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:34:32.5858158Z env: 2024-06-26T06:34:32.5858471Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:32.5858921Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T06:34:32.5859495Z BASE_REVISION: 4b9c9a9cc9c9283380a011310ba180c105c3dcb9 2024-06-26T06:34:32.5860917Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:32.5862348Z DOCKER_TAG: 91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:32.5863099Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T06:34:32.5863749Z ##[endgroup] 2024-06-26T06:34:32.5888775Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T06:34:32.5889580Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T06:34:32.5891129Z + aws ecr get-login-password --region us-east-1 2024-06-26T06:34:32.5893203Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T06:34:33.0670946Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T06:34:33.0672063Z Configure a credential helper to remove this warning. See 2024-06-26T06:34:33.0673608Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T06:34:33.0674284Z 2024-06-26T06:34:33.0674433Z Login Succeeded 2024-06-26T06:34:33.0684737Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:33.2681575Z { 2024-06-26T06:34:33.2681978Z "schemaVersion": 2, 2024-06-26T06:34:33.2682622Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-06-26T06:34:33.2683379Z "config": { 2024-06-26T06:34:33.2684003Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-06-26T06:34:33.2684673Z "size": 47470, 2024-06-26T06:34:33.2685373Z "digest": "sha256:1fde65d68e1ba467b7c461d097349da1965328db5e6e5252ccb25702fcc3b83e" 2024-06-26T06:34:33.2686164Z }, 2024-06-26T06:34:33.2686471Z "layers": [ 2024-06-26T06:34:33.2686813Z { 2024-06-26T06:34:33.2687358Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2688055Z "size": 28580681, 2024-06-26T06:34:33.2688750Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-06-26T06:34:33.2689547Z }, 2024-06-26T06:34:33.2689835Z { 2024-06-26T06:34:33.2690382Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2691102Z "size": 7943353, 2024-06-26T06:34:33.2692176Z "digest": "sha256:09d415c238d76b32a7ea4a6e6add9542db9a5641f7f183af70aae185d0709e58" 2024-06-26T06:34:33.2692985Z }, 2024-06-26T06:34:33.2693296Z { 2024-06-26T06:34:33.2694483Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2695208Z "size": 56226389, 2024-06-26T06:34:33.2695917Z "digest": "sha256:9fe6e2e61518cba6844870c03b285737daec35e62baf25ae7744629ed3a7b470" 2024-06-26T06:34:33.2696714Z + exit 0 2024-06-26T06:34:33.2697022Z }, 2024-06-26T06:34:33.2697325Z { 2024-06-26T06:34:33.2697871Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2698556Z "size": 187, 2024-06-26T06:34:33.2699227Z "digest": "sha256:41f16248e682693ff20b3032c1d5e5541cc87c5af898ae2ff9b24d2940e59100" 2024-06-26T06:34:33.2700009Z }, 2024-06-26T06:34:33.2700293Z { 2024-06-26T06:34:33.2700830Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2701539Z "size": 6885, 2024-06-26T06:34:33.2702214Z "digest": "sha256:95d7b781703928cf3c4eece39d800cccb76728c375fedf51ecd83833fb25e458" 2024-06-26T06:34:33.2703002Z }, 2024-06-26T06:34:33.2703305Z { 2024-06-26T06:34:33.2704032Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2704847Z "size": 1383662475, 2024-06-26T06:34:33.2705526Z "digest": "sha256:8f6c9048534734f4c873935293b7296225846ceb31c1a158400a67ea170dde7f" 2024-06-26T06:34:33.2706537Z }, 2024-06-26T06:34:33.2706825Z { 2024-06-26T06:34:33.2707362Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2708065Z "size": 62282, 2024-06-26T06:34:33.2708721Z "digest": "sha256:ab17245097e491b9368790714f9d90ed447bf0973bd677cfe6f2456d62b72a13" 2024-06-26T06:34:33.2709493Z }, 2024-06-26T06:34:33.2709799Z { 2024-06-26T06:34:33.2710328Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2711024Z "size": 1684, 2024-06-26T06:34:33.2711716Z "digest": "sha256:dfecd7e9912b76ed460b8edd5a85f1943666e38a973ab5458177cf2c7c3110e3" 2024-06-26T06:34:33.2712492Z }, 2024-06-26T06:34:33.2712793Z { 2024-06-26T06:34:33.2713327Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2714031Z "size": 1522, 2024-06-26T06:34:33.2714714Z "digest": "sha256:464a8f74544589bf7b57f9a4cadcb6681e5ed00758f6c35025e691df4e88e890" 2024-06-26T06:34:33.2715497Z }, 2024-06-26T06:34:33.2715781Z { 2024-06-26T06:34:33.2716319Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2717026Z "size": 2445884909, 2024-06-26T06:34:33.2717697Z "digest": "sha256:d67c111fa58886d8904335b5122039816c8e51af5844cb6fe4e14154d9a7c016" 2024-06-26T06:34:33.2718461Z }, 2024-06-26T06:34:33.2718762Z { 2024-06-26T06:34:33.2719279Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2719973Z "size": 85882, 2024-06-26T06:34:33.2720654Z "digest": "sha256:0723abde19c09431db80b5d0f0e003541f6232c2da5c4f35c635c861875937ae" 2024-06-26T06:34:33.2721409Z }, 2024-06-26T06:34:33.2721714Z { 2024-06-26T06:34:33.2722354Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2723046Z "size": 1822, 2024-06-26T06:34:33.2723748Z "digest": "sha256:bb93d41d4cb4616eabdd81918bbf6c8673164e995dd0f0cc6d05989fdce3df21" 2024-06-26T06:34:33.2724546Z }, 2024-06-26T06:34:33.2724830Z { 2024-06-26T06:34:33.2725367Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2726075Z "size": 654858224, 2024-06-26T06:34:33.2726768Z "digest": "sha256:4b89839dd06631d80ca192bae8be4ea835acfde539e7e775339edcce9714c418" 2024-06-26T06:34:33.2727550Z }, 2024-06-26T06:34:33.2727853Z { 2024-06-26T06:34:33.2728369Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2729106Z "size": 546, 2024-06-26T06:34:33.2729798Z "digest": "sha256:caa9986c765baa228b63ef2e4e24be796d1aa0f41e20f1a9ccc44423d8a9a3f4" 2024-06-26T06:34:33.2730571Z }, 2024-06-26T06:34:33.2760840Z { 2024-06-26T06:34:33.2761612Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2762614Z "size": 1284, 2024-06-26T06:34:33.2763316Z "digest": "sha256:e17c09bdf243963c1185f01eca2a5ed4ae8e627181e77073452ea8a2811b661d" 2024-06-26T06:34:33.2764125Z }, 2024-06-26T06:34:33.2764443Z { 2024-06-26T06:34:33.2764999Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2765739Z "size": 484, 2024-06-26T06:34:33.2766446Z "digest": "sha256:bd2e32de75c90c1ed985c413bb08813271eeb40a7cb9c926e261d381c6fb4d82" 2024-06-26T06:34:33.2767233Z }, 2024-06-26T06:34:33.2767551Z { 2024-06-26T06:34:33.2768109Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2768813Z "size": 91710828, 2024-06-26T06:34:33.2769542Z "digest": "sha256:559a1daa0f9885fe07609ca2641d3e9e3d4d6aedca6c4d36ad90c1c765582bf0" 2024-06-26T06:34:33.2770348Z }, 2024-06-26T06:34:33.2770645Z { 2024-06-26T06:34:33.2771200Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2772231Z "size": 3235, 2024-06-26T06:34:33.2773109Z "digest": "sha256:d96cec7a425ccddeae1efd3e0f73355515e86e146cbaab2da9739cd651741068" 2024-06-26T06:34:33.2773965Z }, 2024-06-26T06:34:33.2774285Z { 2024-06-26T06:34:33.2774822Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2775539Z "size": 1905, 2024-06-26T06:34:33.2776246Z "digest": "sha256:1db6fc6c25902dd64b1dad171454f5b492ecb8895b4f8b6270de2a65750cfe12" 2024-06-26T06:34:33.2777034Z }, 2024-06-26T06:34:33.2777340Z { 2024-06-26T06:34:33.2777896Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2778596Z "size": 700, 2024-06-26T06:34:33.2779305Z "digest": "sha256:ac522bc703457eff4f1ceef81fd329f630b0feb067578ffebb5f721f5dd7a120" 2024-06-26T06:34:33.2780120Z }, 2024-06-26T06:34:33.2780420Z { 2024-06-26T06:34:33.2781161Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2781881Z "size": 2812517131, 2024-06-26T06:34:33.2782596Z "digest": "sha256:a05e0234ee85b8358a1b4c4bf7d9d7c4bc38e991a22895c0f9ffcbc709645fd9" 2024-06-26T06:34:33.2783424Z }, 2024-06-26T06:34:33.2783746Z { 2024-06-26T06:34:33.2784378Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2785188Z "size": 381, 2024-06-26T06:34:33.2786157Z "digest": "sha256:44f27f91d8fa036ce2af809752cba0d79d7da4ffe91d4981827319d759d6a230" 2024-06-26T06:34:33.2793732Z }, 2024-06-26T06:34:33.2794054Z { 2024-06-26T06:34:33.2794584Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2795289Z "size": 11891, 2024-06-26T06:34:33.2795975Z "digest": "sha256:0a0ded10616759d17568ed6ae0f22ebfe503413d2d4710223b5852c8eeb131b3" 2024-06-26T06:34:33.2796742Z }, 2024-06-26T06:34:33.2797050Z { 2024-06-26T06:34:33.2797596Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2798270Z "size": 803, 2024-06-26T06:34:33.2798951Z "digest": "sha256:0c62bfb80b7cdf32a713040cc54d11d45eec4105a51976da8b88f6e8dd79bb49" 2024-06-26T06:34:33.2820867Z }, 2024-06-26T06:34:33.2821569Z { 2024-06-26T06:34:33.2822636Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2823316Z "size": 106, 2024-06-26T06:34:33.2823969Z "digest": "sha256:0c8d1e7f32823458cc5bf0e51b63e2abb70878dea21687edffcf738af41051b6" 2024-06-26T06:34:33.2824733Z }, 2024-06-26T06:34:33.2825007Z { 2024-06-26T06:34:33.2825514Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2826190Z "size": 503, 2024-06-26T06:34:33.2826806Z "digest": "sha256:846366e78885c05844e15926128f2d3113c617522f700e485df868fbbba1c876" 2024-06-26T06:34:33.2827528Z }, 2024-06-26T06:34:33.2827799Z { 2024-06-26T06:34:33.2828309Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2828981Z "size": 121477247, 2024-06-26T06:34:33.2829664Z "digest": "sha256:efa11c713d18f2a9dcf7901ba63df5b9766293ca7c7824e5f3c089c960f45162" 2024-06-26T06:34:33.2830639Z }, 2024-06-26T06:34:33.2830911Z { 2024-06-26T06:34:33.2831429Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2832102Z "size": 109, 2024-06-26T06:34:33.2832757Z "digest": "sha256:32f0221befebf94bb7f0f81bdaef0f566e32695e7a98cb9020b555956fc9b89b" 2024-06-26T06:34:33.2833520Z }, 2024-06-26T06:34:33.2833795Z { 2024-06-26T06:34:33.2834305Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2834975Z "size": 490, 2024-06-26T06:34:33.2835620Z "digest": "sha256:a1d57c94ba33612a149ca2fea27363c0f960fd8516723c166d5b2783aabdaf16" 2024-06-26T06:34:33.2836371Z }, 2024-06-26T06:34:33.2836646Z { 2024-06-26T06:34:33.2837154Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2837827Z "size": 295, 2024-06-26T06:34:33.2838464Z "digest": "sha256:3dc15441e6d000a89a95092afb061c4c3c36b263a5159e68b4385e453b793faf" 2024-06-26T06:34:33.2839204Z }, 2024-06-26T06:34:33.2839476Z { 2024-06-26T06:34:33.2840151Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2840845Z "size": 103, 2024-06-26T06:34:33.2841490Z "digest": "sha256:668eadf990e86d6ac337335a56269ee7d21c3b7ab9d73a08b6234234070fcd70" 2024-06-26T06:34:33.2842236Z }, 2024-06-26T06:34:33.2842508Z { 2024-06-26T06:34:33.2843045Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2843730Z "size": 1473, 2024-06-26T06:34:33.2844413Z "digest": "sha256:fd5a832fe26e3c2c26c20395f500fc681ab28e9f9812e4b6b3e95aa15442103d" 2024-06-26T06:34:33.2845180Z }, 2024-06-26T06:34:33.2845483Z { 2024-06-26T06:34:33.2846020Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2846703Z "size": 446711131, 2024-06-26T06:34:33.2847405Z "digest": "sha256:b581c3782fd627bac2e6cc292144b92a21b499a07e2b598eaa2a48252e57a537" 2024-06-26T06:34:33.2848187Z }, 2024-06-26T06:34:33.2848471Z { 2024-06-26T06:34:33.2849006Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2849733Z "size": 160, 2024-06-26T06:34:33.2850367Z "digest": "sha256:3f42611c56360ba670f886b3c62442f9f556652382512f0153dbd888798fe690" 2024-06-26T06:34:33.2851132Z }, 2024-06-26T06:34:33.2851437Z { 2024-06-26T06:34:33.2852178Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2852887Z "size": 564, 2024-06-26T06:34:33.2853621Z "digest": "sha256:fe8dd930eda117742ffbb011f0c821bd08f9d183b46a81a9dc5fc3fca741b8bf" 2024-06-26T06:34:33.2854401Z }, 2024-06-26T06:34:33.2854709Z { 2024-06-26T06:34:33.2855247Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2855932Z "size": 35807552, 2024-06-26T06:34:33.2856631Z "digest": "sha256:6a9711a8a50dc2f9871121744f60deeb06e436ea2282ab2ed3d153ed4f2360d7" 2024-06-26T06:34:33.2857409Z }, 2024-06-26T06:34:33.2857693Z { 2024-06-26T06:34:33.2858232Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2858944Z "size": 104, 2024-06-26T06:34:33.2859615Z "digest": "sha256:76cd931663c9f6f95baad24c47a3f0125db05e116cc221dd313fb9f089af72d6" 2024-06-26T06:34:33.2860408Z }, 2024-06-26T06:34:33.2860715Z { 2024-06-26T06:34:33.2861233Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2861935Z "size": 426, 2024-06-26T06:34:33.2862611Z "digest": "sha256:c9ec95615433cc9fb674bbd2820228a5977f981d70ff677927b0caf42d8c56b8" 2024-06-26T06:34:33.2863370Z }, 2024-06-26T06:34:33.2863678Z { 2024-06-26T06:34:33.2864214Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2864902Z "size": 20262060, 2024-06-26T06:34:33.2865602Z "digest": "sha256:7da33ab0319397119e259c5ee2c37d7c6cb854983c8a320c59746a27d4826ed2" 2024-06-26T06:34:33.2866385Z }, 2024-06-26T06:34:33.2866669Z { 2024-06-26T06:34:33.2867214Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2868103Z "size": 439, 2024-06-26T06:34:33.2868776Z "digest": "sha256:da956bbe6180c03f03d667edd26b2f9a43df083200e13847da28ad27a9f83489" 2024-06-26T06:34:33.2869577Z }, 2024-06-26T06:34:33.2869883Z { 2024-06-26T06:34:33.2870409Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2871210Z "size": 700, 2024-06-26T06:34:33.2871886Z "digest": "sha256:ac522bc703457eff4f1ceef81fd329f630b0feb067578ffebb5f721f5dd7a120" 2024-06-26T06:34:33.2872686Z }, 2024-06-26T06:34:33.2872990Z { 2024-06-26T06:34:33.2873509Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2874211Z "size": 143, 2024-06-26T06:34:33.2874885Z "digest": "sha256:73e6b5984544e73e642e8a11fd1dddf14a05913257132ee47f7782b6fb519ee6" 2024-06-26T06:34:33.2875637Z }, 2024-06-26T06:34:33.2875946Z { 2024-06-26T06:34:33.2876485Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2877169Z "size": 135, 2024-06-26T06:34:33.2878000Z "digest": "sha256:e5a6ec0a41d77fcb1af98a4aa079b9219c47c2e5eb696b52883c390373b1af8e" 2024-06-26T06:34:33.2878813Z }, 2024-06-26T06:34:33.2879098Z { 2024-06-26T06:34:33.2879639Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2880335Z "size": 32, 2024-06-26T06:34:33.2880999Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T06:34:33.2881787Z }, 2024-06-26T06:34:33.2882096Z { 2024-06-26T06:34:33.2882615Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2883316Z "size": 191, 2024-06-26T06:34:33.2884000Z "digest": "sha256:4b25a744b402ec8a8cdfd55f95c346bcf9178c450cd08e8bb2834efa41bb621e" 2024-06-26T06:34:33.2884770Z }, 2024-06-26T06:34:33.2885068Z { 2024-06-26T06:34:33.2885608Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2886290Z "size": 564, 2024-06-26T06:34:33.2886957Z "digest": "sha256:21f74857bd1744f0b55351469eec060cd60d78fac461e695f550de53cec8edc0" 2024-06-26T06:34:33.2887733Z }, 2024-06-26T06:34:33.2888026Z { 2024-06-26T06:34:33.2888574Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2889285Z "size": 43163151, 2024-06-26T06:34:33.2889962Z "digest": "sha256:6c12f547d202783d251e0813abaeaad2b30ba464d28f5902171b637f829ab84a" 2024-06-26T06:34:33.2890747Z }, 2024-06-26T06:34:33.2891061Z { 2024-06-26T06:34:33.2891585Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2892562Z "size": 106, 2024-06-26T06:34:33.2893257Z "digest": "sha256:ed4e5a4f8526fca89a9272c39328cb84dbec72a12e74d4abde50a706f2983b55" 2024-06-26T06:34:33.2894029Z }, 2024-06-26T06:34:33.2894331Z { 2024-06-26T06:34:33.2894873Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2895555Z "size": 1212, 2024-06-26T06:34:33.2896244Z "digest": "sha256:6f09f741ead60ea209087bf9071ef1a6fcc299c42ce4b3a297c6d96a6393f67f" 2024-06-26T06:34:33.2897042Z }, 2024-06-26T06:34:33.2897331Z { 2024-06-26T06:34:33.2897887Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2898584Z "size": 700, 2024-06-26T06:34:33.2899260Z "digest": "sha256:ac522bc703457eff4f1ceef81fd329f630b0feb067578ffebb5f721f5dd7a120" 2024-06-26T06:34:33.2900062Z }, 2024-06-26T06:34:33.2900368Z { 2024-06-26T06:34:33.2900892Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2901596Z "size": 138, 2024-06-26T06:34:33.2902290Z "digest": "sha256:411fce2db725b6d9a9d7b7f953cdd354ceef707e1dec7a2ef5115beba5a6788e" 2024-06-26T06:34:33.2903091Z }, 2024-06-26T06:34:33.2903377Z { 2024-06-26T06:34:33.2903915Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2904601Z "size": 120, 2024-06-26T06:34:33.2905468Z "digest": "sha256:87f7902d12a4c612e9be6b4cceae10e66baf25332f924496d3b70f1967b83360" 2024-06-26T06:34:33.2906252Z }, 2024-06-26T06:34:33.2906687Z { 2024-06-26T06:34:33.2907241Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2907947Z "size": 1905185822, 2024-06-26T06:34:33.2908645Z "digest": "sha256:9777c07909ada379d44a538999c6492011749a4f0fcbd5dfee534795401df9d3" 2024-06-26T06:34:33.2909391Z }, 2024-06-26T06:34:33.2909695Z { 2024-06-26T06:34:33.2910218Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2910915Z "size": 172, 2024-06-26T06:34:33.2911579Z "digest": "sha256:9755a952d3ecbc817b3290445710170b8c63a35f902a9bddec1837eee9a4db57" 2024-06-26T06:34:33.2912350Z }, 2024-06-26T06:34:33.2912634Z { 2024-06-26T06:34:33.2913166Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2913872Z "size": 908, 2024-06-26T06:34:33.2914539Z "digest": "sha256:5ca5d2e8bebe5a5641fd35fc6585f4c83ae8ae4528b84682b276a04b9006faa2" 2024-06-26T06:34:33.2915329Z }, 2024-06-26T06:34:33.2915630Z { 2024-06-26T06:34:33.2916161Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2916984Z "size": 700, 2024-06-26T06:34:33.2917696Z "digest": "sha256:ac522bc703457eff4f1ceef81fd329f630b0feb067578ffebb5f721f5dd7a120" 2024-06-26T06:34:33.2918471Z }, 2024-06-26T06:34:33.2918775Z { 2024-06-26T06:34:33.2919312Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2919993Z "size": 135, 2024-06-26T06:34:33.2920679Z "digest": "sha256:d07af843db47bac709de490bf12e9f993080a666ddda8ee54210dd4b6526f97d" 2024-06-26T06:34:33.2921465Z }, 2024-06-26T06:34:33.2921747Z { 2024-06-26T06:34:33.2922285Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2922988Z "size": 32, 2024-06-26T06:34:33.2923651Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T06:34:33.2924442Z }, 2024-06-26T06:34:33.2924745Z { 2024-06-26T06:34:33.2925267Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2925978Z "size": 157, 2024-06-26T06:34:33.2926684Z "digest": "sha256:dbbd7a4ca8469b1fc21ae5f549fc832e79c4dae36515cc0b594a4f8d6531832e" 2024-06-26T06:34:33.2927467Z }, 2024-06-26T06:34:33.2927773Z { 2024-06-26T06:34:33.2928314Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2928995Z "size": 1841, 2024-06-26T06:34:33.2929682Z "digest": "sha256:6a18c61a96dd8f2f6cd96d48d62262059c9b024ffad4cf0df6a7d4c02f58d9e2" 2024-06-26T06:34:33.2930473Z }, 2024-06-26T06:34:33.2930764Z { 2024-06-26T06:34:33.2931302Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2932221Z "size": 7529784, 2024-06-26T06:34:33.2932894Z "digest": "sha256:bd12222c02373373bcdd92c079d4d6e405dee01eb986422d608659c4bb0cd809" 2024-06-26T06:34:33.2933670Z }, 2024-06-26T06:34:33.2933971Z { 2024-06-26T06:34:33.2934490Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2935203Z "size": 163, 2024-06-26T06:34:33.2935899Z "digest": "sha256:a5dfe6ba36fd2223ebbc9e888593b6341d5bd3b56b1674cbd9064f55c26de604" 2024-06-26T06:34:33.2936672Z }, 2024-06-26T06:34:33.2936979Z { 2024-06-26T06:34:33.2937529Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2938213Z "size": 7944, 2024-06-26T06:34:33.2938903Z "digest": "sha256:4b990eb76e8f2de191d0c2e639846adb8df24ce3380ec150ead1baa95b26d7ec" 2024-06-26T06:34:33.2939695Z }, 2024-06-26T06:34:33.2939980Z { 2024-06-26T06:34:33.2940519Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2941224Z "size": 8069, 2024-06-26T06:34:33.2941896Z "digest": "sha256:d9df85e8d42263f8089be7b63aaf1e2bfc3207f8d81b1d79f9f86e96c17298ee" 2024-06-26T06:34:33.2942690Z }, 2024-06-26T06:34:33.2942990Z { 2024-06-26T06:34:33.2943512Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2944216Z "size": 304, 2024-06-26T06:34:33.2945065Z "digest": "sha256:47d89094b1a7e9cc21b8db8ab9c66fefcc9bf7544c170508f41efb5d26e3ccd2" 2024-06-26T06:34:33.2945836Z }, 2024-06-26T06:34:33.2946141Z { 2024-06-26T06:34:33.2946677Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2947363Z "size": 7628836, 2024-06-26T06:34:33.2948052Z "digest": "sha256:94beee2c356a83536890159dc698da22baca90a4c8289622ed556f5a47947d79" 2024-06-26T06:34:33.2948826Z }, 2024-06-26T06:34:33.2949111Z { 2024-06-26T06:34:33.2949646Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2950353Z "size": 108, 2024-06-26T06:34:33.2951015Z "digest": "sha256:322c644984f492ea6b5b3d0b9eaab0bdd8d29e156e25a60d475a7dbe0c08621f" 2024-06-26T06:34:33.2951806Z }, 2024-06-26T06:34:33.2952109Z { 2024-06-26T06:34:33.2952628Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2953379Z "size": 54145774, 2024-06-26T06:34:33.2954083Z "digest": "sha256:b43ee04d974d5ef5236d381dd8e9d87677715edc695f235bb1e954b54a8f1143" 2024-06-26T06:34:33.2954974Z }, 2024-06-26T06:34:33.2955298Z { 2024-06-26T06:34:33.2955836Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2956518Z "size": 474, 2024-06-26T06:34:33.2957180Z "digest": "sha256:40f95cbeec2c62a99f4548e3f7139c89986c010bce3742c85d203fdd5a120c4e" 2024-06-26T06:34:33.2957919Z }, 2024-06-26T06:34:33.2958190Z { 2024-06-26T06:34:33.2958818Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2959517Z "size": 1373229352, 2024-06-26T06:34:33.2960228Z "digest": "sha256:4b2a3c41d6ae93889d0aed7b8b3e87eb334b8fb8aedb20bab9d7fea2759b7055" 2024-06-26T06:34:33.2961028Z }, 2024-06-26T06:34:33.2961328Z { 2024-06-26T06:34:33.2961842Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2962528Z "size": 106, 2024-06-26T06:34:33.2963174Z "digest": "sha256:b2689866a0b9bc7b622498695ae245e7f4996dfa2db1311cff0b6fae0ff7e6e8" 2024-06-26T06:34:33.2963923Z }, 2024-06-26T06:34:33.2964204Z { 2024-06-26T06:34:33.2964740Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2965421Z "size": 556, 2024-06-26T06:34:33.2966112Z "digest": "sha256:9b36f9d4ace6c70a0fddeca47d17eb19639fd636a6084e02555f7d862bf44c40" 2024-06-26T06:34:33.2966908Z }, 2024-06-26T06:34:33.2967193Z { 2024-06-26T06:34:33.2967732Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2968432Z "size": 36532084, 2024-06-26T06:34:33.2969105Z "digest": "sha256:a863a1c01885d8f3506984d7a7d8179273b3d29ed4b4e2dee41471a6437155f4" 2024-06-26T06:34:33.2969878Z }, 2024-06-26T06:34:33.2970183Z { 2024-06-26T06:34:33.2970703Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2971407Z "size": 111, 2024-06-26T06:34:33.2972288Z "digest": "sha256:b9b8edd214be18381ccf301a0b92a7853f7ecba0b219699cd74c9b2e064183fd" 2024-06-26T06:34:33.2973072Z }, 2024-06-26T06:34:33.2973375Z { 2024-06-26T06:34:33.2973920Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2974597Z "size": 32, 2024-06-26T06:34:33.2975278Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T06:34:33.2976055Z }, 2024-06-26T06:34:33.2976326Z { 2024-06-26T06:34:33.2976858Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2977562Z "size": 32, 2024-06-26T06:34:33.2978220Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T06:34:33.2978994Z }, 2024-06-26T06:34:33.2979297Z { 2024-06-26T06:34:33.2979815Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2980513Z "size": 32, 2024-06-26T06:34:33.2981192Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T06:34:33.2981957Z }, 2024-06-26T06:34:33.2982427Z { 2024-06-26T06:34:33.2982967Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T06:34:33.2983643Z "size": 32, 2024-06-26T06:34:33.2984318Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T06:34:33.2985098Z } 2024-06-26T06:34:33.2985380Z ] 2024-06-26T06:34:33.2985684Z } 2024-06-26T06:34:33.3121658Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T06:34:33.3122286Z tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T06:34:33.3122967Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-06-26T06:34:33.3131519Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:34:33.3132422Z env: 2024-06-26T06:34:33.3132768Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:33.3134067Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:33.3135401Z ##[endgroup] 2024-06-26T06:34:33.3162958Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9-91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:33.3227830Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-06-26T06:34:33.3228572Z with: 2024-06-26T06:34:33.3229760Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:33.3231276Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T06:34:33.3231903Z env: 2024-06-26T06:34:33.3232235Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:33.3232663Z ##[endgroup] 2024-06-26T06:34:33.3255942Z ##[group]Run set -x 2024-06-26T06:34:33.3256361Z set -x 2024-06-26T06:34:33.3256730Z set +e 2024-06-26T06:34:33.3257085Z  2024-06-26T06:34:33.3257405Z login() { 2024-06-26T06:34:33.3258235Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T06:34:33.3259161Z } 2024-06-26T06:34:33.3259480Z  2024-06-26T06:34:33.3259875Z retry () { 2024-06-26T06:34:33.3260349Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T06:34:33.3260906Z } 2024-06-26T06:34:33.3261241Z  2024-06-26T06:34:33.3261602Z retry login "${DOCKER_REGISTRY}" 2024-06-26T06:34:33.3262116Z  2024-06-26T06:34:33.3262444Z set -e 2024-06-26T06:34:33.3263012Z # ignore output since only exit code is used for conditional 2024-06-26T06:34:33.3263892Z # only pull docker image if it's not available locally 2024-06-26T06:34:33.3264861Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-06-26T06:34:33.3265735Z  retry docker pull "${DOCKER_IMAGE}" 2024-06-26T06:34:33.3266262Z fi 2024-06-26T06:34:33.3273621Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:34:33.3274259Z env: 2024-06-26T06:34:33.3274574Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:34:33.3275861Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:33.3277353Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T06:34:33.3277987Z ##[endgroup] 2024-06-26T06:34:33.3303409Z + set +e 2024-06-26T06:34:33.3304483Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T06:34:33.3306031Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T06:34:33.3307384Z + aws ecr get-login-password --region us-east-1 2024-06-26T06:34:33.3308758Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T06:34:33.8106660Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T06:34:33.8107562Z Configure a credential helper to remove this warning. See 2024-06-26T06:34:33.8108964Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T06:34:33.8109579Z 2024-06-26T06:34:33.8109777Z Login Succeeded 2024-06-26T06:34:33.8119966Z + set -e 2024-06-26T06:34:33.8122335Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:33.8293210Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:33.8295641Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:34:34.0569319Z 91382da70d5719cd7007b6b80b71d2f48398f6b7: Pulling from pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9 2024-06-26T06:34:34.0570935Z 7a2c55901189: Pulling fs layer 2024-06-26T06:34:34.0571444Z 09d415c238d7: Pulling fs layer 2024-06-26T06:34:34.0572500Z 9fe6e2e61518: Pulling fs layer 2024-06-26T06:34:34.0572868Z 41f16248e682: Pulling fs layer 2024-06-26T06:34:34.0573615Z 95d7b7817039: Pulling fs layer 2024-06-26T06:34:34.0574134Z 8f6c90485347: Pulling fs layer 2024-06-26T06:34:34.0574600Z ab17245097e4: Pulling fs layer 2024-06-26T06:34:34.0575083Z dfecd7e9912b: Pulling fs layer 2024-06-26T06:34:34.0575451Z 464a8f745445: Pulling fs layer 2024-06-26T06:34:34.0575875Z d67c111fa588: Pulling fs layer 2024-06-26T06:34:34.0576450Z 0723abde19c0: Pulling fs layer 2024-06-26T06:34:34.0576883Z bb93d41d4cb4: Pulling fs layer 2024-06-26T06:34:34.0577339Z 4b89839dd066: Pulling fs layer 2024-06-26T06:34:34.0577768Z caa9986c765b: Pulling fs layer 2024-06-26T06:34:34.0578262Z e17c09bdf243: Pulling fs layer 2024-06-26T06:34:34.0578767Z bd2e32de75c9: Pulling fs layer 2024-06-26T06:34:34.0579138Z 559a1daa0f98: Pulling fs layer 2024-06-26T06:34:34.0579632Z d96cec7a425c: Pulling fs layer 2024-06-26T06:34:34.0580023Z 1db6fc6c2590: Pulling fs layer 2024-06-26T06:34:34.0580533Z ac522bc70345: Pulling fs layer 2024-06-26T06:34:34.0580994Z a05e0234ee85: Pulling fs layer 2024-06-26T06:34:34.0581465Z 44f27f91d8fa: Pulling fs layer 2024-06-26T06:34:34.0581920Z 0a0ded106167: Pulling fs layer 2024-06-26T06:34:34.0582399Z 0c62bfb80b7c: Pulling fs layer 2024-06-26T06:34:34.0582823Z 0c8d1e7f3282: Pulling fs layer 2024-06-26T06:34:34.0583307Z 846366e78885: Pulling fs layer 2024-06-26T06:34:34.0583737Z efa11c713d18: Pulling fs layer 2024-06-26T06:34:34.0584087Z 32f0221befeb: Pulling fs layer 2024-06-26T06:34:34.0584551Z a1d57c94ba33: Pulling fs layer 2024-06-26T06:34:34.0585017Z 3dc15441e6d0: Pulling fs layer 2024-06-26T06:34:34.0585620Z 668eadf990e8: Pulling fs layer 2024-06-26T06:34:34.0586002Z fd5a832fe26e: Pulling fs layer 2024-06-26T06:34:34.0586442Z b581c3782fd6: Pulling fs layer 2024-06-26T06:34:34.0586899Z 3f42611c5636: Pulling fs layer 2024-06-26T06:34:34.0587340Z fe8dd930eda1: Pulling fs layer 2024-06-26T06:34:34.0587822Z 41f16248e682: Waiting 2024-06-26T06:34:34.0588195Z 6a9711a8a50d: Pulling fs layer 2024-06-26T06:34:34.0588666Z 76cd931663c9: Pulling fs layer 2024-06-26T06:34:34.0589042Z 8f6c90485347: Waiting 2024-06-26T06:34:34.0589509Z c9ec95615433: Pulling fs layer 2024-06-26T06:34:34.0590090Z 7da33ab03193: Pulling fs layer 2024-06-26T06:34:34.0590398Z 95d7b7817039: Waiting 2024-06-26T06:34:34.0590785Z ab17245097e4: Waiting 2024-06-26T06:34:34.0591172Z 464a8f745445: Waiting 2024-06-26T06:34:34.0591559Z da956bbe6180: Pulling fs layer 2024-06-26T06:34:34.0592012Z dfecd7e9912b: Waiting 2024-06-26T06:34:34.0592401Z 73e6b5984544: Pulling fs layer 2024-06-26T06:34:34.0592843Z 559a1daa0f98: Waiting 2024-06-26T06:34:34.0593230Z d67c111fa588: Waiting 2024-06-26T06:34:34.0593598Z 0c8d1e7f3282: Waiting 2024-06-26T06:34:34.0594005Z e5a6ec0a41d7: Pulling fs layer 2024-06-26T06:34:34.0594472Z 4f4fb700ef54: Pulling fs layer 2024-06-26T06:34:34.0594903Z 0723abde19c0: Waiting 2024-06-26T06:34:34.0595415Z 4b25a744b402: Pulling fs layer 2024-06-26T06:34:34.0595755Z d96cec7a425c: Waiting 2024-06-26T06:34:34.0596452Z 21f74857bd17: Pulling fs layer 2024-06-26T06:34:34.0596855Z 6c12f547d202: Pulling fs layer 2024-06-26T06:34:34.0597220Z caa9986c765b: Waiting 2024-06-26T06:34:34.0597747Z ed4e5a4f8526: Pulling fs layer 2024-06-26T06:34:34.0598073Z 1db6fc6c2590: Waiting 2024-06-26T06:34:34.0598612Z 6f09f741ead6: Pulling fs layer 2024-06-26T06:34:34.0598903Z 846366e78885: Waiting 2024-06-26T06:34:34.0599292Z efa11c713d18: Waiting 2024-06-26T06:34:34.0599695Z 411fce2db725: Pulling fs layer 2024-06-26T06:34:34.0600274Z 32f0221befeb: Waiting 2024-06-26T06:34:34.0600549Z 87f7902d12a4: Pulling fs layer 2024-06-26T06:34:34.0601123Z e17c09bdf243: Waiting 2024-06-26T06:34:34.0601416Z 9777c07909ad: Pulling fs layer 2024-06-26T06:34:34.0601837Z bd2e32de75c9: Waiting 2024-06-26T06:34:34.0602230Z 9755a952d3ec: Pulling fs layer 2024-06-26T06:34:34.0602660Z a1d57c94ba33: Waiting 2024-06-26T06:34:34.0603056Z b581c3782fd6: Waiting 2024-06-26T06:34:34.0603469Z 5ca5d2e8bebe: Pulling fs layer 2024-06-26T06:34:34.0603921Z fd5a832fe26e: Waiting 2024-06-26T06:34:34.0604319Z 6a9711a8a50d: Waiting 2024-06-26T06:34:34.0604826Z 76cd931663c9: Waiting 2024-06-26T06:34:34.0605247Z d07af843db47: Pulling fs layer 2024-06-26T06:34:34.0605670Z 44f27f91d8fa: Waiting 2024-06-26T06:34:34.0606081Z dbbd7a4ca846: Pulling fs layer 2024-06-26T06:34:34.0606512Z a05e0234ee85: Waiting 2024-06-26T06:34:34.0606919Z 6a18c61a96dd: Pulling fs layer 2024-06-26T06:34:34.0607481Z 6c12f547d202: Waiting 2024-06-26T06:34:34.0607884Z bb93d41d4cb4: Waiting 2024-06-26T06:34:34.0608129Z 0a0ded106167: Waiting 2024-06-26T06:34:34.0608501Z 3dc15441e6d0: Waiting 2024-06-26T06:34:34.0608943Z ed4e5a4f8526: Waiting 2024-06-26T06:34:34.0609371Z 0c62bfb80b7c: Waiting 2024-06-26T06:34:34.0609673Z bd12222c0237: Pulling fs layer 2024-06-26T06:34:34.0610193Z 3f42611c5636: Waiting 2024-06-26T06:34:34.0610555Z 4f4fb700ef54: Waiting 2024-06-26T06:34:34.0610899Z a5dfe6ba36fd: Pulling fs layer 2024-06-26T06:34:34.0611448Z fe8dd930eda1: Waiting 2024-06-26T06:34:34.0612127Z 4b990eb76e8f: Pulling fs layer 2024-06-26T06:34:34.0612554Z 4b89839dd066: Waiting 2024-06-26T06:34:34.0612850Z d9df85e8d422: Pulling fs layer 2024-06-26T06:34:34.0613324Z 47d89094b1a7: Pulling fs layer 2024-06-26T06:34:34.0613761Z 94beee2c356a: Pulling fs layer 2024-06-26T06:34:34.0614228Z 322c644984f4: Pulling fs layer 2024-06-26T06:34:34.0614723Z b43ee04d974d: Pulling fs layer 2024-06-26T06:34:34.0615136Z 40f95cbeec2c: Pulling fs layer 2024-06-26T06:34:34.0615600Z 4b2a3c41d6ae: Pulling fs layer 2024-06-26T06:34:34.0616038Z 73e6b5984544: Waiting 2024-06-26T06:34:34.0616480Z b2689866a0b9: Pulling fs layer 2024-06-26T06:34:34.0616893Z 9b36f9d4ace6: Pulling fs layer 2024-06-26T06:34:34.0617354Z a863a1c01885: Pulling fs layer 2024-06-26T06:34:34.0617778Z 9755a952d3ec: Waiting 2024-06-26T06:34:34.0618168Z ac522bc70345: Waiting 2024-06-26T06:34:34.0618676Z b9b8edd214be: Pulling fs layer 2024-06-26T06:34:34.0619016Z 5ca5d2e8bebe: Waiting 2024-06-26T06:34:34.0619451Z 87f7902d12a4: Waiting 2024-06-26T06:34:34.0619806Z 411fce2db725: Waiting 2024-06-26T06:34:34.0620178Z d07af843db47: Waiting 2024-06-26T06:34:34.0620565Z c9ec95615433: Waiting 2024-06-26T06:34:34.0620972Z b9b8edd214be: Waiting 2024-06-26T06:34:34.0621344Z dbbd7a4ca846: Waiting 2024-06-26T06:34:34.0621732Z b2689866a0b9: Waiting 2024-06-26T06:34:34.0622117Z 4b2a3c41d6ae: Waiting 2024-06-26T06:34:34.0622485Z 6a18c61a96dd: Waiting 2024-06-26T06:34:34.0622872Z 9b36f9d4ace6: Waiting 2024-06-26T06:34:34.0623261Z a863a1c01885: Waiting 2024-06-26T06:34:34.0623647Z 668eadf990e8: Waiting 2024-06-26T06:34:34.0624013Z 322c644984f4: Waiting 2024-06-26T06:34:34.0624394Z 6f09f741ead6: Waiting 2024-06-26T06:34:34.0624776Z b43ee04d974d: Waiting 2024-06-26T06:34:34.0625146Z 40f95cbeec2c: Waiting 2024-06-26T06:34:34.0625535Z e5a6ec0a41d7: Waiting 2024-06-26T06:34:34.0625903Z 4b25a744b402: Waiting 2024-06-26T06:34:34.0626288Z da956bbe6180: Waiting 2024-06-26T06:34:34.0626679Z a5dfe6ba36fd: Waiting 2024-06-26T06:34:34.0627050Z 9777c07909ad: Waiting 2024-06-26T06:34:34.2209060Z 09d415c238d7: Download complete 2024-06-26T06:34:34.3066684Z 41f16248e682: Verifying Checksum 2024-06-26T06:34:34.3067447Z 41f16248e682: Download complete 2024-06-26T06:34:34.3933128Z 95d7b7817039: Verifying Checksum 2024-06-26T06:34:34.3933659Z 95d7b7817039: Download complete 2024-06-26T06:34:34.4112372Z 7a2c55901189: Download complete 2024-06-26T06:34:34.5559661Z ab17245097e4: Verifying Checksum 2024-06-26T06:34:34.5560308Z ab17245097e4: Download complete 2024-06-26T06:34:34.6313478Z dfecd7e9912b: Verifying Checksum 2024-06-26T06:34:34.6313988Z dfecd7e9912b: Download complete 2024-06-26T06:34:34.6953454Z 464a8f745445: Verifying Checksum 2024-06-26T06:34:34.6954570Z 464a8f745445: Download complete 2024-06-26T06:34:34.6962075Z 9fe6e2e61518: Verifying Checksum 2024-06-26T06:34:34.6962603Z 9fe6e2e61518: Download complete 2024-06-26T06:34:34.7954305Z 0723abde19c0: Download complete 2024-06-26T06:34:34.8692274Z bb93d41d4cb4: Verifying Checksum 2024-06-26T06:34:34.8692911Z bb93d41d4cb4: Download complete 2024-06-26T06:34:35.7144806Z 7a2c55901189: Pull complete 2024-06-26T06:34:36.0838738Z 09d415c238d7: Pull complete 2024-06-26T06:34:37.2512434Z 9fe6e2e61518: Pull complete 2024-06-26T06:34:37.2954291Z 41f16248e682: Pull complete 2024-06-26T06:34:37.3464941Z 95d7b7817039: Pull complete 2024-06-26T06:34:41.6155943Z 4b89839dd066: Verifying Checksum 2024-06-26T06:34:41.6156540Z 4b89839dd066: Download complete 2024-06-26T06:34:41.6914135Z caa9986c765b: Verifying Checksum 2024-06-26T06:34:41.6914685Z caa9986c765b: Download complete 2024-06-26T06:34:41.7700285Z e17c09bdf243: Verifying Checksum 2024-06-26T06:34:41.7700817Z e17c09bdf243: Download complete 2024-06-26T06:34:41.8311922Z bd2e32de75c9: Verifying Checksum 2024-06-26T06:34:41.8312451Z bd2e32de75c9: Download complete 2024-06-26T06:34:42.8208016Z 559a1daa0f98: Verifying Checksum 2024-06-26T06:34:42.8208658Z 559a1daa0f98: Download complete 2024-06-26T06:34:42.9114174Z d96cec7a425c: Verifying Checksum 2024-06-26T06:34:42.9114780Z d96cec7a425c: Download complete 2024-06-26T06:34:42.9811976Z 1db6fc6c2590: Download complete 2024-06-26T06:34:43.0905213Z ac522bc70345: Verifying Checksum 2024-06-26T06:34:43.0905741Z ac522bc70345: Download complete 2024-06-26T06:34:48.3063475Z 8f6c90485347: Verifying Checksum 2024-06-26T06:34:48.3063936Z 8f6c90485347: Download complete 2024-06-26T06:34:48.3904258Z 44f27f91d8fa: Verifying Checksum 2024-06-26T06:34:48.3904827Z 44f27f91d8fa: Download complete 2024-06-26T06:34:48.4767032Z 0a0ded106167: Verifying Checksum 2024-06-26T06:34:48.4767568Z 0a0ded106167: Download complete 2024-06-26T06:34:48.5605994Z 0c62bfb80b7c: Verifying Checksum 2024-06-26T06:34:48.5606652Z 0c62bfb80b7c: Download complete 2024-06-26T06:34:48.6403389Z 0c8d1e7f3282: Verifying Checksum 2024-06-26T06:34:48.6404028Z 0c8d1e7f3282: Download complete 2024-06-26T06:34:48.7344710Z 846366e78885: Verifying Checksum 2024-06-26T06:34:48.7345312Z 846366e78885: Download complete 2024-06-26T06:34:50.0848323Z efa11c713d18: Verifying Checksum 2024-06-26T06:34:50.0848933Z efa11c713d18: Download complete 2024-06-26T06:34:50.1635962Z 32f0221befeb: Verifying Checksum 2024-06-26T06:34:50.1636537Z 32f0221befeb: Download complete 2024-06-26T06:34:50.2902042Z a1d57c94ba33: Download complete 2024-06-26T06:34:50.3645497Z 3dc15441e6d0: Download complete 2024-06-26T06:34:50.4330401Z 668eadf990e8: Download complete 2024-06-26T06:34:50.5228334Z fd5a832fe26e: Verifying Checksum 2024-06-26T06:34:50.5228908Z fd5a832fe26e: Download complete 2024-06-26T06:34:55.6551273Z b581c3782fd6: Verifying Checksum 2024-06-26T06:34:55.6552357Z b581c3782fd6: Download complete 2024-06-26T06:34:55.7396201Z 3f42611c5636: Verifying Checksum 2024-06-26T06:34:55.7396959Z 3f42611c5636: Download complete 2024-06-26T06:34:55.8167584Z fe8dd930eda1: Verifying Checksum 2024-06-26T06:34:55.8168642Z fe8dd930eda1: Download complete 2024-06-26T06:34:56.3908684Z 6a9711a8a50d: Verifying Checksum 2024-06-26T06:34:56.3909442Z 6a9711a8a50d: Download complete 2024-06-26T06:34:56.4695533Z 76cd931663c9: Verifying Checksum 2024-06-26T06:34:56.4696700Z 76cd931663c9: Download complete 2024-06-26T06:34:56.5902576Z c9ec95615433: Verifying Checksum 2024-06-26T06:34:56.5903058Z c9ec95615433: Download complete 2024-06-26T06:34:56.9070115Z 7da33ab03193: Verifying Checksum 2024-06-26T06:34:56.9070691Z 7da33ab03193: Download complete 2024-06-26T06:34:56.9705385Z da956bbe6180: Verifying Checksum 2024-06-26T06:34:56.9705921Z da956bbe6180: Download complete 2024-06-26T06:34:57.0346725Z 73e6b5984544: Verifying Checksum 2024-06-26T06:34:57.0347294Z 73e6b5984544: Download complete 2024-06-26T06:34:57.1208753Z e5a6ec0a41d7: Verifying Checksum 2024-06-26T06:34:57.1209306Z e5a6ec0a41d7: Download complete 2024-06-26T06:34:57.1317341Z 4f4fb700ef54: Verifying Checksum 2024-06-26T06:34:57.1317774Z 4f4fb700ef54: Download complete 2024-06-26T06:34:57.2168196Z 4b25a744b402: Verifying Checksum 2024-06-26T06:34:57.2168760Z 4b25a744b402: Download complete 2024-06-26T06:34:57.2946997Z 21f74857bd17: Verifying Checksum 2024-06-26T06:34:57.2947572Z 21f74857bd17: Download complete 2024-06-26T06:34:57.9196679Z 6c12f547d202: Verifying Checksum 2024-06-26T06:34:57.9197259Z 6c12f547d202: Download complete 2024-06-26T06:34:57.9966936Z ed4e5a4f8526: Verifying Checksum 2024-06-26T06:34:57.9967754Z ed4e5a4f8526: Download complete 2024-06-26T06:34:58.1102923Z 6f09f741ead6: Download complete 2024-06-26T06:34:58.2067997Z 411fce2db725: Verifying Checksum 2024-06-26T06:34:58.2070018Z 411fce2db725: Download complete 2024-06-26T06:34:58.3164583Z 87f7902d12a4: Verifying Checksum 2024-06-26T06:34:58.3165073Z 87f7902d12a4: Download complete 2024-06-26T06:34:59.2472667Z d67c111fa588: Verifying Checksum 2024-06-26T06:34:59.2473431Z d67c111fa588: Download complete 2024-06-26T06:34:59.3477909Z 9755a952d3ec: Download complete 2024-06-26T06:34:59.4258730Z 5ca5d2e8bebe: Verifying Checksum 2024-06-26T06:34:59.4259259Z 5ca5d2e8bebe: Download complete 2024-06-26T06:34:59.5336535Z d07af843db47: Verifying Checksum 2024-06-26T06:34:59.5337105Z d07af843db47: Download complete 2024-06-26T06:34:59.6444075Z dbbd7a4ca846: Verifying Checksum 2024-06-26T06:34:59.6445086Z dbbd7a4ca846: Download complete 2024-06-26T06:34:59.7267794Z 6a18c61a96dd: Verifying Checksum 2024-06-26T06:34:59.7268331Z 6a18c61a96dd: Download complete 2024-06-26T06:34:59.9116951Z bd12222c0237: Download complete 2024-06-26T06:34:59.9917123Z a5dfe6ba36fd: Verifying Checksum 2024-06-26T06:34:59.9917649Z a5dfe6ba36fd: Download complete 2024-06-26T06:35:00.0745883Z 4b990eb76e8f: Verifying Checksum 2024-06-26T06:35:00.0746425Z 4b990eb76e8f: Download complete 2024-06-26T06:35:00.1587135Z d9df85e8d422: Verifying Checksum 2024-06-26T06:35:00.1587709Z d9df85e8d422: Download complete 2024-06-26T06:35:00.2353043Z 47d89094b1a7: Verifying Checksum 2024-06-26T06:35:00.2353564Z 47d89094b1a7: Download complete 2024-06-26T06:35:00.4228101Z 94beee2c356a: Verifying Checksum 2024-06-26T06:35:00.4228647Z 94beee2c356a: Download complete 2024-06-26T06:35:00.4947620Z 322c644984f4: Verifying Checksum 2024-06-26T06:35:00.4948178Z 322c644984f4: Download complete 2024-06-26T06:35:01.1082238Z b43ee04d974d: Verifying Checksum 2024-06-26T06:35:01.1082777Z b43ee04d974d: Download complete 2024-06-26T06:35:01.2174736Z 40f95cbeec2c: Verifying Checksum 2024-06-26T06:35:01.2175274Z 40f95cbeec2c: Download complete 2024-06-26T06:35:06.8871402Z 8f6c90485347: Pull complete 2024-06-26T06:35:06.9303827Z ab17245097e4: Pull complete 2024-06-26T06:35:06.9823692Z dfecd7e9912b: Pull complete 2024-06-26T06:35:07.0509567Z 464a8f745445: Pull complete 2024-06-26T06:35:11.3098703Z a05e0234ee85: Verifying Checksum 2024-06-26T06:35:11.3099522Z a05e0234ee85: Download complete 2024-06-26T06:35:11.3875320Z b2689866a0b9: Download complete 2024-06-26T06:35:11.4692386Z 9b36f9d4ace6: Verifying Checksum 2024-06-26T06:35:11.4692981Z 9b36f9d4ace6: Download complete 2024-06-26T06:35:12.0016545Z a863a1c01885: Verifying Checksum 2024-06-26T06:35:12.0017596Z a863a1c01885: Download complete 2024-06-26T06:35:12.0747955Z b9b8edd214be: Verifying Checksum 2024-06-26T06:35:12.0749042Z b9b8edd214be: Download complete 2024-06-26T06:35:15.0235506Z 4b2a3c41d6ae: Verifying Checksum 2024-06-26T06:35:15.0236111Z 4b2a3c41d6ae: Download complete 2024-06-26T06:35:17.5257827Z 9777c07909ad: Verifying Checksum 2024-06-26T06:35:17.5258432Z 9777c07909ad: Download complete 2024-06-26T06:35:45.4212028Z d67c111fa588: Pull complete 2024-06-26T06:35:47.7063865Z 0723abde19c0: Pull complete 2024-06-26T06:35:50.1496475Z bb93d41d4cb4: Pull complete 2024-06-26T06:36:06.7109373Z 4b89839dd066: Pull complete 2024-06-26T06:36:09.3913195Z caa9986c765b: Pull complete 2024-06-26T06:36:11.8346953Z e17c09bdf243: Pull complete 2024-06-26T06:36:14.2401360Z bd2e32de75c9: Pull complete 2024-06-26T06:36:19.3870565Z 559a1daa0f98: Pull complete 2024-06-26T06:36:21.7970607Z d96cec7a425c: Pull complete 2024-06-26T06:36:24.2802408Z 1db6fc6c2590: Pull complete 2024-06-26T06:36:26.9729950Z ac522bc70345: Pull complete 2024-06-26T06:37:32.8569934Z a05e0234ee85: Pull complete 2024-06-26T06:37:36.8232546Z 44f27f91d8fa: Pull complete 2024-06-26T06:37:41.2924117Z 0a0ded106167: Pull complete 2024-06-26T06:37:44.0372356Z 0c62bfb80b7c: Pull complete 2024-06-26T06:37:47.0095208Z 0c8d1e7f3282: Pull complete 2024-06-26T06:37:49.9612874Z 846366e78885: Pull complete 2024-06-26T06:37:55.3613178Z efa11c713d18: Pull complete 2024-06-26T06:37:55.3991934Z 32f0221befeb: Pull complete 2024-06-26T06:37:55.4372330Z a1d57c94ba33: Pull complete 2024-06-26T06:37:55.4707642Z 3dc15441e6d0: Pull complete 2024-06-26T06:37:55.5108695Z 668eadf990e8: Pull complete 2024-06-26T06:37:55.5599579Z fd5a832fe26e: Pull complete 2024-06-26T06:38:07.3418458Z b581c3782fd6: Pull complete 2024-06-26T06:38:07.3837913Z 3f42611c5636: Pull complete 2024-06-26T06:38:07.4267581Z fe8dd930eda1: Pull complete 2024-06-26T06:38:08.4045232Z 6a9711a8a50d: Pull complete 2024-06-26T06:38:08.4436735Z 76cd931663c9: Pull complete 2024-06-26T06:38:08.4917982Z c9ec95615433: Pull complete 2024-06-26T06:38:08.8770007Z 7da33ab03193: Pull complete 2024-06-26T06:38:08.9209082Z da956bbe6180: Pull complete 2024-06-26T06:38:09.0365625Z 73e6b5984544: Pull complete 2024-06-26T06:38:09.0756563Z e5a6ec0a41d7: Pull complete 2024-06-26T06:38:09.1295560Z 4f4fb700ef54: Pull complete 2024-06-26T06:38:09.1785811Z 4b25a744b402: Pull complete 2024-06-26T06:38:09.2349502Z 21f74857bd17: Pull complete 2024-06-26T06:38:11.7362757Z 6c12f547d202: Pull complete 2024-06-26T06:38:11.7754738Z ed4e5a4f8526: Pull complete 2024-06-26T06:38:11.8237506Z 6f09f741ead6: Pull complete 2024-06-26T06:38:11.9322169Z 411fce2db725: Pull complete 2024-06-26T06:38:11.9760957Z 87f7902d12a4: Pull complete 2024-06-26T06:38:56.9010207Z 9777c07909ad: Pull complete 2024-06-26T06:38:59.3416701Z 9755a952d3ec: Pull complete 2024-06-26T06:39:01.7818502Z 5ca5d2e8bebe: Pull complete 2024-06-26T06:39:06.7475538Z d07af843db47: Pull complete 2024-06-26T06:39:11.7248434Z dbbd7a4ca846: Pull complete 2024-06-26T06:39:14.8219628Z 6a18c61a96dd: Pull complete 2024-06-26T06:39:17.4126390Z bd12222c0237: Pull complete 2024-06-26T06:39:20.2856005Z a5dfe6ba36fd: Pull complete 2024-06-26T06:39:22.5712464Z 4b990eb76e8f: Pull complete 2024-06-26T06:39:25.3572684Z d9df85e8d422: Pull complete 2024-06-26T06:39:25.5509646Z 47d89094b1a7: Pull complete 2024-06-26T06:39:26.7151639Z 94beee2c356a: Pull complete 2024-06-26T06:39:26.7510779Z 322c644984f4: Pull complete 2024-06-26T06:39:29.0556117Z b43ee04d974d: Pull complete 2024-06-26T06:39:29.0928910Z 40f95cbeec2c: Pull complete 2024-06-26T06:39:45.8284751Z 4b2a3c41d6ae: Pull complete 2024-06-26T06:39:45.8850130Z b2689866a0b9: Pull complete 2024-06-26T06:39:45.9277444Z 9b36f9d4ace6: Pull complete 2024-06-26T06:39:46.6094383Z a863a1c01885: Pull complete 2024-06-26T06:39:46.6512467Z b9b8edd214be: Pull complete 2024-06-26T06:39:46.9325820Z Digest: sha256:493391c3b293fbc39c8ca4208a5b70ab17031329cb9c855a641ab5a208c52fe6 2024-06-26T06:39:46.9360122Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:39:46.9390293Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:39:46.9444275Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T06:39:46.9445465Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T06:39:46.9454391Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:39:46.9455028Z env: 2024-06-26T06:39:46.9455351Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:39:46.9455780Z ##[endgroup] 2024-06-26T06:39:46.9658502Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-06-26T06:39:46.9659201Z with: 2024-06-26T06:39:46.9659548Z driver-version: 550.54.15 2024-06-26T06:39:46.9659973Z env: 2024-06-26T06:39:46.9660291Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:39:46.9660717Z ##[endgroup] 2024-06-26T06:39:46.9715206Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T06:39:46.9715920Z with: 2024-06-26T06:39:46.9716251Z timeout_minutes: 10 2024-06-26T06:39:46.9716656Z max_attempts: 3 2024-06-26T06:39:46.9758823Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" 2024-06-26T06:39:46.9801233Z retry_wait_seconds: 10 2024-06-26T06:39:46.9801692Z polling_interval_seconds: 1 2024-06-26T06:39:46.9802143Z warning_on_retry: true 2024-06-26T06:39:46.9802574Z continue_on_error: false 2024-06-26T06:39:46.9802992Z env: 2024-06-26T06:39:46.9803307Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:39:46.9803760Z DRIVER_VERSION: 550.54.15 2024-06-26T06:39:46.9804192Z ##[endgroup] 2024-06-26T06:39:47.0566486Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-06-26T06:39:47.0568399Z + pre_install_nvidia_driver_amzn2 2024-06-26T06:39:47.0569067Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-06-26T06:39:47.5674378Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T06:39:47.6240370Z No Match for argument: nvidia-driver-latest-dkms 2024-06-26T06:39:47.6612230Z No Packages marked for removal 2024-06-26T06:39:47.6769597Z + install_nvidia_driver_common 2024-06-26T06:39:47.6773270Z + echo 'Before installing NVIDIA driver' 2024-06-26T06:39:47.6774334Z + lspci 2024-06-26T06:39:47.6776134Z Before installing NVIDIA driver 2024-06-26T06:39:47.7009344Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-06-26T06:39:47.7010331Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T06:39:47.7011360Z 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 2024-06-26T06:39:47.7012563Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 2024-06-26T06:39:47.7013435Z 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 2024-06-26T06:39:47.7014279Z 00:03.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T06:39:47.7015298Z 00:1d.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-06-26T06:39:47.7016673Z 00:1e.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-06-26T06:39:47.7017708Z 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 2024-06-26T06:39:47.7018447Z + lsmod 2024-06-26T06:39:47.7033235Z Module Size Used by 2024-06-26T06:39:47.7033769Z xt_nat 16384 0 2024-06-26T06:39:47.7034211Z ib_core 266240 0 2024-06-26T06:39:47.7034677Z nvidia_modeset 1339392 0 2024-06-26T06:39:47.7035279Z veth 16384 0 2024-06-26T06:39:47.7035789Z nvidia_uvm 4599808 0 2024-06-26T06:39:47.7036344Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T06:39:47.7036990Z drm 421888 1 nvidia 2024-06-26T06:39:47.7037517Z i2c_core 77824 2 nvidia,drm 2024-06-26T06:39:47.7038039Z backlight 16384 1 nvidia_modeset 2024-06-26T06:39:47.7038613Z xt_conntrack 16384 1 2024-06-26T06:39:47.7039082Z ipt_MASQUERADE 16384 1 2024-06-26T06:39:47.7039600Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T06:39:47.7040182Z nf_conntrack_netlink 49152 0 2024-06-26T06:39:47.7041276Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T06:39:47.7042502Z xfrm_user 45056 1 2024-06-26T06:39:47.7043337Z xfrm_algo 16384 1 xfrm_user 2024-06-26T06:39:47.7043844Z iptable_nat 16384 1 2024-06-26T06:39:47.7044223Z nf_conntrack_ipv4 16384 3 2024-06-26T06:39:47.7045236Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T06:39:47.7046438Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T06:39:47.7047727Z nf_nat 32768 3 nf_nat_masquerade_ipv4,nf_nat_ipv4,xt_nat 2024-06-26T06:39:47.7050308Z nf_conntrack 155648 8 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,xt_nat,nf_conntrack_netlink 2024-06-26T06:39:47.7051377Z xt_addrtype 16384 2 2024-06-26T06:39:47.7052338Z iptable_filter 16384 1 2024-06-26T06:39:47.7052755Z br_netfilter 24576 0 2024-06-26T06:39:47.7053231Z bridge 172032 1 br_netfilter 2024-06-26T06:39:47.7053843Z stp 16384 1 bridge 2024-06-26T06:39:47.7054367Z llc 16384 2 bridge,stp 2024-06-26T06:39:47.7054844Z overlay 86016 0 2024-06-26T06:39:47.7055300Z sunrpc 393216 1 2024-06-26T06:39:47.7055836Z dm_mirror 28672 0 2024-06-26T06:39:47.7056331Z dm_region_hash 20480 1 dm_mirror 2024-06-26T06:39:47.7056827Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T06:39:47.7057458Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T06:39:47.7058010Z dax 69632 1 dm_mod 2024-06-26T06:39:47.7058482Z sb_edac 24576 0 2024-06-26T06:39:47.7058938Z crc32_pclmul 16384 0 2024-06-26T06:39:47.7059409Z ghash_clmulni_intel 16384 0 2024-06-26T06:39:47.7059861Z pcbc 16384 0 2024-06-26T06:39:47.7060328Z aesni_intel 188416 0 2024-06-26T06:39:47.7060815Z aes_x86_64 20480 1 aesni_intel 2024-06-26T06:39:47.7061319Z mousedev 24576 0 2024-06-26T06:39:47.7061807Z crypto_simd 16384 1 aesni_intel 2024-06-26T06:39:47.7062369Z glue_helper 16384 1 aesni_intel 2024-06-26T06:39:47.7063049Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T06:39:47.7063729Z pcc_cpufreq 16384 0 2024-06-26T06:39:47.7064320Z psmouse 32768 0 2024-06-26T06:39:47.7064772Z evdev 20480 3 2024-06-26T06:39:47.7065221Z button 16384 0 2024-06-26T06:39:47.7065672Z ena 139264 0 2024-06-26T06:39:47.7066246Z ptp 20480 1 ena 2024-06-26T06:39:47.7066741Z pps_core 20480 1 ptp 2024-06-26T06:39:47.7067230Z ata_piix 36864 0 2024-06-26T06:39:47.7067694Z libata 266240 1 ata_piix 2024-06-26T06:39:47.7068419Z xen_blkfront 49152 2 2024-06-26T06:39:47.7068880Z crc32c_intel 24576 0 2024-06-26T06:39:47.7069331Z scsi_mod 245760 1 libata 2024-06-26T06:39:47.7069823Z autofs4 49152 2 2024-06-26T06:39:47.7070268Z + modinfo nvidia 2024-06-26T06:39:47.7071083Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T06:39:47.7071976Z alias: char-major-195-* 2024-06-26T06:39:47.7072448Z version: 550.54.15 2024-06-26T06:39:47.7072981Z supported: external 2024-06-26T06:39:47.7073423Z license: NVIDIA 2024-06-26T06:39:47.7073889Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T06:39:47.7074474Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T06:39:47.7075044Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T06:39:47.7075621Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T06:39:47.7076224Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T06:39:47.7076843Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T06:39:47.7077465Z depends: i2c-core,drm 2024-06-26T06:39:47.7077893Z retpoline: Y 2024-06-26T06:39:47.7078267Z name: nvidia 2024-06-26T06:39:47.7078966Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T06:39:47.7079843Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T06:39:47.7080640Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T06:39:47.7081400Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T06:39:47.7081957Z parm: NVreg_RmLogonRC:int 2024-06-26T06:39:47.7082474Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T06:39:47.7083039Z parm: NVreg_DeviceFileUID:int 2024-06-26T06:39:47.7083580Z parm: NVreg_DeviceFileGID:int 2024-06-26T06:39:47.7084107Z parm: NVreg_DeviceFileMode:int 2024-06-26T06:39:47.7084760Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T06:39:47.7085473Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T06:39:47.7086052Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T06:39:47.7086584Z parm: NVreg_EnableMSI:int 2024-06-26T06:39:47.7087096Z parm: NVreg_TCEBypassMode:int 2024-06-26T06:39:47.7087646Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T06:39:47.7088304Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T06:39:47.7089035Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T06:39:47.7089727Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T06:39:47.7090492Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T06:39:47.7091247Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T06:39:47.7092325Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T06:39:47.7093076Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T06:39:47.7093690Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T06:39:47.7094351Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T06:39:47.7095030Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T06:39:47.7095645Z parm: NVreg_MemoryPoolSize:int 2024-06-26T06:39:47.7096197Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T06:39:47.7096797Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T06:39:47.7097378Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T06:39:47.7097917Z parm: NVreg_NvLinkDisable:int 2024-06-26T06:39:47.7098548Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T06:39:47.7099205Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T06:39:47.7099800Z parm: NVreg_EnableResizableBar:int 2024-06-26T06:39:47.7100391Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T06:39:47.7101014Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T06:39:47.7101623Z parm: NVreg_RegistryDwords:charp 2024-06-26T06:39:47.7102400Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T06:39:47.7103001Z parm: NVreg_RmMsg:charp 2024-06-26T06:39:47.7103507Z parm: NVreg_GpuBlacklist:charp 2024-06-26T06:39:47.7104064Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T06:39:47.7104646Z parm: NVreg_ExcludedGpus:charp 2024-06-26T06:39:47.7105206Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T06:39:47.7105786Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T06:39:47.7106488Z parm: NVreg_ImexChannelCount:int 2024-06-26T06:39:47.7107075Z parm: rm_firmware_active:charp 2024-06-26T06:39:47.7107572Z + HAS_NVIDIA_DRIVER=0 2024-06-26T06:39:47.7108053Z ++ command -v nvidia-smi 2024-06-26T06:39:47.7108551Z + '[' -x /usr/bin/nvidia-smi ']' 2024-06-26T06:39:47.7108989Z + set +e 2024-06-26T06:39:47.7109626Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-06-26T06:39:49.8584378Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-06-26T06:39:49.8585547Z + NVIDIA_SMI_STATUS=0 2024-06-26T06:39:49.8586296Z + '[' 0 -ne 0 ']' 2024-06-26T06:39:49.8586749Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-06-26T06:39:49.8587222Z + HAS_NVIDIA_DRIVER=1 2024-06-26T06:39:49.8588357Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-06-26T06:39:49.8589886Z + set -e 2024-06-26T06:39:49.8590268Z + '[' 1 -eq 0 ']' 2024-06-26T06:39:49.8590975Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-06-26T06:39:49.8591864Z + post_install_nvidia_driver_common 2024-06-26T06:39:49.8592376Z + sudo modprobe nvidia 2024-06-26T06:39:49.8735773Z + echo 'After installing NVIDIA driver' 2024-06-26T06:39:49.8738676Z After installing NVIDIA driver 2024-06-26T06:39:49.8739585Z + lspci 2024-06-26T06:39:49.8956429Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-06-26T06:39:49.8957557Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T06:39:49.8958578Z 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 2024-06-26T06:39:49.8959526Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 2024-06-26T06:39:49.8960323Z 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 2024-06-26T06:39:49.8961186Z 00:03.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T06:39:49.8962221Z 00:1d.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-06-26T06:39:49.8963294Z 00:1e.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-06-26T06:39:49.8964344Z 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 2024-06-26T06:39:49.8965081Z + lsmod 2024-06-26T06:39:49.8979525Z Module Size Used by 2024-06-26T06:39:49.8980413Z xt_nat 16384 0 2024-06-26T06:39:49.8980886Z ib_core 266240 0 2024-06-26T06:39:49.8981332Z nvidia_modeset 1339392 0 2024-06-26T06:39:49.8981811Z veth 16384 0 2024-06-26T06:39:49.8982276Z nvidia_uvm 4599808 0 2024-06-26T06:39:49.8982812Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T06:39:49.8983443Z drm 421888 1 nvidia 2024-06-26T06:39:49.8983962Z i2c_core 77824 2 nvidia,drm 2024-06-26T06:39:49.8984537Z backlight 16384 1 nvidia_modeset 2024-06-26T06:39:49.8985071Z xt_conntrack 16384 1 2024-06-26T06:39:49.8985539Z ipt_MASQUERADE 16384 1 2024-06-26T06:39:49.8986080Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T06:39:49.8986647Z nf_conntrack_netlink 49152 0 2024-06-26T06:39:49.8987199Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T06:39:49.8987774Z xfrm_user 45056 1 2024-06-26T06:39:49.8988238Z xfrm_algo 16384 1 xfrm_user 2024-06-26T06:39:49.8988754Z iptable_nat 16384 1 2024-06-26T06:39:49.8989220Z nf_conntrack_ipv4 16384 3 2024-06-26T06:39:49.8990006Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T06:39:49.8990622Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T06:39:49.8991893Z nf_nat 32768 3 nf_nat_masquerade_ipv4,nf_nat_ipv4,xt_nat 2024-06-26T06:39:49.8993278Z nf_conntrack 155648 8 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,xt_nat,nf_conntrack_netlink 2024-06-26T06:39:49.8994389Z xt_addrtype 16384 2 2024-06-26T06:39:49.8994862Z iptable_filter 16384 1 2024-06-26T06:39:49.8995883Z br_netfilter 24576 0 2024-06-26T06:39:49.8996672Z bridge 172032 1 br_netfilter 2024-06-26T06:39:49.8997225Z stp 16384 1 bridge 2024-06-26T06:39:49.8997723Z llc 16384 2 bridge,stp 2024-06-26T06:39:49.8998240Z overlay 86016 0 2024-06-26T06:39:49.8998696Z sunrpc 393216 1 2024-06-26T06:39:49.8999134Z dm_mirror 28672 0 2024-06-26T06:39:49.8999620Z dm_region_hash 20480 1 dm_mirror 2024-06-26T06:39:49.9000237Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T06:39:49.9000855Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T06:39:49.9001404Z dax 69632 1 dm_mod 2024-06-26T06:39:49.9001890Z sb_edac 24576 0 2024-06-26T06:39:49.9002330Z crc32_pclmul 16384 0 2024-06-26T06:39:49.9002799Z ghash_clmulni_intel 16384 0 2024-06-26T06:39:49.9003268Z pcbc 16384 0 2024-06-26T06:39:49.9003712Z aesni_intel 188416 0 2024-06-26T06:39:49.9004199Z aes_x86_64 20480 1 aesni_intel 2024-06-26T06:39:49.9004719Z mousedev 24576 0 2024-06-26T06:39:49.9005188Z crypto_simd 16384 1 aesni_intel 2024-06-26T06:39:49.9005747Z glue_helper 16384 1 aesni_intel 2024-06-26T06:39:49.9006588Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T06:39:49.9007272Z pcc_cpufreq 16384 0 2024-06-26T06:39:49.9007733Z psmouse 32768 0 2024-06-26T06:39:49.9008313Z evdev 20480 3 2024-06-26T06:39:49.9008763Z button 16384 0 2024-06-26T06:39:49.9009192Z ena 139264 0 2024-06-26T06:39:49.9009642Z ptp 20480 1 ena 2024-06-26T06:39:49.9010129Z pps_core 20480 1 ptp 2024-06-26T06:39:49.9010593Z ata_piix 36864 0 2024-06-26T06:39:49.9011068Z libata 266240 1 ata_piix 2024-06-26T06:39:49.9011582Z xen_blkfront 49152 2 2024-06-26T06:39:49.9012297Z crc32c_intel 24576 0 2024-06-26T06:39:49.9012770Z scsi_mod 245760 1 libata 2024-06-26T06:39:49.9013263Z autofs4 49152 2 2024-06-26T06:39:49.9013694Z + modinfo nvidia 2024-06-26T06:39:49.9014566Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T06:39:49.9015462Z alias: char-major-195-* 2024-06-26T06:39:49.9015914Z version: 550.54.15 2024-06-26T06:39:49.9016347Z supported: external 2024-06-26T06:39:49.9016763Z license: NVIDIA 2024-06-26T06:39:49.9017207Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T06:39:49.9017808Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T06:39:49.9018376Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T06:39:49.9018932Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T06:39:49.9019551Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T06:39:49.9020169Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T06:39:49.9020765Z depends: i2c-core,drm 2024-06-26T06:39:49.9021217Z retpoline: Y 2024-06-26T06:39:49.9021589Z name: nvidia 2024-06-26T06:39:49.9022268Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T06:39:49.9023092Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T06:39:49.9023902Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T06:39:49.9024824Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T06:39:49.9025378Z parm: NVreg_RmLogonRC:int 2024-06-26T06:39:49.9025913Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T06:39:49.9026461Z parm: NVreg_DeviceFileUID:int 2024-06-26T06:39:49.9027001Z parm: NVreg_DeviceFileGID:int 2024-06-26T06:39:49.9027548Z parm: NVreg_DeviceFileMode:int 2024-06-26T06:39:49.9028204Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T06:39:49.9029019Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T06:39:49.9029635Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T06:39:49.9030166Z parm: NVreg_EnableMSI:int 2024-06-26T06:39:49.9030662Z parm: NVreg_TCEBypassMode:int 2024-06-26T06:39:49.9031227Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T06:39:49.9031883Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T06:39:49.9032592Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T06:39:49.9033304Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T06:39:49.9034066Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T06:39:49.9034798Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T06:39:49.9035565Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T06:39:49.9036308Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T06:39:49.9036900Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T06:39:49.9037569Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T06:39:49.9038253Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T06:39:49.9038854Z parm: NVreg_MemoryPoolSize:int 2024-06-26T06:39:49.9039422Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T06:39:49.9040022Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T06:39:49.9040591Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T06:39:49.9041142Z parm: NVreg_NvLinkDisable:int 2024-06-26T06:39:49.9041775Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T06:39:49.9042412Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T06:39:49.9043007Z parm: NVreg_EnableResizableBar:int 2024-06-26T06:39:49.9043611Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T06:39:49.9044214Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T06:39:49.9044818Z parm: NVreg_RegistryDwords:charp 2024-06-26T06:39:49.9045438Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T06:39:49.9046024Z parm: NVreg_RmMsg:charp 2024-06-26T06:39:49.9046537Z parm: NVreg_GpuBlacklist:charp 2024-06-26T06:39:49.9047115Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T06:39:49.9047682Z parm: NVreg_ExcludedGpus:charp 2024-06-26T06:39:49.9048247Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T06:39:49.9048838Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T06:39:49.9049411Z parm: NVreg_ImexChannelCount:int 2024-06-26T06:39:49.9049982Z parm: rm_firmware_active:charp 2024-06-26T06:39:49.9050476Z + set +e 2024-06-26T06:39:49.9050832Z + nvidia-smi 2024-06-26T06:39:51.2655335Z Wed Jun 26 06:39:51 2024 2024-06-26T06:39:51.2656276Z +-----------------------------------------------------------------------------------------+ 2024-06-26T06:39:51.2657102Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-06-26T06:39:51.2658141Z |-----------------------------------------+------------------------+----------------------+ 2024-06-26T06:39:51.2659159Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-06-26T06:39:51.2660300Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-06-26T06:39:51.2661163Z | | | MIG M. | 2024-06-26T06:39:51.2661854Z |=========================================+========================+======================| 2024-06-26T06:39:51.2703583Z | 0 Tesla M60 Off | 00000000:00:1D.0 Off | 0 | 2024-06-26T06:39:51.2704438Z | N/A 34C P0 39W / 150W | 0MiB / 7680MiB | 99% Default | 2024-06-26T06:39:51.2705109Z | | | N/A | 2024-06-26T06:39:51.2705962Z +-----------------------------------------+------------------------+----------------------+ 2024-06-26T06:39:51.2751287Z | 1 Tesla M60 Off | 00000000:00:1E.0 Off | 0 | 2024-06-26T06:39:51.2752157Z | N/A 40C P0 38W / 150W | 0MiB / 7680MiB | 6% Default | 2024-06-26T06:39:51.2752820Z | | | N/A | 2024-06-26T06:39:51.2753681Z +-----------------------------------------+------------------------+----------------------+ 2024-06-26T06:39:51.2754434Z 2024-06-26T06:39:51.2755319Z +-----------------------------------------------------------------------------------------+ 2024-06-26T06:39:51.2756072Z | Processes: | 2024-06-26T06:39:51.2757097Z | GPU GI CI PID Type Process name GPU Memory | 2024-06-26T06:39:51.2757894Z | ID ID Usage | 2024-06-26T06:39:51.2758628Z |=========================================================================================| 2024-06-26T06:39:51.2759357Z | No running processes found | 2024-06-26T06:39:51.2760331Z +-----------------------------------------------------------------------------------------+ 2024-06-26T06:39:52.1062779Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-06-26T06:39:53.4689454Z Tesla M60 2024-06-26T06:39:54.2610346Z + NVIDIA_SMI_STATUS=0 2024-06-26T06:39:54.2611074Z + '[' 0 -eq 0 ']' 2024-06-26T06:39:54.2611483Z + echo 'INFO: Ignoring allowed status 0' 2024-06-26T06:39:54.2612630Z + set -e 2024-06-26T06:39:54.2613190Z INFO: Ignoring allowed status 0 2024-06-26T06:39:54.2617493Z == Installing nvidia container toolkit for amzn2 == 2024-06-26T06:39:54.2621161Z + sudo yum install -y yum-utils 2024-06-26T06:39:54.7708051Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T06:39:56.4399407Z Package yum-utils-1.1.31-46.amzn2.0.1.noarch already installed and latest version 2024-06-26T06:39:56.4400305Z Nothing to do 2024-06-26T06:39:56.5811290Z + [[ amzn2 == \a\m\z\n\2\0\2\3 ]] 2024-06-26T06:39:56.5812911Z + YUM_REPO_URL=https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T06:39:56.5814286Z + sudo yum-config-manager --add-repo https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T06:39:57.1060146Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T06:39:57.1377956Z adding repo from: https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T06:39:57.1379460Z grabbing file https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T06:39:57.1380740Z repo saved to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T06:39:57.1518997Z + sudo yum install -y nvidia-docker2 2024-06-26T06:39:57.6697453Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T06:39:59.3269318Z Package nvidia-docker2-2.13.0-1.noarch already installed and latest version 2024-06-26T06:39:59.3270147Z Nothing to do 2024-06-26T06:39:59.4670189Z + sudo systemctl restart docker 2024-06-26T06:40:19.0893797Z Command completed after 1 attempt(s). 2024-06-26T06:40:19.0972380Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T06:40:19.0973423Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T06:40:19.0974532Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-06-26T06:40:19.0975344Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-06-26T06:40:19.0983974Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:40:19.0984609Z env: 2024-06-26T06:40:19.0984951Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:40:19.0985493Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:40:19.0986106Z ##[endgroup] 2024-06-26T06:40:19.4046581Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T06:40:19.4263815Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.7/site-packages (5.9.1) 2024-06-26T06:40:19.4391816Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.7/site-packages (11.525.84) 2024-06-26T06:40:19.5742690Z Prepare all required actions 2024-06-26T06:40:19.5743378Z Getting action download info 2024-06-26T06:40:19.6920987Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-06-26T06:40:19.9168749Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-06-26T06:40:20.0449985Z ##[group]Run ./.github/actions/download-build-artifacts 2024-06-26T06:40:20.0450592Z with: 2024-06-26T06:40:20.0450962Z name: linux-focal-cuda11.8-py3.10-gcc9 2024-06-26T06:40:20.0451506Z s3-bucket: gha-artifacts 2024-06-26T06:40:20.0452403Z env: 2024-06-26T06:40:20.0452721Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:40:20.0453284Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:40:20.0453886Z ##[endgroup] 2024-06-26T06:40:20.0497899Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T06:40:20.0498437Z with: 2024-06-26T06:40:20.0498890Z name: linux-focal-cuda11.8-py3.10-gcc9 2024-06-26T06:40:20.0499416Z s3-bucket: gha-artifacts 2024-06-26T06:40:20.0499870Z region: us-east-1 2024-06-26T06:40:20.0500233Z env: 2024-06-26T06:40:20.0500544Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:40:20.0501105Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:40:20.0501706Z ##[endgroup] 2024-06-26T06:40:20.6584548Z (node:101103) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T06:40:20.6585459Z 2024-06-26T06:40:20.6586246Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T06:40:20.6587157Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T06:40:20.6588221Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T06:40:20.8153679Z Found 1 objects with prefix pytorch/pytorch/9673645538/linux-focal-cuda11.8-py3.10-gcc9/ 2024-06-26T06:40:20.8155079Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T06:40:42.8696614Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T06:40:42.8712221Z Artifact download has finished successfully 2024-06-26T06:40:42.8911114Z ##[group]Run unzip -o artifacts.zip 2024-06-26T06:40:42.8911653Z unzip -o artifacts.zip 2024-06-26T06:40:42.8920062Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:40:42.8920689Z env: 2024-06-26T06:40:42.8921028Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:40:42.8921571Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:40:42.8922170Z ##[endgroup] 2024-06-26T06:40:42.8960963Z Archive: artifacts.zip 2024-06-26T06:40:42.8963344Z creating: dist/ 2024-06-26T06:40:45.3451324Z inflating: dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T06:40:45.3452852Z creating: build/custom_test_artifacts/ 2024-06-26T06:40:45.3453606Z creating: build/custom_test_artifacts/custom-op-build/ 2024-06-26T06:40:45.3454538Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-06-26T06:40:45.3455932Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-06-26T06:40:45.3462268Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T06:40:45.3464414Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-06-26T06:40:45.3465630Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T06:40:45.3467845Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T06:40:45.3469851Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T06:40:45.3473026Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T06:40:45.3475553Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T06:40:45.3478056Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T06:40:45.3480035Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T06:40:45.3482798Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T06:40:45.3484361Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T06:40:45.3485834Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T06:40:45.3487304Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T06:40:45.3488799Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T06:40:45.3490293Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T06:40:45.3491934Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T06:40:45.3493278Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T06:40:45.3542891Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T06:40:45.3547149Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T06:40:45.3549839Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T06:40:45.3552090Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T06:40:45.3554018Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T06:40:45.3555802Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T06:40:45.3557638Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T06:40:45.3559453Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T06:40:45.3561258Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T06:40:45.3599711Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T06:40:45.3650516Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T06:40:45.3655149Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T06:40:45.3658851Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T06:40:45.3660494Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T06:40:45.3662075Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T06:40:45.3663669Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T06:40:45.3665213Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T06:40:45.3666780Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T06:40:45.3735246Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T06:40:45.3811522Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T06:40:45.3815316Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T06:40:45.3818168Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-06-26T06:40:45.3820680Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-06-26T06:40:45.3821854Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-06-26T06:40:45.3823042Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-06-26T06:40:45.3824362Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-06-26T06:40:45.3825867Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-06-26T06:40:45.3827288Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-06-26T06:40:45.3828636Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-06-26T06:40:45.3830021Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-06-26T06:40:45.3831393Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-06-26T06:40:45.3832789Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-06-26T06:40:45.3834177Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-06-26T06:40:45.3835545Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-06-26T06:40:45.3846943Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-06-26T06:40:45.4015707Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-06-26T06:40:45.4018600Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-06-26T06:40:45.4021819Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-06-26T06:40:45.4023905Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-06-26T06:40:45.4025414Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-06-26T06:40:45.4026814Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-06-26T06:40:45.4028266Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-06-26T06:40:45.4029738Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-06-26T06:40:45.4031207Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-06-26T06:40:45.4032880Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-06-26T06:40:45.4034297Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-06-26T06:40:45.4049041Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-06-26T06:40:45.4153919Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-06-26T06:40:45.4157329Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T06:40:45.4159653Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-06-26T06:40:45.4161135Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-06-26T06:40:45.4163836Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-06-26T06:40:45.4165254Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-06-26T06:40:45.4166416Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-06-26T06:40:45.4167505Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-06-26T06:40:45.4168506Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-06-26T06:40:45.4169523Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-06-26T06:40:45.4305621Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-06-26T06:40:45.4384582Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-06-26T06:40:45.4386027Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-06-26T06:40:45.4386927Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-06-26T06:40:45.4387955Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-06-26T06:40:45.4395342Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T06:40:45.4397360Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-06-26T06:40:45.4398807Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T06:40:45.4400929Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T06:40:45.4403138Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T06:40:45.4404862Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T06:40:45.4407477Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T06:40:45.4409675Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T06:40:45.4412086Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T06:40:45.4415324Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T06:40:45.4416807Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T06:40:45.4418257Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T06:40:45.4419713Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T06:40:45.4421195Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T06:40:45.4422661Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T06:40:45.4423986Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T06:40:45.4425556Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T06:40:45.4474467Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T06:40:45.4478658Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T06:40:45.4481533Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T06:40:45.4483443Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T06:40:45.4485348Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T06:40:45.4487312Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T06:40:45.4489167Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T06:40:45.4490974Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T06:40:45.4493061Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T06:40:45.4531465Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T06:40:45.4582296Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T06:40:45.4586415Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T06:40:45.4589321Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T06:40:45.4591470Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T06:40:45.4593038Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T06:40:45.4594629Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T06:40:45.4596138Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T06:40:45.4597696Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T06:40:45.4666469Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T06:40:45.4743347Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T06:40:45.4746761Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T06:40:45.4749584Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-06-26T06:40:45.4752093Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-06-26T06:40:45.4753351Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-06-26T06:40:45.4754544Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-06-26T06:40:45.4755916Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-06-26T06:40:45.4757467Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-06-26T06:40:45.4758932Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-06-26T06:40:45.4760536Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-06-26T06:40:45.4761959Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-06-26T06:40:45.4763386Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-06-26T06:40:45.4764807Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-06-26T06:40:45.4766230Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-06-26T06:40:45.4767639Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-06-26T06:40:45.4779064Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-06-26T06:40:45.4860267Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-06-26T06:40:45.4863389Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T06:40:45.4865598Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-06-26T06:40:45.4866821Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-06-26T06:40:45.4869247Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-06-26T06:40:45.4871017Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-06-26T06:40:45.4872174Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-06-26T06:40:45.4873228Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-06-26T06:40:45.4874213Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-06-26T06:40:45.4875244Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-06-26T06:40:45.4932923Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-06-26T06:40:45.4934443Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-06-26T06:40:45.4935410Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-06-26T06:40:45.4936544Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-06-26T06:40:45.4943537Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T06:40:45.4945581Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-06-26T06:40:45.4947333Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T06:40:45.4949702Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T06:40:45.4951643Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T06:40:45.4954036Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T06:40:45.4956556Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T06:40:45.4959217Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T06:40:45.4961690Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T06:40:45.4964018Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T06:40:45.4965614Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T06:40:45.4967149Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T06:40:45.4968703Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T06:40:45.4970485Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T06:40:45.4972346Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T06:40:45.4973784Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T06:40:45.4975156Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T06:40:45.5022639Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T06:40:45.5027083Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T06:40:45.5030150Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T06:40:45.5032198Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T06:40:45.5034177Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T06:40:45.5036042Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T06:40:45.5037948Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T06:40:45.5039853Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T06:40:45.5041751Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T06:40:45.5079313Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T06:40:45.5130043Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T06:40:45.5134166Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T06:40:45.5137439Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T06:40:45.5139878Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T06:40:45.5141556Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T06:40:45.5143244Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T06:40:45.5144888Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T06:40:45.5146530Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T06:40:45.5214935Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T06:40:45.5291329Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T06:40:45.5295283Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T06:40:45.5298348Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-06-26T06:40:45.5300719Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-06-26T06:40:45.5301979Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-06-26T06:40:45.5303471Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-06-26T06:40:45.5305422Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-06-26T06:40:45.5307525Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-06-26T06:40:45.5309108Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-06-26T06:40:45.5310564Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-06-26T06:40:45.5312082Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-06-26T06:40:45.5313760Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-06-26T06:40:45.5315328Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-06-26T06:40:45.5316854Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-06-26T06:40:45.5318372Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-06-26T06:40:45.5319989Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-06-26T06:40:45.5465187Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-06-26T06:40:45.5468591Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-06-26T06:40:45.5472024Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-06-26T06:40:45.5473757Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-06-26T06:40:45.5475394Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-06-26T06:40:45.5476946Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-06-26T06:40:45.5478559Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-06-26T06:40:45.5480167Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-06-26T06:40:45.5481763Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-06-26T06:40:45.5483368Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-06-26T06:40:45.5485014Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-06-26T06:40:45.5498712Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-06-26T06:40:45.5568966Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-06-26T06:40:45.5572819Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T06:40:45.5574972Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-06-26T06:40:45.5576606Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-06-26T06:40:45.5579381Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-06-26T06:40:45.5580676Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-06-26T06:40:45.5581924Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-06-26T06:40:45.5583294Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-06-26T06:40:45.5584415Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-06-26T06:40:45.5585519Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-06-26T06:40:45.5709482Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-06-26T06:40:45.5763263Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-06-26T06:40:45.5764250Z creating: build/lib/ 2024-06-26T06:40:45.5775766Z inflating: build/lib/libpthreadpool.a 2024-06-26T06:40:45.5777008Z inflating: build/lib/libclog.a 2024-06-26T06:40:45.5787242Z inflating: build/lib/libcpuinfo.a 2024-06-26T06:40:45.5797447Z inflating: build/lib/libcpuinfo_internals.a 2024-06-26T06:40:45.5800718Z inflating: build/lib/libnnpack_reference_layers.a 2024-06-26T06:40:45.5911722Z inflating: build/lib/libprotobuf-lite.a 2024-06-26T06:40:45.6480567Z inflating: build/lib/libprotobuf.a 2024-06-26T06:40:45.6562350Z inflating: build/lib/libgtest.a 2024-06-26T06:40:45.6656061Z inflating: build/lib/libbenchmark.a 2024-06-26T06:40:45.6665774Z inflating: build/lib/libittnotify.a 2024-06-26T06:40:45.6700898Z inflating: build/lib/libtensorpipe_uv.a 2024-06-26T06:40:45.6780502Z inflating: build/lib/libasmjit.a 2024-06-26T06:40:45.6939659Z inflating: build/lib/libgloo.a 2024-06-26T06:40:45.6966826Z inflating: build/lib/libfmt.a 2024-06-26T06:40:45.6969596Z inflating: build/lib/libcaffe2_nvrtc.so 2024-06-26T06:40:45.7084423Z inflating: build/lib/libc10.so 2024-06-26T06:40:45.7085599Z inflating: build/lib/libfoxi_loader.a 2024-06-26T06:40:45.7087577Z inflating: build/lib/libtorch_global_deps.so 2024-06-26T06:40:45.7112588Z inflating: build/lib/libpytorch_qnnpack.a 2024-06-26T06:40:45.7135260Z inflating: build/lib/libgmock.a 2024-06-26T06:40:45.7136496Z inflating: build/lib/libgtest_main.a 2024-06-26T06:40:45.7137795Z inflating: build/lib/libbenchmark_main.a 2024-06-26T06:40:45.7765859Z inflating: build/lib/libprotoc.a 2024-06-26T06:40:45.7788927Z inflating: build/lib/libnnpack.a 2024-06-26T06:40:47.0437456Z inflating: build/lib/libdnnl.a 2024-06-26T06:40:47.1151673Z inflating: build/lib/libtensorpipe.a 2024-06-26T06:40:47.1220103Z inflating: build/lib/libc10_cuda.so 2024-06-26T06:40:47.1221301Z inflating: build/lib/libgmock_main.a 2024-06-26T06:40:47.2782465Z inflating: build/lib/libfbgemm.a 2024-06-26T06:40:47.3424957Z inflating: build/lib/libkineto.a 2024-06-26T06:40:47.3744128Z inflating: build/lib/libtensorpipe_cuda.a 2024-06-26T06:40:47.3974181Z inflating: build/lib/libXNNPACK.a 2024-06-26T06:40:47.4459411Z inflating: build/lib/libgloo_cuda.a 2024-06-26T06:40:47.4512703Z inflating: build/lib/libonnx_proto.a 2024-06-26T06:40:47.5375265Z inflating: build/lib/libonnx.a 2024-06-26T06:40:50.5530671Z inflating: build/lib/libtorch_cpu.so 2024-06-26T06:40:50.5535516Z inflating: build/lib/libshm.so 2024-06-26T06:40:50.5541794Z inflating: build/lib/libunbox_lib.a 2024-06-26T06:40:53.0158230Z inflating: build/lib/libtorch_cuda.so 2024-06-26T06:40:53.0159365Z inflating: build/lib/libtorch.so 2024-06-26T06:40:53.0162732Z inflating: build/lib/libc10d_cuda_test.so 2024-06-26T06:40:54.1572915Z inflating: build/lib/libtorch_cuda_linalg.so 2024-06-26T06:40:54.1598278Z inflating: build/lib/libjitbackend_test.so 2024-06-26T06:40:54.1684283Z inflating: build/lib/libtorchbind_test.so 2024-06-26T06:40:54.1716534Z inflating: build/lib/libbackend_with_compiler.so 2024-06-26T06:40:54.4172472Z inflating: build/lib/libtorch_python.so 2024-06-26T06:40:54.4215209Z inflating: build/lib/libnnapi_backend.so 2024-06-26T06:40:54.4216399Z creating: build/bin/ 2024-06-26T06:40:54.4279076Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-06-26T06:40:54.4343966Z inflating: build/bin/c10_DeviceGuard_test 2024-06-26T06:40:54.4406613Z inflating: build/bin/c10_Device_test 2024-06-26T06:40:54.4480875Z inflating: build/bin/c10_DispatchKeySet_test 2024-06-26T06:40:54.4548189Z inflating: build/bin/c10_Scalar_test 2024-06-26T06:40:54.4609787Z inflating: build/bin/c10_StreamGuard_test 2024-06-26T06:40:54.4673777Z inflating: build/bin/c10_SymInt_test 2024-06-26T06:40:54.4742536Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-06-26T06:40:54.4812265Z inflating: build/bin/c10_InlineStreamGuard_test 2024-06-26T06:40:54.4882847Z inflating: build/bin/c10_SizesAndStrides_test 2024-06-26T06:40:54.4971913Z inflating: build/bin/c10_cow_test 2024-06-26T06:40:54.5038819Z inflating: build/bin/c10_Bitset_test 2024-06-26T06:40:54.5100608Z inflating: build/bin/c10_ConstexprCrc_test 2024-06-26T06:40:54.5163408Z inflating: build/bin/c10_DeadlockDetection_test 2024-06-26T06:40:54.5227104Z inflating: build/bin/c10_Half_test 2024-06-26T06:40:54.5297837Z inflating: build/bin/c10_LeftRight_test 2024-06-26T06:40:54.5366951Z inflating: build/bin/c10_Metaprogramming_test 2024-06-26T06:40:54.5430034Z inflating: build/bin/c10_Synchronized_test 2024-06-26T06:40:54.5500059Z inflating: build/bin/c10_ThreadLocal_test 2024-06-26T06:40:54.5564658Z inflating: build/bin/c10_TypeIndex_test 2024-06-26T06:40:54.5629035Z inflating: build/bin/c10_TypeList_test 2024-06-26T06:40:54.5690506Z inflating: build/bin/c10_TypeTraits_test 2024-06-26T06:40:54.5755561Z inflating: build/bin/c10_accumulate_test 2024-06-26T06:40:54.5825300Z inflating: build/bin/c10_bfloat16_test 2024-06-26T06:40:54.5888716Z inflating: build/bin/c10_bit_cast_test 2024-06-26T06:40:54.5959651Z inflating: build/bin/c10_complex_math_test 2024-06-26T06:40:54.6029762Z inflating: build/bin/c10_complex_test 2024-06-26T06:40:54.6096251Z inflating: build/bin/c10_exception_test 2024-06-26T06:40:54.6159438Z inflating: build/bin/c10_flags_test 2024-06-26T06:40:54.6222557Z inflating: build/bin/c10_generic_math_test 2024-06-26T06:40:54.6429277Z inflating: build/bin/c10_intrusive_ptr_test 2024-06-26T06:40:54.6493119Z inflating: build/bin/c10_irange_test 2024-06-26T06:40:54.6560362Z inflating: build/bin/c10_lazy_test 2024-06-26T06:40:54.6632170Z inflating: build/bin/c10_logging_test 2024-06-26T06:40:54.6726237Z inflating: build/bin/c10_optional_test 2024-06-26T06:40:54.6804733Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-06-26T06:40:54.6872434Z inflating: build/bin/c10_registry_test 2024-06-26T06:40:54.7062213Z inflating: build/bin/c10_small_vector_test 2024-06-26T06:40:54.7127055Z inflating: build/bin/c10_ssize_test 2024-06-26T06:40:54.7192517Z inflating: build/bin/c10_string_util_test 2024-06-26T06:40:54.7266466Z inflating: build/bin/c10_string_view_test 2024-06-26T06:40:54.7329622Z inflating: build/bin/c10_tempfile_test 2024-06-26T06:40:54.7390753Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-06-26T06:40:54.7461343Z inflating: build/bin/c10_typeid_test 2024-06-26T06:40:54.8020320Z inflating: build/bin/protoc-3.13.0.0 2024-06-26T06:40:54.8579336Z inflating: build/bin/protoc 2024-06-26T06:40:54.8645981Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-06-26T06:40:54.8712657Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-06-26T06:40:54.8779176Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-06-26T06:40:54.8844275Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-06-26T06:40:54.8911009Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-06-26T06:40:54.8977627Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-06-26T06:40:54.9043480Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-06-26T06:40:54.9105177Z inflating: build/bin/c10_cuda_CUDATest 2024-06-26T06:40:54.9519198Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-06-26T06:40:54.9950460Z inflating: build/bin/vec_test_all_types_AVX512 2024-06-26T06:40:55.0400109Z inflating: build/bin/vec_test_all_types_AVX2 2024-06-26T06:40:55.0467489Z inflating: build/bin/HashStoreTest 2024-06-26T06:40:55.0534197Z inflating: build/bin/FileStoreTest 2024-06-26T06:40:55.0604026Z inflating: build/bin/TCPStoreTest 2024-06-26T06:40:55.0621614Z inflating: build/bin/ProcessGroupMPITest 2024-06-26T06:40:55.0627142Z inflating: build/bin/torch_shm_manager 2024-06-26T06:40:55.0695085Z inflating: build/bin/test_edge_op_registration 2024-06-26T06:40:55.0698878Z inflating: build/bin/example_allreduce 2024-06-26T06:40:55.0764331Z inflating: build/bin/Dimname_test 2024-06-26T06:40:55.0857046Z inflating: build/bin/Dict_test 2024-06-26T06:40:55.0938654Z inflating: build/bin/MaybeOwned_test 2024-06-26T06:40:55.1012453Z inflating: build/bin/apply_utils_test 2024-06-26T06:40:55.1084543Z inflating: build/bin/NamedTensor_test 2024-06-26T06:40:55.1158639Z inflating: build/bin/atest 2024-06-26T06:40:55.1238029Z inflating: build/bin/basic 2024-06-26T06:40:55.1307352Z inflating: build/bin/broadcast_test 2024-06-26T06:40:55.1371106Z inflating: build/bin/cpu_allocator_test 2024-06-26T06:40:55.1444564Z inflating: build/bin/cpu_generator_test 2024-06-26T06:40:55.1511611Z inflating: build/bin/cpu_profiling_allocator_test 2024-06-26T06:40:55.1628675Z inflating: build/bin/cpu_rng_test 2024-06-26T06:40:55.1691525Z inflating: build/bin/dispatch_key_set_test 2024-06-26T06:40:55.1755422Z inflating: build/bin/dlconvertor_test 2024-06-26T06:40:55.1828896Z inflating: build/bin/extension_backend_test 2024-06-26T06:40:55.1898263Z inflating: build/bin/half_test 2024-06-26T06:40:55.2018675Z inflating: build/bin/ivalue_test 2024-06-26T06:40:55.2081223Z inflating: build/bin/lazy_tensor_test 2024-06-26T06:40:55.2149857Z inflating: build/bin/math_kernel_test 2024-06-26T06:40:55.2217965Z inflating: build/bin/memory_format_test 2024-06-26T06:40:55.2284599Z inflating: build/bin/memory_overlapping_test 2024-06-26T06:40:55.2351515Z inflating: build/bin/mobile_memory_cleanup 2024-06-26T06:40:55.2415334Z inflating: build/bin/operator_name_test 2024-06-26T06:40:55.2485540Z inflating: build/bin/native_test 2024-06-26T06:40:55.2550617Z inflating: build/bin/operators_test 2024-06-26T06:40:55.2616333Z inflating: build/bin/packedtensoraccessor_test 2024-06-26T06:40:55.2700835Z inflating: build/bin/pow_test 2024-06-26T06:40:55.2772860Z inflating: build/bin/quantized_test 2024-06-26T06:40:55.2835747Z inflating: build/bin/reduce_ops_test 2024-06-26T06:40:55.2900592Z inflating: build/bin/reportMemoryUsage_test 2024-06-26T06:40:55.2971381Z inflating: build/bin/scalar_tensor_test 2024-06-26T06:40:55.3045213Z inflating: build/bin/scalar_test 2024-06-26T06:40:55.3110647Z inflating: build/bin/StorageUtils_test 2024-06-26T06:40:55.3176906Z inflating: build/bin/stride_properties_test 2024-06-26T06:40:55.3275907Z inflating: build/bin/tensor_iterator_test 2024-06-26T06:40:55.3279782Z inflating: build/bin/thread_init_test 2024-06-26T06:40:55.3348220Z inflating: build/bin/test_parallel 2024-06-26T06:40:55.3418353Z inflating: build/bin/type_ptr_test 2024-06-26T06:40:55.3494002Z inflating: build/bin/type_test 2024-06-26T06:40:55.3559857Z inflating: build/bin/undefined_tensor_test 2024-06-26T06:40:55.3561822Z inflating: build/bin/verify_api_visibility 2024-06-26T06:40:55.3626951Z inflating: build/bin/weakref_test 2024-06-26T06:40:55.3714018Z inflating: build/bin/legacy_vmap_test 2024-06-26T06:40:55.3778939Z inflating: build/bin/wrapdim_test 2024-06-26T06:40:55.3843873Z inflating: build/bin/xla_tensor_test 2024-06-26T06:40:55.3920355Z inflating: build/bin/IListRef_test 2024-06-26T06:40:55.4054072Z inflating: build/bin/List_test 2024-06-26T06:40:55.4206839Z inflating: build/bin/kernel_function_legacy_test 2024-06-26T06:40:55.4291044Z inflating: build/bin/KernelFunction_test 2024-06-26T06:40:55.4412629Z inflating: build/bin/kernel_function_test 2024-06-26T06:40:55.4574068Z inflating: build/bin/kernel_lambda_legacy_test 2024-06-26T06:40:55.4703879Z inflating: build/bin/kernel_lambda_test 2024-06-26T06:40:55.4780936Z inflating: build/bin/kernel_stackbased_test 2024-06-26T06:40:55.4902256Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-06-26T06:40:55.4965955Z inflating: build/bin/CppSignature_test 2024-06-26T06:40:55.5036348Z inflating: build/bin/backend_fallback_test 2024-06-26T06:40:55.5097969Z inflating: build/bin/op_allowlist_test 2024-06-26T06:40:55.5178035Z inflating: build/bin/inline_container_test 2024-06-26T06:40:55.5557072Z inflating: build/bin/op_registration_test 2024-06-26T06:40:55.5623719Z inflating: build/bin/cuda_apply_test 2024-06-26T06:40:55.5687429Z inflating: build/bin/cuda_allocator_test 2024-06-26T06:40:55.5761737Z inflating: build/bin/cuda_atomic_ops_test 2024-06-26T06:40:55.5830502Z inflating: build/bin/cuda_caching_host_allocator_test 2024-06-26T06:40:55.5918814Z inflating: build/bin/cuda_complex_math_test 2024-06-26T06:40:55.5981309Z inflating: build/bin/cuda_device_test 2024-06-26T06:40:55.6055733Z inflating: build/bin/cuda_complex_test 2024-06-26T06:40:55.6128233Z inflating: build/bin/cuda_cub_test 2024-06-26T06:40:55.6192430Z inflating: build/bin/cuda_dlconvertor_test 2024-06-26T06:40:55.6257001Z inflating: build/bin/cuda_integer_divider_test 2024-06-26T06:40:55.6339345Z inflating: build/bin/cuda_distributions_test 2024-06-26T06:40:55.6410335Z inflating: build/bin/cuda_generator_test 2024-06-26T06:40:55.6473025Z inflating: build/bin/cuda_half_test 2024-06-26T06:40:55.6535430Z inflating: build/bin/cuda_optional_test 2024-06-26T06:40:55.6601289Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-06-26T06:40:55.6664419Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-06-26T06:40:55.6729059Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-06-26T06:40:55.6805275Z inflating: build/bin/cuda_stream_test 2024-06-26T06:40:55.6867607Z inflating: build/bin/cuda_cudnn_test 2024-06-26T06:40:55.6933230Z inflating: build/bin/cuda_vectorized_test 2024-06-26T06:40:55.6952380Z inflating: build/bin/tutorial_tensorexpr 2024-06-26T06:40:55.7034932Z inflating: build/bin/ProcessGroupGlooTest 2024-06-26T06:40:55.7114495Z inflating: build/bin/ProcessGroupNCCLTest 2024-06-26T06:40:55.7187066Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-06-26T06:40:55.7253167Z inflating: build/bin/ProcessGroupUCCTest 2024-06-26T06:40:55.7331008Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-06-26T06:40:55.7400129Z inflating: build/bin/test_dist_autograd 2024-06-26T06:40:55.7487069Z inflating: build/bin/test_cpp_rpc 2024-06-26T06:40:55.7490550Z inflating: build/bin/parallel_benchmark 2024-06-26T06:40:55.8532307Z inflating: build/bin/test_tensorexpr 2024-06-26T06:40:55.8543522Z inflating: build/bin/aot_model_compiler_test 2024-06-26T06:40:55.8627567Z inflating: build/bin/test_mobile_nnc 2024-06-26T06:40:55.9061112Z inflating: build/bin/test_lazy 2024-06-26T06:40:56.0542391Z inflating: build/bin/test_api 2024-06-26T06:40:56.1265029Z inflating: build/bin/test_jit 2024-06-26T06:40:56.1266168Z creating: .additional_ci_files/ 2024-06-26T06:40:56.1335378Z inflating: .additional_ci_files/test-times.json 2024-06-26T06:40:56.1606861Z inflating: .additional_ci_files/test-class-times.json 2024-06-26T06:40:56.1643952Z ##[group]Run rm artifacts.zip 2024-06-26T06:40:56.1644443Z rm artifacts.zip 2024-06-26T06:40:56.1653162Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:40:56.1653809Z env: 2024-06-26T06:40:56.1654153Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:40:56.1654885Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:40:56.1655493Z ##[endgroup] 2024-06-26T06:40:56.5821111Z ##[group]Run df -H 2024-06-26T06:40:56.5821491Z df -H 2024-06-26T06:40:56.5829513Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:40:56.5830188Z env: 2024-06-26T06:40:56.5830460Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:40:56.5831015Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:40:56.5831622Z ##[endgroup] 2024-06-26T06:40:56.5867063Z Filesystem Size Used Avail Use% Mounted on 2024-06-26T06:40:56.5868331Z devtmpfs 129G 0 129G 0% /dev 2024-06-26T06:40:56.5869553Z tmpfs 129G 20M 129G 1% /dev/shm 2024-06-26T06:40:56.5870752Z tmpfs 129G 533k 129G 1% /run 2024-06-26T06:40:56.5871341Z tmpfs 129G 0 129G 0% /sys/fs/cgroup 2024-06-26T06:40:56.5872138Z /dev/xvda1 162G 42G 120G 26% / 2024-06-26T06:40:56.5921873Z Prepare all required actions 2024-06-26T06:40:56.5922470Z Getting action download info 2024-06-26T06:40:56.7106038Z ##[group]Run ./.github/actions/download-td-artifacts 2024-06-26T06:40:56.7106614Z with: 2024-06-26T06:40:56.7106924Z env: 2024-06-26T06:40:56.7107257Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:40:56.7107798Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:40:56.7108403Z ##[endgroup] 2024-06-26T06:40:56.7152476Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T06:40:56.7153037Z with: 2024-06-26T06:40:56.7153352Z name: td_results 2024-06-26T06:40:56.7153746Z s3-bucket: gha-artifacts 2024-06-26T06:40:56.7154186Z region: us-east-1 2024-06-26T06:40:56.7154534Z env: 2024-06-26T06:40:56.7154871Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:40:56.7155433Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:40:56.7156023Z ##[endgroup] 2024-06-26T06:40:57.3183682Z (node:101127) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T06:40:57.3184662Z 2024-06-26T06:40:57.3185029Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T06:40:57.3185943Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T06:40:57.3187013Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T06:40:57.4215182Z Found 1 objects with prefix pytorch/pytorch/9673645538/td_results/ 2024-06-26T06:40:57.4216410Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-06-26T06:40:57.5146702Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-06-26T06:40:57.5157204Z Artifact download has finished successfully 2024-06-26T06:40:57.5342996Z ##[group]Run mkdir -p .additional_ci_files 2024-06-26T06:40:57.5343598Z mkdir -p .additional_ci_files 2024-06-26T06:40:57.5344318Z mv td_results.json .additional_ci_files/td_results.json 2024-06-26T06:40:57.5352710Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:40:57.5353313Z env: 2024-06-26T06:40:57.5353652Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:40:57.5354213Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:40:57.5354800Z ##[endgroup] 2024-06-26T06:40:57.5440759Z ##[group]Run .github/scripts/parse_ref.py 2024-06-26T06:40:57.5441393Z .github/scripts/parse_ref.py 2024-06-26T06:40:57.5449292Z shell: /usr/bin/bash -e {0} 2024-06-26T06:40:57.5449742Z env: 2024-06-26T06:40:57.5450078Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:40:57.5450631Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:40:57.5451243Z ##[endgroup] 2024-06-26T06:40:57.5766438Z Prepare all required actions 2024-06-26T06:40:57.5817980Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-06-26T06:40:57.5818553Z with: 2024-06-26T06:40:57.5819287Z github-token: *** 2024-06-26T06:40:57.5819655Z env: 2024-06-26T06:40:57.5819988Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:40:57.5820761Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:40:57.5821348Z ##[endgroup] 2024-06-26T06:40:57.5843543Z ##[group]Run set -eux 2024-06-26T06:40:57.5843962Z set -eux 2024-06-26T06:40:57.5844699Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-06-26T06:40:57.5852928Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:40:57.5853543Z env: 2024-06-26T06:40:57.5853885Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:40:57.5854443Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:40:57.5855247Z GITHUB_TOKEN: *** 2024-06-26T06:40:57.5855639Z ##[endgroup] 2024-06-26T06:40:57.5879764Z + python3 .github/scripts/get_workflow_job_id.py 9673645538 i-011558b4ba4829615 2024-06-26T06:41:00.3612801Z setting job-id=26688546814 2024-06-26T06:41:00.3613998Z setting job-name=linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-06-26T06:41:00.3876809Z Prepare all required actions 2024-06-26T06:41:00.3877444Z Getting action download info 2024-06-26T06:41:00.5264921Z ##[group]Run ./.github/actions/filter-test-configs 2024-06-26T06:41:00.5265501Z with: 2024-06-26T06:41:00.5266088Z github-token: *** 2024-06-26T06:41:00.5267924Z test-matrix: {"include": [{"config": "distributed", "shard": 1, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 2, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 3, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}]} 2024-06-26T06:41:00.5270261Z job-name: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-06-26T06:41:00.5271173Z env: 2024-06-26T06:41:00.5271515Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:41:00.5272064Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:41:00.5272676Z ##[endgroup] 2024-06-26T06:41:00.5328775Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T06:41:00.5329492Z with: 2024-06-26T06:41:00.5329818Z shell: bash 2024-06-26T06:41:00.5330186Z timeout_minutes: 10 2024-06-26T06:41:00.5330567Z max_attempts: 5 2024-06-26T06:41:00.5330952Z retry_wait_seconds: 30 2024-06-26T06:41:00.5332685Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T06:41:00.5334199Z polling_interval_seconds: 1 2024-06-26T06:41:00.5334640Z warning_on_retry: true 2024-06-26T06:41:00.5335068Z continue_on_error: false 2024-06-26T06:41:00.5335479Z env: 2024-06-26T06:41:00.5335788Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:41:00.5336342Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:41:00.5337161Z GITHUB_TOKEN: *** 2024-06-26T06:41:00.5337529Z ##[endgroup] 2024-06-26T06:41:00.6062783Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T06:41:00.9019659Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T06:41:00.9239900Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.7/site-packages (2.27.1) 2024-06-26T06:41:00.9431156Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.7/site-packages (6.0.1) 2024-06-26T06:41:00.9446043Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (3.7) 2024-06-26T06:41:00.9466163Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (1.26.19) 2024-06-26T06:41:00.9720519Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2.0.12) 2024-06-26T06:41:00.9749941Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2024.6.2) 2024-06-26T06:41:01.6058452Z Command completed after 1 attempt(s). 2024-06-26T06:41:01.6112812Z ##[group]Run set -x 2024-06-26T06:41:01.6113234Z set -x 2024-06-26T06:41:01.6113585Z  2024-06-26T06:41:01.6114269Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T06:41:01.6115139Z # in runner workspace 2024-06-26T06:41:01.6115778Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-06-26T06:41:01.6124400Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:41:01.6125036Z env: 2024-06-26T06:41:01.6125358Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:41:01.6125931Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:41:01.6126548Z ##[endgroup] 2024-06-26T06:41:01.6153675Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-06-26T06:41:01.6428031Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T06:41:01.6428745Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T06:41:01.6429305Z echo "Job name: ${JOB_NAME}" 2024-06-26T06:41:01.6429802Z  2024-06-26T06:41:01.6430490Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T06:41:01.6431338Z # in runner workspace 2024-06-26T06:41:01.6432061Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-06-26T06:41:01.6432873Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-06-26T06:41:01.6433446Z  --job-name "${JOB_NAME}" \ 2024-06-26T06:41:01.6435457Z  --test-matrix "{"include": [{"config": "distributed", "shard": 1, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 2, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 3, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}]}" \ 2024-06-26T06:41:01.6437545Z  --selected-test-configs "" \ 2024-06-26T06:41:01.6438121Z  --pr-number "${PR_NUMBER}" \ 2024-06-26T06:41:01.6438645Z  --tag "${TAG}" \ 2024-06-26T06:41:01.6439114Z  --event-name "${EVENT_NAME}" \ 2024-06-26T06:41:01.6439671Z  --schedule "${SCHEDULE}" \ 2024-06-26T06:41:01.6440204Z  --branch "${HEAD_BRANCH}" 2024-06-26T06:41:01.6447844Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:41:01.6448475Z env: 2024-06-26T06:41:01.6448816Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:41:01.6449365Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:41:01.6450205Z GITHUB_TOKEN: *** 2024-06-26T06:41:01.6451024Z JOB_NAME: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-06-26T06:41:01.6452182Z PR_NUMBER: 129470 2024-06-26T06:41:01.6452567Z TAG: 2024-06-26T06:41:01.6452912Z EVENT_NAME: pull_request 2024-06-26T06:41:01.6453335Z SCHEDULE: 2024-06-26T06:41:01.6453703Z HEAD_BRANCH: 2024-06-26T06:41:01.6454072Z ##[endgroup] 2024-06-26T06:41:01.6478393Z Workflow: pull 2024-06-26T06:41:01.6479255Z Job name: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-06-26T06:41:01.9010288Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-06-26T06:41:02.0623371Z ##[group]Run echo "Filtered matrix:" 2024-06-26T06:41:02.0623938Z echo "Filtered matrix:" 2024-06-26T06:41:02.0625873Z echo "{"include": [{"config": "distributed", "shard": 1, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 2, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}, {"config": "distributed", "shard": 3, "num_shards": 3, "runner": "linux.8xlarge.nvidia.gpu"}]}" 2024-06-26T06:41:02.0627823Z  2024-06-26T06:41:02.0628141Z echo 2024-06-26T06:41:02.0628631Z echo "Is the current job unstable? False" 2024-06-26T06:41:02.0629439Z  2024-06-26T06:41:02.0629750Z echo 2024-06-26T06:41:02.0630198Z echo "Is keep-going label set? False" 2024-06-26T06:41:02.0630759Z  2024-06-26T06:41:02.0631069Z echo 2024-06-26T06:41:02.0631459Z echo "Renabled issues? " 2024-06-26T06:41:02.0639768Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:41:02.0640376Z env: 2024-06-26T06:41:02.0640719Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:41:02.0641289Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:41:02.0641880Z ##[endgroup] 2024-06-26T06:41:02.0668005Z Filtered matrix: 2024-06-26T06:41:02.0669915Z {include: [{config: distributed, shard: 1, num_shards: 3, runner: linux.8xlarge.nvidia.gpu}, {config: distributed, shard: 2, num_shards: 3, runner: linux.8xlarge.nvidia.gpu}, {config: distributed, shard: 3, num_shards: 3, runner: linux.8xlarge.nvidia.gpu}]} 2024-06-26T06:41:02.0671582Z 2024-06-26T06:41:02.0672040Z Is the current job unstable? False 2024-06-26T06:41:02.0672457Z 2024-06-26T06:41:02.0672787Z Is keep-going label set? False 2024-06-26T06:41:02.0673045Z 2024-06-26T06:41:02.0673243Z Renabled issues? 2024-06-26T06:41:02.0726832Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T06:41:02.0727756Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T06:41:02.0735986Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:41:02.0736594Z env: 2024-06-26T06:41:02.0736934Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:41:02.0737500Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:41:02.0738088Z JOB_TIMEOUT: 360 2024-06-26T06:41:02.0738462Z ##[endgroup] 2024-06-26T06:41:02.0831402Z ##[group]Run set -x 2024-06-26T06:41:02.0831931Z set -x 2024-06-26T06:41:02.0832273Z  2024-06-26T06:41:02.0832712Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-06-26T06:41:02.0833400Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-06-26T06:41:02.0834103Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-06-26T06:41:02.0834740Z  TEST_COMMAND=.ci/onnx/test.sh 2024-06-26T06:41:02.0835257Z else 2024-06-26T06:41:02.0835659Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T06:41:02.0836184Z fi 2024-06-26T06:41:02.0836518Z  2024-06-26T06:41:02.0837075Z # detached container should get cleaned up by teardown_ec2_linux 2024-06-26T06:41:02.0838040Z # TODO: Stop building test binaries as part of the build phase 2024-06-26T06:41:02.0838900Z # Used for GPU_FLAG since that doesn't play nice 2024-06-26T06:41:02.0839638Z # shellcheck disable=SC2086,SC2090 2024-06-26T06:41:02.0840203Z container_name=$(docker run \ 2024-06-26T06:41:02.0840728Z  ${GPU_FLAG:-} \ 2024-06-26T06:41:02.0841188Z  -e BUILD_ENVIRONMENT \ 2024-06-26T06:41:02.0841668Z  -e PR_NUMBER \ 2024-06-26T06:41:02.0842116Z  -e GITHUB_ACTIONS \ 2024-06-26T06:41:02.0842615Z  -e GITHUB_REPOSITORY \ 2024-06-26T06:41:02.0843106Z  -e GITHUB_WORKFLOW \ 2024-06-26T06:41:02.0843586Z  -e GITHUB_JOB \ 2024-06-26T06:41:02.0844035Z  -e GITHUB_RUN_ID \ 2024-06-26T06:41:02.0844515Z  -e GITHUB_RUN_NUMBER \ 2024-06-26T06:41:02.0845000Z  -e GITHUB_RUN_ATTEMPT \ 2024-06-26T06:41:02.0845495Z  -e JOB_ID \ 2024-06-26T06:41:02.0845916Z  -e JOB_NAME \ 2024-06-26T06:41:02.0846328Z  -e BASE_SHA \ 2024-06-26T06:41:02.0846752Z  -e BRANCH \ 2024-06-26T06:41:02.0847168Z  -e SHA1 \ 2024-06-26T06:41:02.0847574Z  -e AWS_DEFAULT_REGION \ 2024-06-26T06:41:02.0848075Z  -e IN_WHEEL_TEST \ 2024-06-26T06:41:02.0848546Z  -e SHARD_NUMBER \ 2024-06-26T06:41:02.0848982Z  -e TEST_CONFIG \ 2024-06-26T06:41:02.0849442Z  -e NUM_TEST_SHARDS \ 2024-06-26T06:41:02.0849929Z  -e REENABLED_ISSUES \ 2024-06-26T06:41:02.0850588Z  -e CONTINUE_THROUGH_ERROR \ 2024-06-26T06:41:02.0851118Z  -e VERBOSE_TEST_LOGS \ 2024-06-26T06:41:02.0851861Z  -e NO_TEST_TIMEOUT \ 2024-06-26T06:41:02.0852346Z  -e NO_TD \ 2024-06-26T06:41:02.0852774Z  -e TD_DISTRIBUTED \ 2024-06-26T06:41:02.0853248Z  -e PR_LABELS \ 2024-06-26T06:41:02.0853733Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-06-26T06:41:02.0854305Z  -e SCCACHE_BUCKET \ 2024-06-26T06:41:02.0854803Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-06-26T06:41:02.0855292Z  -e XLA_CUDA \ 2024-06-26T06:41:02.0855799Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-06-26T06:41:02.0856440Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-06-26T06:41:02.0857070Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-06-26T06:41:02.0857717Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-06-26T06:41:02.0858300Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-06-26T06:41:02.0858821Z  -e DASHBOARD_TAG \ 2024-06-26T06:41:02.0859422Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-06-26T06:41:02.0860115Z  --security-opt seccomp=unconfined \ 2024-06-26T06:41:02.0860695Z  --cap-add=SYS_PTRACE \ 2024-06-26T06:41:02.0861171Z  --ipc=host \ 2024-06-26T06:41:02.0861610Z  --shm-size="${SHM_SIZE}" \ 2024-06-26T06:41:02.0862102Z  --tty \ 2024-06-26T06:41:02.0862472Z  --detach \ 2024-06-26T06:41:02.0862910Z  --name="${container_name}" \ 2024-06-26T06:41:02.0863432Z  --user jenkins \ 2024-06-26T06:41:02.0864163Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-06-26T06:41:02.0864898Z  -w /var/lib/jenkins/workspace \ 2024-06-26T06:41:02.0865443Z  "${DOCKER_IMAGE}" 2024-06-26T06:41:02.0865856Z ) 2024-06-26T06:41:02.0866361Z # Propagate download.pytorch.org IP to container 2024-06-26T06:41:02.0867549Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-06-26T06:41:02.0868768Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-06-26T06:41:02.0869931Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-06-26T06:41:02.0877834Z shell: /usr/bin/bash -e {0} 2024-06-26T06:41:02.0878257Z env: 2024-06-26T06:41:02.0878602Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:41:02.0879168Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:41:02.0879892Z BUILD_ENVIRONMENT: linux-focal-cuda11.8-py3.10-gcc9 2024-06-26T06:41:02.0880500Z PR_NUMBER: 129470 2024-06-26T06:41:02.0880922Z GITHUB_REPOSITORY: pytorch/pytorch 2024-06-26T06:41:02.0881436Z GITHUB_WORKFLOW: pull 2024-06-26T06:41:02.0881831Z GITHUB_JOB: test 2024-06-26T06:41:02.0882218Z GITHUB_RUN_ID: 9673645538 2024-06-26T06:41:02.0882661Z GITHUB_RUN_NUMBER: 219936 2024-06-26T06:41:02.0883100Z GITHUB_RUN_ATTEMPT: 1 2024-06-26T06:41:02.0883507Z JOB_ID: 26688546814 2024-06-26T06:41:02.0884326Z JOB_NAME: linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-06-26T06:41:02.0885228Z BRANCH: pull/129470 2024-06-26T06:41:02.0885763Z SHA1: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T06:41:02.0886413Z BASE_SHA: 4b9c9a9cc9c9283380a011310ba180c105c3dcb9 2024-06-26T06:41:02.0886982Z TEST_CONFIG: distributed 2024-06-26T06:41:02.0887417Z SHARD_NUMBER: 3 2024-06-26T06:41:02.0887794Z NUM_TEST_SHARDS: 3 2024-06-26T06:41:02.0888179Z REENABLED_ISSUES: 2024-06-26T06:41:02.0888608Z CONTINUE_THROUGH_ERROR: False 2024-06-26T06:41:02.0889092Z VERBOSE_TEST_LOGS: False 2024-06-26T06:41:02.0889510Z NO_TEST_TIMEOUT: False 2024-06-26T06:41:02.0889924Z NO_TD: False 2024-06-26T06:41:02.0890295Z TD_DISTRIBUTED: False 2024-06-26T06:41:02.0890796Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-06-26T06:41:02.0891560Z SCCACHE_S3_KEY_PREFIX: pull 2024-06-26T06:41:02.0892235Z SHM_SIZE: 2g 2024-06-26T06:41:02.0893423Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:41:02.0894739Z XLA_CUDA: 2024-06-26T06:41:02.0895338Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-06-26T06:41:02.0896091Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-06-26T06:41:02.0896634Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-06-26T06:41:02.0897140Z DASHBOARD_TAG: 2024-06-26T06:41:02.0897512Z HUGGING_FACE_HUB_TOKEN: 2024-06-26T06:41:02.0897939Z ##[endgroup] 2024-06-26T06:41:02.0921907Z + [[ distributed == \m\u\l\t\i\g\p\u ]] 2024-06-26T06:41:02.0922661Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *onnx* ]] 2024-06-26T06:41:02.0923284Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T06:41:02.0932377Z +++ nproc --ignore=2 2024-06-26T06:41:02.0950234Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=30 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_9673645538 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:41:13.1673712Z + container_name=f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:41:13.1675594Z + grep download.pytorch.org /etc/hosts 2024-06-26T06:41:13.1677944Z + docker exec -i f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 sudo bash -c '/bin/cat >> /etc/hosts' 2024-06-26T06:41:13.2374524Z + echo DOCKER_CONTAINER_ID=f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:41:13.2378509Z ++ echo dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T06:41:13.2381105Z + docker exec -t f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 sh -c 'pip install dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-06-26T06:41:13.7885665Z Processing ./dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+gitb8c4c54) 2024-06-26T06:41:14.9311646Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.13.1) 2024-06-26T06:41:14.9317978Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (4.12.2) 2024-06-26T06:41:14.9322352Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.12.1) 2024-06-26T06:41:14.9327089Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.8.8) 2024-06-26T06:41:14.9331971Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.1.4) 2024-06-26T06:41:14.9337027Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2024.2.0) 2024-06-26T06:41:14.9375830Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.3.0) 2024-06-26T06:41:14.9460098Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.21.2) 2024-06-26T06:41:15.0075587Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.1.5) 2024-06-26T06:41:15.0303926Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.3.0) 2024-06-26T06:41:16.1566962Z Installing collected packages: torch 2024-06-26T06:41:28.9925478Z Successfully installed torch-2.5.0a0+gitb8c4c54 2024-06-26T06:41:29.0592261Z ++ dirname .ci/pytorch/test.sh 2024-06-26T06:41:29.0598547Z + source .ci/pytorch/common.sh 2024-06-26T06:41:29.0602243Z +++ dirname .ci/pytorch/common.sh 2024-06-26T06:41:29.0609894Z ++ source .ci/pytorch/common_utils.sh 2024-06-26T06:41:29.0612421Z +++ declare -f -t trap_add 2024-06-26T06:41:29.0619185Z ++ set -ex 2024-06-26T06:41:29.0619719Z ++ [[ linux-focal-cuda11.8-py3.10-gcc9 == *rocm* ]] 2024-06-26T06:41:29.0622119Z ++ BUILD_TEST_LIBTORCH=0 2024-06-26T06:41:29.0622834Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *rocm* ]] 2024-06-26T06:41:29.0623947Z ++ stat -c %u /var/lib/jenkins/workspace 2024-06-26T06:41:29.0635783Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-06-26T06:41:29.0636324Z + trap_add cleanup_workspace EXIT 2024-06-26T06:41:29.0638614Z + trap_add_cmd=cleanup_workspace 2024-06-26T06:41:29.0639205Z + shift 2024-06-26T06:41:29.0639546Z + for trap_add_name in "$@" 2024-06-26T06:41:29.0643982Z +++ trap -p EXIT 2024-06-26T06:41:29.0647233Z ++ eval 'extract_trap_cmd ' 2024-06-26T06:41:29.0647709Z +++ extract_trap_cmd 2024-06-26T06:41:29.0648113Z +++ printf '%s\n' '' 2024-06-26T06:41:29.0648572Z ++ printf '%s\n' cleanup_workspace 2024-06-26T06:41:29.0650531Z + trap -- ' 2024-06-26T06:41:29.0650922Z cleanup_workspace' EXIT 2024-06-26T06:41:29.0651443Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-06-26T06:41:29.6061039Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-06-26T06:41:29.6078789Z + echo 'Environment variables:' 2024-06-26T06:41:29.6079291Z Environment variables: 2024-06-26T06:41:29.6079706Z + env 2024-06-26T06:41:29.6087586Z INSTALLED_DB=yes 2024-06-26T06:41:29.6088100Z NV_LIBCUBLAS_VERSION=11.11.3.6-1 2024-06-26T06:41:29.6088613Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T06:41:29.6089107Z NV_NVML_DEV_VERSION=11.8.86-1 2024-06-26T06:41:29.6089847Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T06:41:29.6090598Z CONTINUE_THROUGH_ERROR=False 2024-06-26T06:41:29.6091249Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.16.2-1+cuda11.8 2024-06-26T06:41:29.6092386Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.16.2-1 2024-06-26T06:41:29.6093085Z BUILD_ENVIRONMENT=linux-focal-cuda11.8-py3.10-gcc9 2024-06-26T06:41:29.6093689Z HOSTNAME=f1d373c93f78 2024-06-26T06:41:29.6094778Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_0f16eff9-544f-4e42-b083-bdb347e91b26 2024-06-26T06:41:29.6095803Z GITHUB_ACTION=__self 2024-06-26T06:41:29.6096248Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T06:41:29.6099000Z NVIDIA_REQUIRE_CUDA=cuda>=11.8 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 2024-06-26T06:41:29.6102013Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-11-8=11.11.3.6-1 2024-06-26T06:41:29.6103013Z NV_NVTX_VERSION=11.8.86-1 2024-06-26T06:41:29.6103445Z GITHUB_RUN_NUMBER=219936 2024-06-26T06:41:29.6103873Z TEST_CONFIG=distributed 2024-06-26T06:41:29.6104303Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T06:41:29.6104902Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T06:41:29.6105497Z NV_CUDA_CUDART_DEV_VERSION=11.8.89-1 2024-06-26T06:41:29.6106067Z NV_LIBCUSPARSE_VERSION=11.7.5.86-1 2024-06-26T06:41:29.6106611Z NV_LIBNPP_VERSION=11.8.0.86-1 2024-06-26T06:41:29.6107177Z GITHUB_TRIGGERING_ACTOR=leslie-fang-intel 2024-06-26T06:41:29.6107827Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T06:41:29.6108393Z GITHUB_REF_TYPE=branch 2024-06-26T06:41:29.6108842Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T06:41:29.6109330Z NCCL_VERSION=2.16.2-1 2024-06-26T06:41:29.6109788Z BASE_SHA=4b9c9a9cc9c9283380a011310ba180c105c3dcb9 2024-06-26T06:41:29.6110344Z XLA_CUDA= 2024-06-26T06:41:29.6110694Z HUGGING_FACE_HUB_TOKEN= 2024-06-26T06:41:29.6113265Z *** 2024-06-26T06:41:29.6113650Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T06:41:29.6114151Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T06:41:29.6114589Z GITHUB_ACTIONS=true 2024-06-26T06:41:29.6114998Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:41:29.6115654Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-11-8=11.8.87-1 2024-06-26T06:41:29.6116387Z NV_LIBNPP_PACKAGE=libnpp-11-8=11.8.0.86-1 2024-06-26T06:41:29.6116959Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T06:41:29.6117592Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T06:41:29.6118218Z GITHUB_SHA=4b51b1a62a63a1add1b3a0f7882f5c7dc66b8f8d 2024-06-26T06:41:29.6119318Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/129470/merge 2024-06-26T06:41:29.6120168Z UCC_HOME=/usr 2024-06-26T06:41:29.6120607Z NV_LIBCUBLAS_DEV_VERSION=11.11.3.6-1 2024-06-26T06:41:29.6121095Z VERBOSE_TEST_LOGS=False 2024-06-26T06:41:29.6121519Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T06:41:29.6122095Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-11-8 2024-06-26T06:41:29.6122692Z GITHUB_REF=refs/pull/129470/merge 2024-06-26T06:41:29.6123223Z NV_CUDA_CUDART_VERSION=11.8.89-1 2024-06-26T06:41:29.6123694Z SHARD_NUMBER=3 2024-06-26T06:41:29.6124057Z GITHUB_REF_PROTECTED=false 2024-06-26T06:41:29.6124491Z HOME=/var/lib/jenkins 2024-06-26T06:41:29.6124947Z GITHUB_API_URL=https://api.github.com 2024-06-26T06:41:29.6125481Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T06:41:29.6126074Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T06:41:29.6126679Z SCCACHE_S3_KEY_PREFIX=pull 2024-06-26T06:41:29.6127100Z CUDA_VERSION=11.8.0 2024-06-26T06:41:29.6127633Z NV_LIBCUBLAS_PACKAGE=libcublas-11-8=11.11.3.6-1 2024-06-26T06:41:29.6128196Z NUM_TEST_SHARDS=3 2024-06-26T06:41:29.6128554Z UCX_HOME=/usr 2024-06-26T06:41:29.6129212Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-11-8=11.8.0-1 2024-06-26T06:41:29.6130600Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_0f16eff9-544f-4e42-b083-bdb347e91b26 2024-06-26T06:41:29.6132469Z JOB_NAME=linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-06-26T06:41:29.6134049Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_0f16eff9-544f-4e42-b083-bdb347e91b26 2024-06-26T06:41:29.6135493Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T06:41:29.6136349Z GITHUB_EVENT_NAME=pull_request 2024-06-26T06:41:29.6136790Z DASHBOARD_TAG= 2024-06-26T06:41:29.6137168Z GITHUB_RUN_ID=9673645538 2024-06-26T06:41:29.6137746Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-11-8=11.8.0.86-1 2024-06-26T06:41:29.6138412Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-11-8 2024-06-26T06:41:29.6139701Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_0f16eff9-544f-4e42-b083-bdb347e91b26 2024-06-26T06:41:29.6140882Z GITHUB_ACTOR=leslie-fang-intel 2024-06-26T06:41:29.6141384Z NV_LIBNPP_DEV_VERSION=11.8.0.86-1 2024-06-26T06:41:29.6141871Z PR_NUMBER=129470 2024-06-26T06:41:29.6142475Z GITHUB_RUN_ATTEMPT=1 2024-06-26T06:41:29.6142877Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T06:41:29.6143431Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T06:41:29.6144010Z TERM=xterm 2024-06-26T06:41:29.6144417Z NV_LIBCUSPARSE_DEV_VERSION=11.7.5.86-1 2024-06-26T06:41:29.6144934Z INSTALLED_VISION=yes 2024-06-26T06:41:29.6145334Z BRANCH=pull/129470 2024-06-26T06:41:29.6145716Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T06:41:29.6146219Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T06:41:29.6146745Z CUDA_PATH=/usr/local/cuda 2024-06-26T06:41:29.6147694Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T06:41:29.6148663Z GITHUB_SERVER_URL=https://github.com 2024-06-26T06:41:29.6149272Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T06:41:29.6149839Z REENABLED_ISSUES= 2024-06-26T06:41:29.6150211Z SHLVL=1 2024-06-26T06:41:29.6150534Z MAX_JOBS=30 2024-06-26T06:41:29.6150913Z NV_CUDA_LIB_VERSION=11.8.0-1 2024-06-26T06:41:29.6151375Z NVARCH=x86_64 2024-06-26T06:41:29.6151742Z GITHUB_ACTOR_ID=53841472 2024-06-26T06:41:29.6152442Z GITHUB_WORKFLOW_SHA=4b51b1a62a63a1add1b3a0f7882f5c7dc66b8f8d 2024-06-26T06:41:29.6153316Z GITHUB_REF_NAME=129470/merge 2024-06-26T06:41:29.6153884Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-11-8 2024-06-26T06:41:29.6154707Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T06:41:29.6155435Z GITHUB_JOB=test 2024-06-26T06:41:29.6155945Z NV_LIBNCCL_PACKAGE=libnccl2=2.16.2-1+cuda11.8 2024-06-26T06:41:29.6156634Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T06:41:29.6157279Z NO_TEST_TIMEOUT=False 2024-06-26T06:41:29.6157845Z TD_DISTRIBUTED=False 2024-06-26T06:41:29.6158341Z NV_CUDA_NSIGHT_COMPUTE_VERSION=11.8.0-1 2024-06-26T06:41:29.6158893Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T06:41:29.6159421Z NV_NVPROF_VERSION=11.8.87-1 2024-06-26T06:41:29.6159860Z GITHUB_RETENTION_DAYS=90 2024-06-26T06:41:29.6160309Z OPENSSL_DIR=/opt/openssl 2024-06-26T06:41:29.6160751Z GITHUB_ACTION_REPOSITORY= 2024-06-26T06:41:29.6162074Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T06:41:29.6163608Z GITHUB_BASE_REF=gh/leslie-fang-intel/126/base 2024-06-26T06:41:29.6164196Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T06:41:29.6164638Z CI=true 2024-06-26T06:41:29.6165043Z NV_LIBNCCL_PACKAGE_VERSION=2.16.2-1 2024-06-26T06:41:29.6165570Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T06:41:29.6166019Z JOB_ID=26688546814 2024-06-26T06:41:29.6166416Z INSTALLED_PROTOBUF=yes 2024-06-26T06:41:29.6166975Z GITHUB_HEAD_REF=gh/leslie-fang-intel/126/head 2024-06-26T06:41:29.6167517Z GITHUB_ACTION_REF= 2024-06-26T06:41:29.6168062Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T06:41:29.6168657Z GITHUB_WORKFLOW=pull 2024-06-26T06:41:29.6169065Z DEBIAN_FRONTEND=noninteractive 2024-06-26T06:41:29.6170225Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_0f16eff9-544f-4e42-b083-bdb347e91b26 2024-06-26T06:41:29.6171286Z NO_TD=False 2024-06-26T06:41:29.6171851Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T06:41:29.6172337Z _=/usr/bin/env 2024-06-26T06:41:29.6172950Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-06-26T06:41:29.6315670Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-06-26T06:41:29.6316818Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-06-26T06:41:29.6317962Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-06-26T06:41:29.6319103Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-06-26T06:41:29.6319867Z + BUILD_DIR=build 2024-06-26T06:41:29.6320273Z + BUILD_RENAMED_DIR=build_renamed 2024-06-26T06:41:29.6320757Z + BUILD_BIN_DIR=build/bin 2024-06-26T06:41:29.6321162Z + SHARD_NUMBER=3 2024-06-26T06:41:29.6321772Z + NUM_TEST_SHARDS=3 2024-06-26T06:41:29.6322173Z + export VALGRIND=ON 2024-06-26T06:41:29.6322551Z + VALGRIND=ON 2024-06-26T06:41:29.6323117Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *clang9* ]] 2024-06-26T06:41:29.6323715Z + [[ 0 == \1 ]] 2024-06-26T06:41:29.6324064Z + [[ False == \1 ]] 2024-06-26T06:41:29.6324618Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *bazel* ]] 2024-06-26T06:41:29.6325251Z ++ realpath build/custom_test_artifacts 2024-06-26T06:41:29.6333443Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-06-26T06:41:29.6335180Z + [[ -n '' ]] 2024-06-26T06:41:29.6336081Z + echo 'Environment variables' 2024-06-26T06:41:29.6337051Z Environment variables 2024-06-26T06:41:29.6337869Z + env 2024-06-26T06:41:29.6344994Z INSTALLED_DB=yes 2024-06-26T06:41:29.6345847Z NV_LIBCUBLAS_VERSION=11.11.3.6-1 2024-06-26T06:41:29.6346711Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T06:41:29.6347239Z NV_NVML_DEV_VERSION=11.8.86-1 2024-06-26T06:41:29.6347994Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T06:41:29.6348737Z CONTINUE_THROUGH_ERROR=False 2024-06-26T06:41:29.6349362Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.16.2-1+cuda11.8 2024-06-26T06:41:29.6350005Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.16.2-1 2024-06-26T06:41:29.6350700Z BUILD_ENVIRONMENT=linux-focal-cuda11.8-py3.10-gcc9 2024-06-26T06:41:29.6351304Z HOSTNAME=f1d373c93f78 2024-06-26T06:41:29.6352370Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_0f16eff9-544f-4e42-b083-bdb347e91b26 2024-06-26T06:41:29.6353419Z GITHUB_ACTION=__self 2024-06-26T06:41:29.6353917Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T06:41:29.6356849Z NVIDIA_REQUIRE_CUDA=cuda>=11.8 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 2024-06-26T06:41:29.6359808Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-11-8=11.11.3.6-1 2024-06-26T06:41:29.6360444Z NV_NVTX_VERSION=11.8.86-1 2024-06-26T06:41:29.6360886Z GITHUB_RUN_NUMBER=219936 2024-06-26T06:41:29.6361324Z TEST_CONFIG=distributed 2024-06-26T06:41:29.6361756Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T06:41:29.6362350Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T06:41:29.6362943Z NV_CUDA_CUDART_DEV_VERSION=11.8.89-1 2024-06-26T06:41:29.6363488Z NV_LIBCUSPARSE_VERSION=11.7.5.86-1 2024-06-26T06:41:29.6364019Z NV_LIBNPP_VERSION=11.8.0.86-1 2024-06-26T06:41:29.6364585Z GITHUB_TRIGGERING_ACTOR=leslie-fang-intel 2024-06-26T06:41:29.6365206Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T06:41:29.6365802Z GITHUB_REF_TYPE=branch 2024-06-26T06:41:29.6366229Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T06:41:29.6366694Z NCCL_VERSION=2.16.2-1 2024-06-26T06:41:29.6367184Z BASE_SHA=4b9c9a9cc9c9283380a011310ba180c105c3dcb9 2024-06-26T06:41:29.6367758Z XLA_CUDA= 2024-06-26T06:41:29.6368090Z HUGGING_FACE_HUB_TOKEN= 2024-06-26T06:41:29.6368597Z *** 2024-06-26T06:41:29.6368956Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T06:41:29.6369436Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T06:41:29.6369894Z GITHUB_ACTIONS=true 2024-06-26T06:41:29.6370309Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:41:29.6370912Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-11-8=11.8.87-1 2024-06-26T06:41:29.6371591Z NV_LIBNPP_PACKAGE=libnpp-11-8=11.8.0.86-1 2024-06-26T06:41:29.6372686Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T06:41:29.6373325Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T06:41:29.6373964Z GITHUB_SHA=4b51b1a62a63a1add1b3a0f7882f5c7dc66b8f8d 2024-06-26T06:41:29.6374878Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/129470/merge 2024-06-26T06:41:29.6375695Z UCC_HOME=/usr 2024-06-26T06:41:29.6376103Z NV_LIBCUBLAS_DEV_VERSION=11.11.3.6-1 2024-06-26T06:41:29.6376617Z VERBOSE_TEST_LOGS=False 2024-06-26T06:41:29.6377259Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T06:41:29.6377827Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-11-8 2024-06-26T06:41:29.6378424Z GITHUB_REF=refs/pull/129470/merge 2024-06-26T06:41:29.6378950Z NV_CUDA_CUDART_VERSION=11.8.89-1 2024-06-26T06:41:29.6379403Z SHARD_NUMBER=3 2024-06-26T06:41:29.6379791Z GITHUB_REF_PROTECTED=false 2024-06-26T06:41:29.6380211Z HOME=/var/lib/jenkins 2024-06-26T06:41:29.6380666Z GITHUB_API_URL=https://api.github.com 2024-06-26T06:41:29.6381214Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T06:41:29.6381795Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T06:41:29.6382395Z SCCACHE_S3_KEY_PREFIX=pull 2024-06-26T06:41:29.6382831Z CUDA_VERSION=11.8.0 2024-06-26T06:41:29.6383362Z NV_LIBCUBLAS_PACKAGE=libcublas-11-8=11.11.3.6-1 2024-06-26T06:41:29.6383929Z NUM_TEST_SHARDS=3 2024-06-26T06:41:29.6384302Z UCX_HOME=/usr 2024-06-26T06:41:29.6384936Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-11-8=11.8.0-1 2024-06-26T06:41:29.6386328Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_0f16eff9-544f-4e42-b083-bdb347e91b26 2024-06-26T06:41:29.6387966Z JOB_NAME=linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.8xlarge.nvidia.gpu) 2024-06-26T06:41:29.6389540Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_0f16eff9-544f-4e42-b083-bdb347e91b26 2024-06-26T06:41:29.6390964Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T06:41:29.6391821Z GITHUB_EVENT_NAME=pull_request 2024-06-26T06:41:29.6392283Z DASHBOARD_TAG= 2024-06-26T06:41:29.6392639Z GITHUB_RUN_ID=9673645538 2024-06-26T06:41:29.6393349Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-11-8=11.8.0.86-1 2024-06-26T06:41:29.6394064Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-11-8 2024-06-26T06:41:29.6395327Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_0f16eff9-544f-4e42-b083-bdb347e91b26 2024-06-26T06:41:29.6396503Z GITHUB_ACTOR=leslie-fang-intel 2024-06-26T06:41:29.6397041Z NV_LIBNPP_DEV_VERSION=11.8.0.86-1 2024-06-26T06:41:29.6397502Z PR_NUMBER=129470 2024-06-26T06:41:29.6397883Z GITHUB_RUN_ATTEMPT=1 2024-06-26T06:41:29.6398270Z VALGRIND=ON 2024-06-26T06:41:29.6398618Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T06:41:29.6399172Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T06:41:29.6399757Z TERM=xterm 2024-06-26T06:41:29.6400166Z NV_LIBCUSPARSE_DEV_VERSION=11.7.5.86-1 2024-06-26T06:41:29.6400682Z INSTALLED_VISION=yes 2024-06-26T06:41:29.6401082Z BRANCH=pull/129470 2024-06-26T06:41:29.6401465Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T06:41:29.6401962Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T06:41:29.6402500Z CUDA_PATH=/usr/local/cuda 2024-06-26T06:41:29.6403448Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T06:41:29.6404408Z GITHUB_SERVER_URL=https://github.com 2024-06-26T06:41:29.6405011Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T06:41:29.6405588Z REENABLED_ISSUES= 2024-06-26T06:41:29.6405950Z SHLVL=1 2024-06-26T06:41:29.6406272Z MAX_JOBS=30 2024-06-26T06:41:29.6406651Z NV_CUDA_LIB_VERSION=11.8.0-1 2024-06-26T06:41:29.6407096Z NVARCH=x86_64 2024-06-26T06:41:29.6407460Z GITHUB_ACTOR_ID=53841472 2024-06-26T06:41:29.6408012Z GITHUB_WORKFLOW_SHA=4b51b1a62a63a1add1b3a0f7882f5c7dc66b8f8d 2024-06-26T06:41:29.6408658Z GITHUB_REF_NAME=129470/merge 2024-06-26T06:41:29.6409206Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-11-8 2024-06-26T06:41:29.6410022Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T06:41:29.6410752Z GITHUB_JOB=test 2024-06-26T06:41:29.6411259Z NV_LIBNCCL_PACKAGE=libnccl2=2.16.2-1+cuda11.8 2024-06-26T06:41:29.6412381Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T06:41:29.6413039Z NO_TEST_TIMEOUT=False 2024-06-26T06:41:29.6413457Z TD_DISTRIBUTED=False 2024-06-26T06:41:29.6413926Z NV_CUDA_NSIGHT_COMPUTE_VERSION=11.8.0-1 2024-06-26T06:41:29.6414483Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T06:41:29.6415199Z NV_NVPROF_VERSION=11.8.87-1 2024-06-26T06:41:29.6415686Z GITHUB_RETENTION_DAYS=90 2024-06-26T06:41:29.6416126Z OPENSSL_DIR=/opt/openssl 2024-06-26T06:41:29.6416561Z GITHUB_ACTION_REPOSITORY= 2024-06-26T06:41:29.6417887Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T06:41:29.6419425Z GITHUB_BASE_REF=gh/leslie-fang-intel/126/base 2024-06-26T06:41:29.6420008Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T06:41:29.6420468Z CI=true 2024-06-26T06:41:29.6420847Z NV_LIBNCCL_PACKAGE_VERSION=2.16.2-1 2024-06-26T06:41:29.6421372Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T06:41:29.6421824Z JOB_ID=26688546814 2024-06-26T06:41:29.6422213Z INSTALLED_PROTOBUF=yes 2024-06-26T06:41:29.6422766Z GITHUB_HEAD_REF=gh/leslie-fang-intel/126/head 2024-06-26T06:41:29.6423299Z GITHUB_ACTION_REF= 2024-06-26T06:41:29.6423848Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T06:41:29.6424449Z GITHUB_WORKFLOW=pull 2024-06-26T06:41:29.6424855Z DEBIAN_FRONTEND=noninteractive 2024-06-26T06:41:29.6426015Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_0f16eff9-544f-4e42-b083-bdb347e91b26 2024-06-26T06:41:29.6427062Z NO_TD=False 2024-06-26T06:41:29.6427436Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T06:41:29.6427868Z _=/usr/bin/env 2024-06-26T06:41:29.6428294Z + echo 'Testing pytorch' 2024-06-26T06:41:29.6428707Z Testing pytorch 2024-06-26T06:41:29.6429118Z + export LANG=C.UTF-8 2024-06-26T06:41:29.6429546Z + LANG=C.UTF-8 2024-06-26T06:41:29.6429897Z + PR_NUMBER=129470 2024-06-26T06:41:29.6430320Z + [[ distributed == \d\e\f\a\u\l\t ]] 2024-06-26T06:41:29.6431026Z + [[ distributed == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-06-26T06:41:29.6431740Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *rocm* ]] 2024-06-26T06:41:29.6432347Z + [[ distributed == \s\l\o\w ]] 2024-06-26T06:41:29.6433046Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *slow-gradcheck* ]] 2024-06-26T06:41:29.6433885Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *cuda* ]] 2024-06-26T06:41:29.6434526Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T06:41:29.6435115Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T06:41:29.6435648Z + [[ distributed == *crossref* ]] 2024-06-26T06:41:29.6436271Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *rocm* ]] 2024-06-26T06:41:29.6437003Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *xpu* ]] 2024-06-26T06:41:29.6437752Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *-bazel-* ]] 2024-06-26T06:41:29.6438412Z + pip_install --user ninja==1.10.2 2024-06-26T06:41:29.6439089Z + pip install --progress-bar off --user ninja==1.10.2 2024-06-26T06:41:30.1612977Z Collecting ninja==1.10.2 2024-06-26T06:41:30.1817235Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-06-26T06:41:30.1987999Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-06-26T06:41:31.3585032Z Installing collected packages: ninja 2024-06-26T06:41:31.3682536Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-06-26T06:41:31.3684176Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-06-26T06:41:31.3738319Z Successfully installed ninja-1.10.2 2024-06-26T06:41:31.4435218Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T06:41:31.4438023Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T06:41:31.4439886Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *aarch64* ]] 2024-06-26T06:41:31.4440477Z + install_tlparse 2024-06-26T06:41:31.4440954Z + pip_install --user tlparse==0.3.7 2024-06-26T06:41:31.4441963Z + pip install --progress-bar off --user tlparse==0.3.7 2024-06-26T06:41:31.9548055Z Collecting tlparse==0.3.7 2024-06-26T06:41:32.0221454Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-06-26T06:41:32.0339411Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-06-26T06:41:33.2013799Z Installing collected packages: tlparse 2024-06-26T06:41:33.2503576Z Successfully installed tlparse-0.3.7 2024-06-26T06:41:33.3222563Z ++ python -m site --user-base 2024-06-26T06:41:33.3448664Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T06:41:33.3451688Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *asan* ]] 2024-06-26T06:41:33.3452821Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *-debug* ]] 2024-06-26T06:41:33.3453517Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *-bazel-* ]] 2024-06-26T06:41:33.3455708Z + echo 'We are not in debug mode: linux-focal-cuda11.8-py3.10-gcc9. Expect the assertion to pass' 2024-06-26T06:41:33.3457088Z We are not in debug mode: linux-focal-cuda11.8-py3.10-gcc9. Expect the assertion to pass 2024-06-26T06:41:33.3457951Z + cd test 2024-06-26T06:41:33.3458613Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-06-26T06:41:35.3862348Z + [[ distributed == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-06-26T06:41:35.3863046Z + [[ distributed == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-06-26T06:41:35.3863923Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *-bazel-* ]] 2024-06-26T06:41:35.3864521Z + pushd test 2024-06-26T06:41:35.3865716Z ~/workspace/test ~/workspace 2024-06-26T06:41:35.3866964Z ++ python -c 'import torch; print(torch.version.cuda)' 2024-06-26T06:41:37.4728941Z + CUDA_VERSION=11.8 2024-06-26T06:41:37.4729594Z + '[' 11.8 == 12.4 ']' 2024-06-26T06:41:37.4730007Z + ISCUDA124= 2024-06-26T06:41:37.4730373Z + popd 2024-06-26T06:41:37.4730696Z ~/workspace 2024-06-26T06:41:37.4735783Z + DYNAMO_BENCHMARK_FLAGS=() 2024-06-26T06:41:37.4736338Z + [[ distributed == *dynamo_eager* ]] 2024-06-26T06:41:37.4736855Z + [[ distributed == *aot_eager* ]] 2024-06-26T06:41:37.4737373Z + [[ distributed == *aot_inductor* ]] 2024-06-26T06:41:37.4737878Z + [[ distributed == *inductor* ]] 2024-06-26T06:41:37.4738375Z + [[ distributed == *dynamic* ]] 2024-06-26T06:41:37.4738879Z + [[ distributed == *cpu_inductor* ]] 2024-06-26T06:41:37.4739491Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-06-26T06:41:37.4772351Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *libtorch* ]] 2024-06-26T06:41:37.4773174Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *-bazel-* ]] 2024-06-26T06:41:37.4775815Z + cd test 2024-06-26T06:41:37.4776447Z + python -c 'import torch; print(torch.__config__.show())' 2024-06-26T06:41:39.3709672Z PyTorch built with: 2024-06-26T06:41:39.3710320Z - GCC 9.4 2024-06-26T06:41:39.3710729Z - C++ Version: 201703 2024-06-26T06:41:39.3711818Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T06:41:39.3737351Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T06:41:39.3738310Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T06:41:39.3738964Z - LAPACK is enabled (usually provided by MKL) 2024-06-26T06:41:39.3739558Z - NNPACK is enabled 2024-06-26T06:41:39.3740019Z - CPU capability usage: AVX2 2024-06-26T06:41:39.3740527Z - CUDA Runtime 11.8 2024-06-26T06:41:39.3741183Z - NVCC architecture flags: -gencode;arch=compute_52,code=sm_52 2024-06-26T06:41:39.3741863Z - CuDNN 90.1 2024-06-26T06:41:39.3742299Z - Magma 2.6.1 2024-06-26T06:41:39.3751528Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-06-26T06:41:39.3760924Z 2024-06-26T06:41:39.6159972Z + cd test 2024-06-26T06:41:39.6160836Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-06-26T06:41:41.4506630Z ATen/Parallel: 2024-06-26T06:41:41.4507413Z at::get_num_threads() : 16 2024-06-26T06:41:41.4507962Z at::get_num_interop_threads() : 16 2024-06-26T06:41:41.4508314Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T06:41:41.4508813Z omp_get_max_threads() : 16 2024-06-26T06:41:41.4510038Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T06:41:41.4511055Z mkl_get_max_threads() : 16 2024-06-26T06:41:41.4512214Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T06:41:41.4513052Z std::thread::hardware_concurrency() : 32 2024-06-26T06:41:41.4513570Z Environment variables: 2024-06-26T06:41:41.4514006Z OMP_NUM_THREADS : [not set] 2024-06-26T06:41:41.4514468Z MKL_NUM_THREADS : [not set] 2024-06-26T06:41:41.4514920Z ATen parallel backend: OpenMP 2024-06-26T06:41:41.4515278Z 2024-06-26T06:41:41.6951685Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *aarch64* ]] 2024-06-26T06:41:41.6952343Z + [[ distributed == *backward* ]] 2024-06-26T06:41:41.6952771Z + [[ distributed == *xla* ]] 2024-06-26T06:41:41.6953250Z + [[ distributed == *executorch* ]] 2024-06-26T06:41:41.6953793Z + [[ distributed == \j\i\t\_\l\e\g\a\c\y ]] 2024-06-26T06:41:41.6954511Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *libtorch* ]] 2024-06-26T06:41:41.6955158Z + [[ distributed == distributed ]] 2024-06-26T06:41:41.6955644Z + test_distributed 2024-06-26T06:41:41.6956120Z + echo 'Testing distributed python tests' 2024-06-26T06:41:41.6956713Z Testing distributed python tests 2024-06-26T06:41:41.6957495Z + python test/run_test.py --distributed-tests --shard 3 3 --verbose 2024-06-26T06:41:41.8240894Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-06-26T06:41:41.8242367Z import pkg_resources 2024-06-26T06:41:46.2124310Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-06-26T06:41:46.2635898Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-06-26T06:41:46.3139066Z Ignoring disabled issues: [''] 2024-06-26T06:41:46.3267299Z Found test times from artifacts 2024-06-26T06:41:46.3766609Z Found test times from artifacts 2024-06-26T06:41:46.3783676Z Running 25% of tests based on TD 2024-06-26T06:41:46.3908930Z Running parallel tests on 2 processes 2024-06-26T06:41:46.3910679Z Name: tests to run (est. time: 6.56min) 2024-06-26T06:41:46.3911258Z Serial tests (18): 2024-06-26T06:41:46.3911894Z distributed/test_dynamo_distributed 1/1 2024-06-26T06:41:46.3912609Z distributed/_tensor/test_attention 1/1 2024-06-26T06:41:46.3913291Z distributed/tensor/parallel/test_micro_pipeline_tp 1/1 2024-06-26T06:41:46.3914298Z distributed/pipelining/test_pipe 1/1 2024-06-26T06:41:46.3914859Z distributed/fsdp/test_fsdp_fx 1/1 2024-06-26T06:41:46.3915462Z distributed/test_c10d_functional_native 1/1 2024-06-26T06:41:46.3916087Z distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 2024-06-26T06:41:46.3916725Z distributed/_spmd/test_data_parallel 1/1 2024-06-26T06:41:46.3917432Z distributed/_tensor/experimental/test_tp_transform 1/1 2024-06-26T06:41:46.3918101Z distributed/_tensor/test_common_rules 1/1 2024-06-26T06:41:46.3918703Z distributed/_tensor/test_random_ops 1/1 2024-06-26T06:41:46.3919299Z distributed/pipelining/test_unflatten 1/1 2024-06-26T06:41:46.3919920Z distributed/checkpoint/test_format_utils 1/1 2024-06-26T06:41:46.3920571Z distributed/pipelining/test_microbatch 1/1 2024-06-26T06:41:46.3921206Z distributed/_tensor/test_convolution_ops 1/1 2024-06-26T06:41:46.3921835Z distributed/checkpoint/test_tp_checkpoint 1/1 2024-06-26T06:41:46.3922554Z distributed/tensor/parallel/test_tp_random_state 1/1 2024-06-26T06:41:46.3923199Z distributed/test_device_mesh 1/1 2024-06-26T06:41:46.3923694Z Parallel tests (0): 2024-06-26T06:41:46.3924167Z Name: excluded (est. time: 75.77min) 2024-06-26T06:41:46.3924675Z Serial tests (51): 2024-06-26T06:41:46.3925129Z distributed/test_distributed_spawn 3/9 2024-06-26T06:41:46.3925695Z distributed/test_distributed_spawn 6/9 2024-06-26T06:41:46.3926275Z distributed/test_distributed_spawn 9/9 2024-06-26T06:41:46.3926837Z distributed/test_c10d_nccl 2/2 2024-06-26T06:41:46.3927339Z distributed/fsdp/test_fsdp_core 2/2 2024-06-26T06:41:46.3928133Z distributed/rpc/cuda/test_tensorpipe_agent 1/2 2024-06-26T06:41:46.3928813Z distributed/fsdp/test_fsdp_optim_state 1/1 2024-06-26T06:41:46.3929475Z distributed/optim/test_zero_redundancy_optimizer 1/1 2024-06-26T06:41:46.3930159Z distributed/fsdp/test_fsdp_comm_hooks 1/1 2024-06-26T06:41:46.3930738Z distributed/fsdp/test_wrap 1/1 2024-06-26T06:41:46.3931245Z distributed/test_c10d_common 1/1 2024-06-26T06:41:46.3932372Z distributed/fsdp/test_fsdp_checkpoint 1/1 2024-06-26T06:41:46.3933047Z distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 2024-06-26T06:41:46.3933790Z distributed/algorithms/quantization/test_quantization 1/1 2024-06-26T06:41:46.3934508Z distributed/_tensor/test_optimizers 1/1 2024-06-26T06:41:46.3935101Z distributed/fsdp/test_fsdp_overlap 1/1 2024-06-26T06:41:46.3935695Z distributed/test_c10d_object_collectives 1/1 2024-06-26T06:41:46.3936328Z distributed/_composable/test_compose 1/1 2024-06-26T06:41:46.3937009Z distributed/tensor/parallel/test_parallelize_api 1/1 2024-06-26T06:41:46.3937781Z distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 2024-06-26T06:41:46.3938516Z distributed/fsdp/test_fsdp_exec_order 1/1 2024-06-26T06:41:46.3939177Z distributed/tensor/parallel/test_tp_examples 1/1 2024-06-26T06:41:46.3939835Z distributed/checkpoint/test_checkpoint 1/1 2024-06-26T06:41:46.3940526Z distributed/checkpoint/test_dtensor_resharding 1/1 2024-06-26T06:41:46.3941303Z distributed/_composable/fully_shard/test_fully_shard_init 1/1 2024-06-26T06:41:46.3942215Z distributed/_composable/fully_shard/test_fully_shard_model_checkpoint 1/1 2024-06-26T06:41:46.3943050Z distributed/checkpoint/test_state_dict_utils 1/1 2024-06-26T06:41:46.3943672Z distributed/test_c10d_logger 1/1 2024-06-26T06:41:46.3944330Z distributed/_composable/fsdp/test_fully_shard_autograd 1/1 2024-06-26T06:41:46.3945026Z distributed/_tensor/test_embedding_ops 1/1 2024-06-26T06:41:46.3945666Z distributed/fsdp/test_fsdp_tp_integration 1/1 2024-06-26T06:41:46.3946388Z distributed/_shard/sharding_plan/test_sharding_plan 1/1 2024-06-26T06:41:46.3947112Z distributed/_shard/sharded_tensor/ops/test_init 1/1 2024-06-26T06:41:46.3947965Z distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 2024-06-26T06:41:46.3948759Z distributed/fsdp/test_fsdp_input 1/1 2024-06-26T06:41:46.3949594Z distributed/_shard/sharded_tensor/ops/test_embedding 1/1 2024-06-26T06:41:46.3950265Z distributed/_shard/test_sharder 1/1 2024-06-26T06:41:46.3950891Z distributed/fsdp/test_fsdp_backward_prefetch 1/1 2024-06-26T06:41:46.3951610Z distributed/_composable/fsdp/test_fully_shard_memory 1/1 2024-06-26T06:41:46.3952489Z distributed/_composable/fully_shard/test_fully_shard_mixed_precision 1/1 2024-06-26T06:41:46.3953403Z distributed/_composable/fully_shard/test_fully_shard_util 1/1 2024-06-26T06:41:46.3954115Z distributed/fsdp/test_fsdp_traversal 1/1 2024-06-26T06:41:46.3954720Z distributed/pipelining/test_stage 1/1 2024-06-26T06:41:46.3955340Z distributed/elastic/utils/distributed_test 1/1 2024-06-26T06:41:46.3955992Z distributed/fsdp/test_checkpoint_wrapper 1/1 2024-06-26T06:41:46.3956621Z distributed/checkpoint/test_planner 1/1 2024-06-26T06:41:46.3957212Z distributed/elastic/events/lib_test 1/1 2024-06-26T06:41:46.3957826Z distributed/test_control_collectives 1/1 2024-06-26T06:41:46.3958403Z distributed/checkpoint/test_utils 1/1 2024-06-26T06:41:46.3958990Z distributed/elastic/metrics/api_test 1/1 2024-06-26T06:41:46.3959614Z distributed/_composable/test_checkpoint 1/1 2024-06-26T06:41:46.3960167Z Parallel tests (0): 2024-06-26T06:41:46.3960934Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-06-26T06:41:46.3986637Z Running distributed/test_dynamo_distributed 1/1 ... [2024-06-26 06:41:46.398285] 2024-06-26T06:41:46.3992026Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/test_dynamo_distributed.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:41:46.398711] 2024-06-26T06:42:42.8277229Z 2024-06-26T06:42:42.8279438Z distributed/test_dynamo_distributed 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_dynamo_distributed_1.1_563955a79eb186bd_.log 2024-06-26T06:42:42.8303938Z Running 38 items in this shard: test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_call_method_forward, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_hf_bert_ddp_aot_eager, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_hf_bert_ddp_inductor, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_issue90375, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_symbol_splitting, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_ddp_activation_checkpointing, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_ddp_baseline_aot_eager_multiprocess, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_activation_checkpointing, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_aot_eager, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_inductor, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_setattr, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_aot_eager, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_aot_eager_static_graph, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_inductor, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_inductor_static_graph, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_fsdp, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_fsdp_activation_checkpointing, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_aot_autograd, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_async_subclass_no_specialize, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_custom_layer, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_ddp_baseline_aot_eager, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_ddp_baseline_inductor, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_empty_graph_inductor, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_dup_tensors_diff_source, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_dup_tensors_same_source, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_orig_params_assert, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_skip_guards, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_skip_register_attr_or_module, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_staticmethod, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_ctx_manager, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor_layout_optimizations_inference, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor_layout_optimizations_training, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor_transpose, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_higher_order_op, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_ignored_parameters, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_no_split 2024-06-26T06:42:42.8326678Z 2024-06-26T06:42:42.8327242Z Running distributed/_tensor/test_attention 1/1 ... [2024-06-26 06:42:42.828116] 2024-06-26T06:42:42.8329834Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_attention.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:42:42.828523] 2024-06-26T06:42:50.3084548Z 2024-06-26T06:42:50.3086352Z distributed/_tensor/test_attention 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_attention_1.1_b2adf35da204bedd_.log 2024-06-26T06:42:50.3094906Z Running 8 items in this shard: test/distributed/_tensor/test_attention.py::RingAttentionTest::test_is_causal_behavior, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_compile_attention_fn0, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_compile_attention_fn1, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_custom_transformer, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_native_transformer_is_causal_False, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_native_transformer_is_causal_True, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_sdpa_is_causal_False, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_sdpa_is_causal_True 2024-06-26T06:42:50.3100556Z 2024-06-26T06:42:50.3101276Z Running distributed/tensor/parallel/test_micro_pipeline_tp 1/1 ... [2024-06-26 06:42:50.308813] 2024-06-26T06:42:50.3103954Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/tensor/parallel/test_micro_pipeline_tp.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:42:50.309332] 2024-06-26T06:42:53.8309100Z 2024-06-26T06:42:53.8311428Z distributed/tensor/parallel/test_micro_pipeline_tp 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_micro_pipeline_tp_1.1_3ca7dcb37ba5773f_.log 2024-06-26T06:42:53.8326279Z Running 15 items in this shard: test/distributed/tensor/parallel/test_micro_pipeline_tp.py::test_with_non_cuda_p2p_group, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_dtensor_seq_par_shard_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_dtensor_seq_par_shard_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_2 2024-06-26T06:42:53.8339891Z 2024-06-26T06:42:53.8340461Z Running distributed/pipelining/test_pipe 1/1 ... [2024-06-26 06:42:53.831290] 2024-06-26T06:42:53.8342892Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/pipelining/test_pipe.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:42:53.832627] 2024-06-26T06:42:58.6567890Z 2024-06-26T06:42:58.6570388Z distributed/pipelining/test_pipe 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_pipe_1.1_319fb4c1f5518e76_.log 2024-06-26T06:42:58.6573974Z Running 3 items in this shard: test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass0, test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass1, test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass2 2024-06-26T06:42:58.6576035Z 2024-06-26T06:42:58.6576566Z Running distributed/fsdp/test_fsdp_fx 1/1 ... [2024-06-26 06:42:58.657057] 2024-06-26T06:42:58.6579258Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/fsdp/test_fsdp_fx.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:42:58.657519] 2024-06-26T06:43:02.4813793Z 2024-06-26T06:43:02.4815554Z distributed/fsdp/test_fsdp_fx 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_fx_1.1_cea4023ee80ef919_.log 2024-06-26T06:43:02.4817661Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_fx.py::TestSymbolicTracing::test_symbolic_tracing_outputs 2024-06-26T06:43:02.4818582Z 2024-06-26T06:43:02.4820640Z Running distributed/test_c10d_functional_native 1/1 ... [2024-06-26 06:43:02.481665] 2024-06-26T06:43:02.4825202Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/test_c10d_functional_native.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:43:02.482080] 2024-06-26T06:43:55.9142780Z 2024-06-26T06:43:55.9145187Z distributed/test_c10d_functional_native 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_functional_native_1.1_1aaa329d14802754_.log 2024-06-26T06:43:55.9160222Z Running 23 items in this shard: test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_gather_into_tensor_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_gather_into_tensor_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_coalesced_, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_single_, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_to_all_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_broadcast, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_reduce_scatter_tensor_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_threading, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_unwaited, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_to_all_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_broadcast, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_inplace_op_on_view, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reduce_scatter_tensor_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reuse_buffer_after_inplace_collective, test/distributed/test_c10d_functional_native.py::CompileTest::test_ranks_and_tag 2024-06-26T06:43:55.9174697Z 2024-06-26T06:43:55.9175320Z Running distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 ... [2024-06-26 06:43:55.914617] 2024-06-26T06:43:55.9177894Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/checkpoint/fsdp/test_fsdp_dsd.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:43:55.915063] 2024-06-26T06:44:14.4682212Z 2024-06-26T06:44:14.4684613Z distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.fsdp.test_fsdp_dsd_1.1_7a8807f2d74f0123_.log 2024-06-26T06:44:14.4688976Z Running 3 items in this shard: test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_1d_fsdp_cpu_offload_full_model_state_dict, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_1d_fsdp_get_model_state_dict, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_save_with_fsdp1_and_load_with_fsdp2 2024-06-26T06:44:14.4692375Z 2024-06-26T06:44:14.4692978Z Running distributed/_spmd/test_data_parallel 1/1 ... [2024-06-26 06:44:14.468534] 2024-06-26T06:44:14.4695401Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_spmd/test_data_parallel.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:44:14.468932] 2024-06-26T06:44:17.7191927Z 2024-06-26T06:44:17.7193967Z distributed/_spmd/test_data_parallel 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._spmd.test_data_parallel_1.1_42fa37584c62cf23_.log 2024-06-26T06:44:17.7195350Z 2024-06-26T06:44:17.7197492Z Running distributed/_tensor/experimental/test_tp_transform 1/1 ... [2024-06-26 06:44:17.719419] 2024-06-26T06:44:17.7202782Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/experimental/test_tp_transform.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:44:17.719836] 2024-06-26T06:44:37.1244679Z 2024-06-26T06:44:37.1246906Z distributed/_tensor/experimental/test_tp_transform 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_tp_transform_1.1_5b511c1aa9d6d8d2_.log 2024-06-26T06:44:37.1250823Z Running 3 items in this shard: test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_e2e, test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_no_bias, test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_with_uncovered_op 2024-06-26T06:44:37.1253692Z 2024-06-26T06:44:37.1254234Z Running distributed/_tensor/test_common_rules 1/1 ... [2024-06-26 06:44:37.124783] 2024-06-26T06:44:37.1257127Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_common_rules.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:44:37.125254] 2024-06-26T06:45:18.1354484Z 2024-06-26T06:45:18.1356301Z distributed/_tensor/test_common_rules 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_common_rules_1.1_8a11c16768321899_.log 2024-06-26T06:45:18.1365034Z Running 10 items in this shard: test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_basic_propagation, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_errors, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_linearity, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_merge_sharding, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_pointwise_propagation, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_enforce_sharding_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_rules_broadcasting, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_rules_suggestion 2024-06-26T06:45:18.1371471Z 2024-06-26T06:45:18.1372344Z Running distributed/_tensor/test_random_ops 1/1 ... [2024-06-26 06:45:18.135678] 2024-06-26T06:45:18.1374784Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_random_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:45:18.136082] 2024-06-26T06:45:50.9191782Z 2024-06-26T06:45:50.9195632Z distributed/_tensor/test_random_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_random_ops_1.1_99811cf09a713da1_.log 2024-06-26T06:45:50.9201192Z Running 7 items in this shard: test/distributed/_tensor/test_random_ops.py::DistTensorRandomInitTest::test_init_ops, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_dropout_1d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_rand_1d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_uniform_2d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_manual_seed, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_meta_tensor_init, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_rng_tracker_init 2024-06-26T06:45:50.9206010Z 2024-06-26T06:45:50.9206576Z Running distributed/pipelining/test_unflatten 1/1 ... [2024-06-26 06:45:50.919415] 2024-06-26T06:45:50.9209152Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/pipelining/test_unflatten.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:45:50.919815] 2024-06-26T06:45:55.4434664Z 2024-06-26T06:45:55.4436854Z distributed/pipelining/test_unflatten 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_unflatten_1.1_39a07358b3e70c01_.log 2024-06-26T06:45:55.4438956Z Running 1 items in this shard: test/distributed/pipelining/test_unflatten.py::UnflattenTests::test_unflatten 2024-06-26T06:45:55.4439773Z 2024-06-26T06:45:55.4441087Z Running distributed/checkpoint/test_format_utils 1/1 ... [2024-06-26 06:45:55.443725] 2024-06-26T06:45:55.4446983Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/checkpoint/test_format_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:45:55.444162] 2024-06-26T06:46:10.7413030Z 2024-06-26T06:46:10.7415755Z distributed/checkpoint/test_format_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_format_utils_1.1_a486d6672c90463c_.log 2024-06-26T06:46:10.7420079Z Running 3 items in this shard: test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_dcp_to_torch_save, test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_online_torch_save_to_dcp, test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_torch_save_to_dcp 2024-06-26T06:46:10.7422865Z 2024-06-26T06:46:10.7423756Z Running distributed/pipelining/test_microbatch 1/1 ... [2024-06-26 06:46:10.741739] 2024-06-26T06:46:10.7426808Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/pipelining/test_microbatch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:46:10.742150] 2024-06-26T06:46:15.1153150Z 2024-06-26T06:46:15.1155350Z distributed/pipelining/test_microbatch 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_microbatch_1.1_916a00b84bb0610c_.log 2024-06-26T06:46:15.1158100Z Running 2 items in this shard: test/distributed/pipelining/test_microbatch.py::MicrobatchTests::test_chunk_spec, test/distributed/pipelining/test_microbatch.py::MicrobatchTests::test_split_and_merge 2024-06-26T06:46:15.1159570Z 2024-06-26T06:46:15.1160161Z Running distributed/_tensor/test_convolution_ops 1/1 ... [2024-06-26 06:46:15.115507] 2024-06-26T06:46:15.1163302Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/_tensor/test_convolution_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:46:15.115905] 2024-06-26T06:46:31.7217431Z 2024-06-26T06:46:31.7219285Z distributed/_tensor/test_convolution_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_convolution_ops_1.1_2f29b6499d1ab5ec_.log 2024-06-26T06:46:31.7222597Z Running 2 items in this shard: test/distributed/_tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_depthwise_convolution, test/distributed/_tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_downsampling_convolution 2024-06-26T06:46:31.7224235Z 2024-06-26T06:46:31.7224853Z Running distributed/checkpoint/test_tp_checkpoint 1/1 ... [2024-06-26 06:46:31.721903] 2024-06-26T06:46:31.7228167Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/checkpoint/test_tp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:46:31.722333] 2024-06-26T06:46:44.7641777Z 2024-06-26T06:46:44.7643674Z distributed/checkpoint/test_tp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_tp_checkpoint_1.1_5ddf1f99b5e758f7_.log 2024-06-26T06:46:44.7646658Z Running 2 items in this shard: test/distributed/checkpoint/test_tp_checkpoint.py::TestTpCheckpoint::test_tp_checkpoint, test/distributed/checkpoint/test_tp_checkpoint.py::TestTpCheckpoint::test_tp_checkpoint_load_on_meta_device 2024-06-26T06:46:44.7648333Z 2024-06-26T06:46:44.7649067Z Running distributed/tensor/parallel/test_tp_random_state 1/1 ... [2024-06-26 06:46:44.764431] 2024-06-26T06:46:44.7653056Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/tensor/parallel/test_tp_random_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:46:44.764832] 2024-06-26T06:46:51.9438020Z 2024-06-26T06:46:51.9440107Z distributed/tensor/parallel/test_tp_random_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_tp_random_state_1.1_cf8c68efaf53b5cd_.log 2024-06-26T06:46:51.9442551Z Running 1 items in this shard: test/distributed/tensor/parallel/test_tp_random_state.py::TensorParallelRandomStateTests::test_model_init 2024-06-26T06:46:51.9443712Z 2024-06-26T06:46:51.9444199Z Running distributed/test_device_mesh 1/1 ... [2024-06-26 06:46:51.944042] 2024-06-26T06:46:51.9449618Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'distributed/test_device_mesh.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:46:51.944488] 2024-06-26T06:49:18.5059980Z 2024-06-26T06:49:18.5062022Z distributed/test_device_mesh 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_device_mesh_1.1_548fa7c063c1c465_.log 2024-06-26T06:49:18.5084741Z Running 38 items in this shard: test/distributed/test_device_mesh.py::DeviceMeshTest::test_assert_invalid_mesh_tensor, test/distributed/test_device_mesh.py::DeviceMeshTest::test_device_mesh_2d, test/distributed/test_device_mesh.py::DeviceMeshTest::test_device_mesh_init_backend, test/distributed/test_device_mesh.py::DeviceMeshTest::test_fake_pg_device_mesh, test/distributed/test_device_mesh.py::DeviceMeshTest::test_from_group_with_global_pg, test/distributed/test_device_mesh.py::DeviceMeshTest::test_from_group_with_invalid_mesh, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_group_and_get_all_groups, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_local_rank, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_local_rank_raises_exception, test/distributed/test_device_mesh.py::DeviceMeshTest::test_init_process_group, test/distributed/test_device_mesh.py::DeviceMeshTest::test_raises_invalid_device_type, test/distributed/test_device_mesh.py::DeviceMeshTest::test_set_mesh_dim_group_options, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_hash, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_nd, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_parent_child_hash, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_from_group_with_mesh_shape, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_get_local_rank_3d, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_init_device_mesh, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_raises_duplicate_mesh_dim_names, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_raises_mesh_shape_mesh_dim_names_mismatch, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_cache_and_reuse_submesh_slice_result, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_1d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_2d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_3d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_raises_invalid_mesh_dim_name, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_raises_no_mesh_dim_found, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_mesh_dim_by_name, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_parent_mesh, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_parent_mesh_dim_exist, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_parent_mesh_dim_not_exist, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_all_gather_uneven, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_broadcast_1d, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_broadcast_nd, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_reduce_scatter_contiguous, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_reduce_scatter_uneven, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_1d, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_nd, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_uneven 2024-06-26T06:49:18.5106330Z 2024-06-26T06:49:19.4255237Z 2024-06-26T06:49:19.4255729Z real 7m37.730s 2024-06-26T06:49:19.4256154Z user 22m55.375s 2024-06-26T06:49:19.4258975Z sys 6m53.794s 2024-06-26T06:49:19.4260274Z + assert_git_not_dirty 2024-06-26T06:49:19.4261289Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *rocm* ]] 2024-06-26T06:49:19.4262335Z + [[ linux-focal-cuda11.8-py3.10-gcc9 != *xla* ]] 2024-06-26T06:49:19.4262896Z ++ git status --porcelain 2024-06-26T06:49:19.4263376Z ++ grep -v '?? third_party' 2024-06-26T06:49:22.3097661Z ++ true 2024-06-26T06:49:22.3098492Z + git_status= 2024-06-26T06:49:22.3099280Z + [[ -n '' ]] 2024-06-26T06:49:22.3099951Z + [[ linux-focal-cuda11.8-py3.10-gcc9 == *cuda* ]] 2024-06-26T06:49:22.3100499Z + [[ 3 == 1 ]] 2024-06-26T06:49:22.3100906Z + [[ 3 == 1 ]] 2024-06-26T06:49:22.3101476Z + cleanup_workspace 2024-06-26T06:49:22.3102416Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-06-26T06:49:22.3103782Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-06-26T06:49:22.3104952Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-06-26T06:49:22.3105768Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-06-26T06:49:22.3106794Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-06-26T06:49:22.3107933Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-06-26T06:49:22.3108796Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-06-26T06:49:22.8696486Z ##[group]Run cat test/**/*_toprint.log || true 2024-06-26T06:49:22.8697160Z cat test/**/*_toprint.log || true 2024-06-26T06:49:22.8705495Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:49:22.8706102Z env: 2024-06-26T06:49:22.8706441Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:22.8707001Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:22.8708144Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:22.8708963Z ##[endgroup] 2024-06-26T06:49:22.8780382Z cat: test/**/*_toprint.log: No such file or directory 2024-06-26T06:49:22.8819560Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-06-26T06:49:22.8820166Z kill "$MONITOR_SCRIPT_PID" 2024-06-26T06:49:22.8828235Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:49:22.8828870Z env: 2024-06-26T06:49:22.8829193Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:22.8829762Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:22.8830693Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:22.8831512Z MONITOR_SCRIPT_PID: 101062 2024-06-26T06:49:22.8831961Z ##[endgroup] 2024-06-26T06:49:22.9075055Z Prepare all required actions 2024-06-26T06:49:22.9075650Z Getting action download info 2024-06-26T06:49:23.0653061Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-06-26T06:49:23.2375733Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-06-26T06:49:23.2376338Z with: 2024-06-26T06:49:23.2376953Z file-suffix: test-distributed-3-3-linux.8xlarge.nvidia.gpu_26688546814 2024-06-26T06:49:23.2377736Z s3-bucket: gha-artifacts 2024-06-26T06:49:23.2378160Z env: 2024-06-26T06:49:23.2378504Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:23.2379090Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:23.2380143Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:23.2380857Z ##[endgroup] 2024-06-26T06:49:23.2414891Z ##[group]Run # Remove any previous test jsons if they exist 2024-06-26T06:49:23.2415719Z # Remove any previous test jsons if they exist 2024-06-26T06:49:23.2416342Z rm -f test-jsons-*.zip 2024-06-26T06:49:23.2416992Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-06-26T06:49:23.2424941Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:49:23.2425549Z env: 2024-06-26T06:49:23.2425888Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:23.2426456Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:23.2427427Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:23.2428507Z FILE_SUFFIX: test-distributed-3-3-linux.8xlarge.nvidia.gpu_26688546814 2024-06-26T06:49:23.2429253Z ##[endgroup] 2024-06-26T06:49:23.2672078Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-06-26T06:49:23.2709231Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-06-26T06:49:23.2710120Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-06-26T06:49:23.2717411Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-06-26T06:49:23.2720015Z adding: test/test-reports/td_exclusions-1ec0d1b112b934227e99.json (deflated 86%) 2024-06-26T06:49:23.2721056Z adding: test/.pytorch-slow-tests.json (deflated 77%) 2024-06-26T06:49:23.2730206Z adding: test/.pytorch-disabled-tests.json (deflated 84%) 2024-06-26T06:49:23.2764213Z ##[group]Run # Remove any previous test reports if they exist 2024-06-26T06:49:23.2765011Z # Remove any previous test reports if they exist 2024-06-26T06:49:23.2765675Z rm -f test-reports-*.zip 2024-06-26T06:49:23.2766415Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-06-26T06:49:23.2774868Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:49:23.2775503Z env: 2024-06-26T06:49:23.2775846Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:23.2776388Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:23.2777334Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:23.2778419Z FILE_SUFFIX: test-distributed-3-3-linux.8xlarge.nvidia.gpu_26688546814 2024-06-26T06:49:23.2779389Z ##[endgroup] 2024-06-26T06:49:23.2991778Z adding: test/test-reports/python-pytest/distributed.test_dynamo_distributed/distributed.test_dynamo_distributed-db30c18d17f6ac5f.xml (deflated 88%) 2024-06-26T06:49:23.2993981Z adding: test/test-reports/python-pytest/distributed._tensor.test_attention/distributed._tensor.test_attention-ff9f5a05c5d44bf9.xml (deflated 86%) 2024-06-26T06:49:23.2996362Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_micro_pipeline_tp/distributed.tensor.parallel.test_micro_pipeline_tp-0161fa7d8c816926.xml (deflated 92%) 2024-06-26T06:49:23.2998744Z adding: test/test-reports/python-pytest/distributed.pipelining.test_pipe/distributed.pipelining.test_pipe-91946c3a68782aac.xml (deflated 72%) 2024-06-26T06:49:23.3001058Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_fx/distributed.fsdp.test_fsdp_fx-41280e027d7149c1.xml (deflated 37%) 2024-06-26T06:49:23.3003234Z adding: test/test-reports/python-pytest/distributed.test_c10d_functional_native/distributed.test_c10d_functional_native-2c15615d45924278.xml (deflated 89%) 2024-06-26T06:49:23.3005546Z adding: test/test-reports/python-pytest/distributed.checkpoint.fsdp.test_fsdp_dsd/distributed.checkpoint.fsdp.test_fsdp_dsd-f4868b05e729939a.xml (deflated 59%) 2024-06-26T06:49:23.3008346Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_tp_transform/distributed._tensor.experimental.test_tp_transform-d0f2924627dee255.xml (deflated 59%) 2024-06-26T06:49:23.3010724Z adding: test/test-reports/python-pytest/distributed._tensor.test_common_rules/distributed._tensor.test_common_rules-3a259cbcd4d1b7e0.xml (deflated 78%) 2024-06-26T06:49:23.3013350Z adding: test/test-reports/python-pytest/distributed._tensor.test_random_ops/distributed._tensor.test_random_ops-371fc9e7f6cdd6f0.xml (deflated 73%) 2024-06-26T06:49:23.3015545Z adding: test/test-reports/python-pytest/distributed.pipelining.test_unflatten/distributed.pipelining.test_unflatten-237e304737935a41.xml (deflated 40%) 2024-06-26T06:49:23.3017860Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_format_utils/distributed.checkpoint.test_format_utils-65d900870678e02c.xml (deflated 57%) 2024-06-26T06:49:23.3020177Z adding: test/test-reports/python-pytest/distributed.pipelining.test_microbatch/distributed.pipelining.test_microbatch-8a9d1cafadae8b81.xml (deflated 48%) 2024-06-26T06:49:23.3022458Z adding: test/test-reports/python-pytest/distributed._tensor.test_convolution_ops/distributed._tensor.test_convolution_ops-3615317719c935e0.xml (deflated 62%) 2024-06-26T06:49:23.3024770Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_tp_checkpoint/distributed.checkpoint.test_tp_checkpoint-86e33e089c8ac116.xml (deflated 52%) 2024-06-26T06:49:23.3027316Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_tp_random_state/distributed.tensor.parallel.test_tp_random_state-051188867ac5d260.xml (deflated 51%) 2024-06-26T06:49:23.3029576Z adding: test/test-reports/python-pytest/distributed.test_device_mesh/distributed.test_device_mesh-b395d67664f31d38.xml (deflated 85%) 2024-06-26T06:49:23.3060864Z ##[group]Run # Remove any previous usage logs if they exist 2024-06-26T06:49:23.3061638Z # Remove any previous usage logs if they exist 2024-06-26T06:49:23.3062271Z rm -f logs-*.zip 2024-06-26T06:49:23.3063115Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-06-26T06:49:23.3064122Z # so check to see if the file exists first 2024-06-26T06:49:23.3064736Z if [ -f 'usage_log.txt' ]; then 2024-06-26T06:49:23.3065400Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-06-26T06:49:23.3066009Z fi 2024-06-26T06:49:23.3066439Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-06-26T06:49:23.3067169Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-06-26T06:49:23.3067789Z fi 2024-06-26T06:49:23.3075183Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:49:23.3075992Z env: 2024-06-26T06:49:23.3076340Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:23.3076887Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:23.3077828Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:23.3078918Z FILE_SUFFIX: test-distributed-3-3-linux.8xlarge.nvidia.gpu_26688546814 2024-06-26T06:49:23.3079648Z ##[endgroup] 2024-06-26T06:49:23.3125683Z adding: usage_log.txt (deflated 92%) 2024-06-26T06:49:23.3342766Z adding: test/test-reports/distributed.test_dynamo_distributed_1.1_563955a79eb186bd_.log (deflated 84%) 2024-06-26T06:49:23.3344280Z adding: test/test-reports/distributed._tensor.test_attention_1.1_b2adf35da204bedd_.log (deflated 76%) 2024-06-26T06:49:23.3346110Z adding: test/test-reports/distributed.tensor.parallel.test_micro_pipeline_tp_1.1_3ca7dcb37ba5773f_.log (deflated 85%) 2024-06-26T06:49:23.3347760Z adding: test/test-reports/distributed.pipelining.test_pipe_1.1_319fb4c1f5518e76_.log (deflated 62%) 2024-06-26T06:49:23.3349230Z adding: test/test-reports/distributed.fsdp.test_fsdp_fx_1.1_cea4023ee80ef919_.log (deflated 53%) 2024-06-26T06:49:23.3350729Z adding: test/test-reports/distributed.test_c10d_functional_native_1.1_1aaa329d14802754_.log (deflated 87%) 2024-06-26T06:49:23.3352309Z adding: test/test-reports/distributed.checkpoint.fsdp.test_fsdp_dsd_1.1_7a8807f2d74f0123_.log (deflated 92%) 2024-06-26T06:49:23.3353828Z adding: test/test-reports/distributed._spmd.test_data_parallel_1.1_42fa37584c62cf23_.log (stored 0%) 2024-06-26T06:49:23.3355419Z adding: test/test-reports/distributed._tensor.experimental.test_tp_transform_1.1_5b511c1aa9d6d8d2_.log (deflated 64%) 2024-06-26T06:49:23.3357032Z adding: test/test-reports/distributed._tensor.test_common_rules_1.1_8a11c16768321899_.log (deflated 75%) 2024-06-26T06:49:23.3358521Z adding: test/test-reports/distributed._tensor.test_random_ops_1.1_99811cf09a713da1_.log (deflated 70%) 2024-06-26T06:49:23.3360050Z adding: test/test-reports/distributed.pipelining.test_unflatten_1.1_39a07358b3e70c01_.log (deflated 54%) 2024-06-26T06:49:23.3361609Z adding: test/test-reports/distributed.checkpoint.test_format_utils_1.1_a486d6672c90463c_.log (deflated 82%) 2024-06-26T06:49:23.3363187Z adding: test/test-reports/distributed.pipelining.test_microbatch_1.1_916a00b84bb0610c_.log (deflated 58%) 2024-06-26T06:49:23.3364725Z adding: test/test-reports/distributed._tensor.test_convolution_ops_1.1_2f29b6499d1ab5ec_.log (deflated 56%) 2024-06-26T06:49:23.3366603Z adding: test/test-reports/distributed.checkpoint.test_tp_checkpoint_1.1_5ddf1f99b5e758f7_.log (deflated 82%) 2024-06-26T06:49:23.3368274Z adding: test/test-reports/distributed.tensor.parallel.test_tp_random_state_1.1_cf8c68efaf53b5cd_.log (deflated 52%) 2024-06-26T06:49:23.3369837Z adding: test/test-reports/distributed.test_device_mesh_1.1_548fa7c063c1c465_.log (deflated 83%) 2024-06-26T06:49:23.3401127Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-06-26T06:49:23.3402030Z # Remove any previous debugging artifacts if they exist 2024-06-26T06:49:23.3402729Z rm -f debug-*.zip 2024-06-26T06:49:23.3403204Z if [ -d 'test/debug' ]; then 2024-06-26T06:49:23.3403818Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-06-26T06:49:23.3404415Z fi 2024-06-26T06:49:23.3412249Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:49:23.3412883Z env: 2024-06-26T06:49:23.3413229Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:23.3413802Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:23.3414725Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:23.3415824Z FILE_SUFFIX: test-distributed-3-3-linux.8xlarge.nvidia.gpu_26688546814 2024-06-26T06:49:23.3416580Z ##[endgroup] 2024-06-26T06:49:23.3520882Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T06:49:23.3521417Z with: 2024-06-26T06:49:23.3521944Z s3-bucket: gha-artifacts 2024-06-26T06:49:23.3522474Z s3-prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:49:23.3523043Z retention-days: 14 2024-06-26T06:49:23.3523450Z if-no-files-found: warn 2024-06-26T06:49:23.3523887Z path: test-jsons-*.zip 2024-06-26T06:49:23.3524285Z name: artifact 2024-06-26T06:49:23.3524652Z region: us-east-1 2024-06-26T06:49:23.3525012Z env: 2024-06-26T06:49:23.3525325Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:23.3525882Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:23.3526878Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:23.3527680Z ##[endgroup] 2024-06-26T06:49:23.8063194Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T06:49:23.8065062Z With the provided path, there will be 1 file uploaded 2024-06-26T06:49:23.8065878Z Uploading to s3 prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:49:23.8113090Z Starting upload of test-jsons-test-distributed-3-3-linux.8xlarge.nvidia.gpu_26688546814.zip 2024-06-26T06:49:23.9517777Z Finished upload of test-jsons-test-distributed-3-3-linux.8xlarge.nvidia.gpu_26688546814.zip 2024-06-26T06:49:23.9682372Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T06:49:23.9682933Z with: 2024-06-26T06:49:23.9683264Z s3-bucket: gha-artifacts 2024-06-26T06:49:23.9683799Z s3-prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:49:23.9684392Z retention-days: 14 2024-06-26T06:49:23.9684800Z if-no-files-found: error 2024-06-26T06:49:23.9685255Z path: test-reports-*.zip 2024-06-26T06:49:23.9685689Z name: artifact 2024-06-26T06:49:23.9686044Z region: us-east-1 2024-06-26T06:49:23.9686411Z env: 2024-06-26T06:49:23.9686736Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:23.9687307Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:23.9688244Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:23.9689089Z ##[endgroup] 2024-06-26T06:49:24.4107256Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T06:49:24.4108578Z With the provided path, there will be 1 file uploaded 2024-06-26T06:49:24.4109377Z Uploading to s3 prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:49:24.4155996Z Starting upload of test-reports-test-distributed-3-3-linux.8xlarge.nvidia.gpu_26688546814.zip 2024-06-26T06:49:24.5577373Z Finished upload of test-reports-test-distributed-3-3-linux.8xlarge.nvidia.gpu_26688546814.zip 2024-06-26T06:49:24.5742303Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T06:49:24.5742869Z with: 2024-06-26T06:49:24.5743216Z s3-bucket: gha-artifacts 2024-06-26T06:49:24.5743749Z s3-prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:49:24.5744336Z retention-days: 14 2024-06-26T06:49:24.5744763Z if-no-files-found: ignore 2024-06-26T06:49:24.5745201Z path: logs-*.zip 2024-06-26T06:49:24.5745586Z name: artifact 2024-06-26T06:49:24.5745953Z region: us-east-1 2024-06-26T06:49:24.5746336Z env: 2024-06-26T06:49:24.5746669Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:24.5747233Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:24.5748180Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:24.5749006Z ##[endgroup] 2024-06-26T06:49:25.0172555Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T06:49:25.0173687Z With the provided path, there will be 1 file uploaded 2024-06-26T06:49:25.0174443Z Uploading to s3 prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:49:25.0222378Z Starting upload of logs-test-distributed-3-3-linux.8xlarge.nvidia.gpu_26688546814.zip 2024-06-26T06:49:25.1377942Z Finished upload of logs-test-distributed-3-3-linux.8xlarge.nvidia.gpu_26688546814.zip 2024-06-26T06:49:25.1542835Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T06:49:25.1543390Z with: 2024-06-26T06:49:25.1543726Z s3-bucket: gha-artifacts 2024-06-26T06:49:25.1544259Z s3-prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:49:25.1545035Z retention-days: 14 2024-06-26T06:49:25.1545456Z if-no-files-found: ignore 2024-06-26T06:49:25.1545905Z path: debug-*.zip 2024-06-26T06:49:25.1546270Z name: artifact 2024-06-26T06:49:25.1546643Z region: us-east-1 2024-06-26T06:49:25.1547011Z env: 2024-06-26T06:49:25.1547328Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:25.1547894Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:25.1548832Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:25.1549637Z ##[endgroup] 2024-06-26T06:49:25.5924349Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-06-26T06:49:25.6109128Z ##[group]Run # shellcheck disable=SC2156 2024-06-26T06:49:25.6109758Z # shellcheck disable=SC2156 2024-06-26T06:49:25.6110770Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-06-26T06:49:25.6119457Z shell: /usr/bin/bash -e {0} 2024-06-26T06:49:25.6119894Z env: 2024-06-26T06:49:25.6120232Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:25.6120777Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:25.6121716Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:25.6122536Z ##[endgroup] 2024-06-26T06:49:25.9075393Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-06-26T06:49:25.9076112Z with: 2024-06-26T06:49:25.9076428Z env: 2024-06-26T06:49:25.9076766Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:25.9077315Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:25.9078252Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:25.9079067Z ##[endgroup] 2024-06-26T06:49:25.9104137Z ##[group]Run set -eou pipefail 2024-06-26T06:49:25.9104641Z set -eou pipefail 2024-06-26T06:49:25.9105072Z  2024-06-26T06:49:25.9105724Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-06-26T06:49:25.9106509Z for _ in $(seq 1440); do 2024-06-26T06:49:25.9107096Z  # Break if no ssh session exists anymore 2024-06-26T06:49:25.9107714Z  if [ "$(who)" = "" ]; then 2024-06-26T06:49:25.9108197Z  break 2024-06-26T06:49:25.9108634Z  fi 2024-06-26T06:49:25.9108987Z  echo "." 2024-06-26T06:49:25.9109396Z  sleep 5 2024-06-26T06:49:25.9109780Z done 2024-06-26T06:49:25.9117787Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:49:25.9118414Z env: 2024-06-26T06:49:25.9118757Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:25.9119302Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:25.9120238Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:25.9121056Z ##[endgroup] 2024-06-26T06:49:25.9146151Z Holding runner for 2 hours until all ssh sessions have logged out 2024-06-26T06:49:25.9279690Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T06:49:25.9280667Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T06:49:25.9281413Z # shellcheck disable=SC2046 2024-06-26T06:49:25.9282000Z docker stop $(docker ps -q) || true 2024-06-26T06:49:25.9282584Z # Prune all of the docker images 2024-06-26T06:49:25.9283142Z docker system prune -af 2024-06-26T06:49:25.9290799Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:49:25.9291410Z env: 2024-06-26T06:49:25.9292167Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:49:25.9292753Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T06:49:25.9293677Z DOCKER_CONTAINER_ID: f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:25.9294503Z ##[endgroup] 2024-06-26T06:49:26.2763266Z f1d373c93f78 2024-06-26T06:49:26.8333918Z Deleted Containers: 2024-06-26T06:49:26.8334878Z f1d373c93f784f5a052460eeb0c307a35a4b15dea91608f3ef3b0f8fd85ace49 2024-06-26T06:49:26.8335787Z 2024-06-26T06:49:33.5550858Z Deleted Images: 2024-06-26T06:49:33.5552568Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:49:33.5555169Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda11.8-cudnn9-py3-gcc9@sha256:493391c3b293fbc39c8ca4208a5b70ab17031329cb9c855a641ab5a208c52fe6 2024-06-26T06:49:33.5556946Z deleted: sha256:1fde65d68e1ba467b7c461d097349da1965328db5e6e5252ccb25702fcc3b83e 2024-06-26T06:49:33.5557969Z deleted: sha256:d7f4fd7abbe071a8cbdda2bae83a151d7bcde19abdef895ea3019cdeadd13a18 2024-06-26T06:49:33.5559087Z deleted: sha256:ee085d46602902c38ef79d90a27d337dae20947f1eeb9c4fccabd1a92e01e58e 2024-06-26T06:49:33.5560143Z deleted: sha256:e860cd33597e4e0f5736fd4d6e2f4702fa16a9940209ad8e214ca7081f44a287 2024-06-26T06:49:33.5561243Z deleted: sha256:a18d95c977fe0d23a7007fdc3ee2379aa8a8c2d42e59e8ef95c3e677d2578fcd 2024-06-26T06:49:33.5562335Z deleted: sha256:52e65c84df1c54be8f442eee573bca8c158f9fc906b5fbee0ff6953553aea0e4 2024-06-26T06:49:33.5563411Z deleted: sha256:bb7adddb154f86f311acf257b7a21b39fa5702dda66249163efd1ae6729d3782 2024-06-26T06:49:33.5564746Z deleted: sha256:1997f0e776385e3c40db4b42b13e2e9006fb6204d0d1dcf32f29db95c117372a 2024-06-26T06:49:33.5565881Z deleted: sha256:cad19b1260ec60216e3d0b105b8f813b992973a7c792a45774531d346384edc1 2024-06-26T06:49:33.5567006Z deleted: sha256:ecc44fe33e9088e5eb858752a97ca6e2d729a64cd42d6a5225b567f8da1ec73f 2024-06-26T06:49:33.5568066Z deleted: sha256:f352c3ff12d96e8bd1de0ee0fddddf2df4aaadbad5433589930269739439c345 2024-06-26T06:49:33.5569090Z deleted: sha256:e6f2acebe916afc5f24b61dd806cd8194326c8dd940a061c8b6acd11a1af4683 2024-06-26T06:49:33.5570174Z deleted: sha256:c94a6342cc6b933354589534bdead86649bbc1464c794cc9d21577fe82437b3f 2024-06-26T06:49:33.5571232Z deleted: sha256:73c84e0e40a121ef226ee59ad4e41c40f6f8e3dd3aa6e40b580b2342181ff842 2024-06-26T06:49:33.5572614Z deleted: sha256:5f650bdb1edb4afa0f22a8c2de8706ea07e360c55298d0bac711f5f4dd750406 2024-06-26T06:49:33.5573709Z deleted: sha256:e3c78b112f621be151f2591f8a82ab325ffedd19be99e3b478c3017e051b5355 2024-06-26T06:49:33.5574767Z deleted: sha256:e058ad0c4f09075c613e2fb7abd0ef0e7465733d555b7778902ed13846dfacb6 2024-06-26T06:49:33.5575851Z deleted: sha256:2b91783cec90ca4caa17a1a03b734c733623e7f8f185334d5bbb21fe9a0f1e5d 2024-06-26T06:49:33.5577002Z deleted: sha256:014aca11d6f33d59cda58c68a53f4a22dadeedb3f6b487fa66f556fcec903607 2024-06-26T06:49:33.5578011Z deleted: sha256:32b4d98aa83b992ac9dd109c618da5a02af41078f6d5bd2a174b69f208fedb0c 2024-06-26T06:49:33.5579086Z deleted: sha256:f4dad79cdbcd5b49f4b3526d3f3c0dae331ddad5ee9be01b21d289ce2be060c0 2024-06-26T06:49:33.5580194Z deleted: sha256:755dbc164caaa04e7f3ac761fcfa41914cb61bbefd288772a7fae92e6429c790 2024-06-26T06:49:33.5581282Z deleted: sha256:3bb6a2f1711ddd563f93bd2b39fb20a667aad822781c1586c9d04a049ffff691 2024-06-26T06:49:33.5582350Z deleted: sha256:25731f18356bbce9f9cf003c08edb416482d8c85becb79fd1af3b83806d147b0 2024-06-26T06:49:33.5583429Z deleted: sha256:6f3c5cbd50ad5b397dcac877079fae290058a3af2bb7c5390c202f221fe1d8f2 2024-06-26T06:49:33.5584498Z deleted: sha256:ea584ac8474876e97af40f7148363e9a6df9e84b52540b61d3488f76fe3fda19 2024-06-26T06:49:33.5585547Z deleted: sha256:c99897453df1a53f8d8caf80177dfabc306d7ad968db5e62b0084b2fcc38ff9b 2024-06-26T06:49:33.5586608Z deleted: sha256:0fa44105790cb410839a6f2cc4410c3be63657d803ef603c9ef7a4b355d5f380 2024-06-26T06:49:33.5587686Z deleted: sha256:2fd5ab6ef7d9cb567b70dddfa5a1a5e65e324446bb14f6feba5599f4e730b35b 2024-06-26T06:49:33.5588779Z deleted: sha256:b99692dfc620bf7adfe66568b6fe1f742be148802aa2d7c6028593bef0210571 2024-06-26T06:49:33.5589946Z deleted: sha256:21daffc134560cc20e34bee6ed4684f5c0de46241501e2a7a89cb7bf546d4227 2024-06-26T06:49:33.5590958Z deleted: sha256:819249fb48326412a787749ddc95957655df6636f2e0f1c5265bf6881c7111e9 2024-06-26T06:49:33.5592209Z deleted: sha256:610d96248b1aa37c8d0b05fdf39c38fa1d491f505e454525de8f8ac3b0bd48f4 2024-06-26T06:49:33.5593258Z deleted: sha256:22f47f538c346eb9b1142a65fdc01f85cd5e10621a8a28cd8aae7e205880bada 2024-06-26T06:49:33.5594338Z deleted: sha256:09c6cd1d8ea04ed3b74420884bf62eaed12a455f041633ebec3a2cf41468e057 2024-06-26T06:49:33.5595433Z deleted: sha256:2f81f71a0bfa140425b1f34efbb9e8a09ce133c20bce173876ae51ec55c53767 2024-06-26T06:49:33.5596472Z deleted: sha256:7083f25055a8577c549ee03196e895f72ec1e26bb509cf4f531e68a9458ec9ff 2024-06-26T06:49:33.5597530Z deleted: sha256:2d353f3abd6ba69b7052291fab5b672c6a31571f9ca14eb579f1324928be9a17 2024-06-26T06:49:33.5598603Z deleted: sha256:f8cb1af08a381526d68d11ccadf89749f855c9f643bced1fa51c146109d894fb 2024-06-26T06:49:33.5599682Z deleted: sha256:538faaa902f68d805ad2a2dc8511f8fea33499e28ed8dd54a2e885b6069b94ed 2024-06-26T06:49:33.5600752Z deleted: sha256:6c0cabeb2da5f02c2781f2f0ca3706fdb328da082a3decdbd8f82ae596f513f3 2024-06-26T06:49:33.5601841Z deleted: sha256:5722d912cb634e82f4826ea4170754898a77efee6df3b6ea282ba78b4aaa43d2 2024-06-26T06:49:33.5602910Z deleted: sha256:dc0beaa37049a8d70d2e4da52db402c3876391d86720df30a9b686ae6ac5cb24 2024-06-26T06:49:33.5603956Z deleted: sha256:a2793888ee929a46917ad906e46fa352b0b00c68298d9c47b995cdd3715f0b44 2024-06-26T06:49:33.5605165Z deleted: sha256:fccc9e0f8fd9de6f71e5b000f1d79435e896c89cd8554c718604b56c993d93ed 2024-06-26T06:49:33.5606260Z deleted: sha256:99df2ce205527dab671443417aa5c24070144ad78f7597f7b01a2fbdc75db4a0 2024-06-26T06:49:33.5607300Z deleted: sha256:ab3da06e7304132443910308818f60abaee5e6aaea7aadc74ff905e3071786a6 2024-06-26T06:49:33.5608369Z deleted: sha256:c43d30eed3c50d7f00e7d83ef78a94dcdf07c763c37d83f9185e72377d1a5f20 2024-06-26T06:49:33.5609436Z deleted: sha256:8f2d8265fe3b438415439e09c19ba6a68b4150eaa562224d7e8ac2710dc2eb14 2024-06-26T06:49:33.5610483Z deleted: sha256:28f2bc432836b7a4424f890747021c36c0b33371c62b6fdb714244d6b90203aa 2024-06-26T06:49:33.5611536Z deleted: sha256:f6286286a1dbe990ec2f0b147e01827e45b747cdb10499b6102daa2b25922727 2024-06-26T06:49:33.5612813Z deleted: sha256:c6e97d8ca374fb6c9ca255b80d6c36af85de5fd66e448ea23491ad9958f19b66 2024-06-26T06:49:33.5613914Z deleted: sha256:b88b1ab78ee5abbcbfd6500e00629d125d67a3037ce62eddb8e0866ffef82fe9 2024-06-26T06:49:33.5614963Z deleted: sha256:1512ffb3610f68874de48684966014158e1bdd95d80f21f97570bd10e88b4ed4 2024-06-26T06:49:33.5616027Z deleted: sha256:fd6f5133b44e40b8da2ecb4666c25328e71d7c29daf9247f740daba07621adc2 2024-06-26T06:49:33.5617093Z deleted: sha256:f9f206b62ad20270a766d66358f0a8cda72f3819c9805b5fe5e929be08487da6 2024-06-26T06:49:33.5618151Z deleted: sha256:f73b05159be211077e8f3d9ff623e66c1c4735d4450aeb21229f951235dc0adb 2024-06-26T06:49:33.5619181Z deleted: sha256:ef71994fba318c56792ce905d971dc876347c60961aa4fd4a2bdf22f026df5ba 2024-06-26T06:49:33.5620240Z deleted: sha256:2c70714aac7394ad9e1e42a55b728f30f38fbd46e545b118bf0d625f4b1eec4e 2024-06-26T06:49:33.5621319Z deleted: sha256:49adb4771e439b38cd6de3c8ef02299fc5672fdfe39ed6a08ecc41893a00d16f 2024-06-26T06:49:33.5622388Z deleted: sha256:fa77ab1294ff3ad236f4a809e697cb4b9b150b11fc7641ea3c508198e0059582 2024-06-26T06:49:33.5623463Z deleted: sha256:06332c1fdd9151f70f43605c3bd5a2967db0f7d03398d97147e645c6ad1034b6 2024-06-26T06:49:33.5624535Z deleted: sha256:c217de6cbf77b426bac257f8f17b0ff3ca3a514a5511deaa7bdfc6f42d405375 2024-06-26T06:49:33.5625597Z deleted: sha256:1248854931e90cf6124b5f6d68fe5bcb43a328f6c246cad949ef775809a2dd40 2024-06-26T06:49:33.5626659Z deleted: sha256:903fa51ff1654718fa673df7ccde5b99f43fc40cd5d5c541f674554c68b9dd00 2024-06-26T06:49:33.5627761Z deleted: sha256:80c567c8b1b790b0859417382b69fb69bd01a15552aadf1ab1aba69882cba1d4 2024-06-26T06:49:33.5628846Z deleted: sha256:8fc1dd27a3b4d2bbf11127923b7bd5063a9faacb77ed389a2eb18bc7d9732abc 2024-06-26T06:49:33.5629916Z deleted: sha256:08147e1d91de147ab86d7cab3a87eac310ab1d06d857b2506890fd3ea12fbf6c 2024-06-26T06:49:33.5630977Z deleted: sha256:89edf8c906678c217b9a88160e1561991f53909c3fd27a4a5e1f0ca0f7ff7888 2024-06-26T06:49:33.5632213Z deleted: sha256:d5ba67f8f9b0c2b6da06d2d5d9d5fedc9f17cbe1a5e13a1a4033a4619f4528ba 2024-06-26T06:49:33.5633265Z deleted: sha256:5d61311e1a293c0c9642e34871719525dfeec7028e4b45e08344a469e3fe24bf 2024-06-26T06:49:33.5634330Z deleted: sha256:b89bb70f745a94c0edd5e974aee294e160735f76fd666a1a06a0b9b1be3caf63 2024-06-26T06:49:33.5635420Z deleted: sha256:9e5477f1f21a38c3dfe21a56a7d77d1b1a42453aed3e11729afcbb9d5731770b 2024-06-26T06:49:33.5636498Z deleted: sha256:b76f4af91cffe2bb7b3f09960f5bdbb26f741419defda65c451c4407fc5b0a92 2024-06-26T06:49:33.5637586Z deleted: sha256:3f8748a00ebd394c0e9d9d481972d764dff58896841ef0150b3f7efaac0f4a4a 2024-06-26T06:49:33.5638639Z deleted: sha256:13989c771e3ba4d40f618344e55e5cb7062a35041b5efd69e2f740ba819f46ee 2024-06-26T06:49:33.5639713Z deleted: sha256:9ebb27837f6c67ad8c17e9af4e63c008faae03fa226c3641dd8bbbfeefb106c4 2024-06-26T06:49:33.5640757Z deleted: sha256:8c16907d735950f8093582bc3cf392b70e002456bf9c4224de5a5b86b78b3ea2 2024-06-26T06:49:33.5641810Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-06-26T06:49:33.5642464Z 2024-06-26T06:49:33.5642656Z Total reclaimed space: 25.98GB 2024-06-26T06:49:33.5727667Z Post job cleanup. 2024-06-26T06:49:33.5793575Z Post job cleanup. 2024-06-26T06:49:33.6969339Z [command]/usr/bin/git version 2024-06-26T06:49:33.7023362Z git version 2.40.1 2024-06-26T06:49:33.7086014Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/28f3c0bd-cb6b-4b51-b2ea-6f66be576d8a' before making global git config changes 2024-06-26T06:49:33.7087642Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T06:49:33.7093956Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T06:49:33.7135772Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T06:49:33.7172139Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T06:49:33.7489049Z Entering 'android/libs/fbjni' 2024-06-26T06:49:33.7540788Z Entering 'third_party/FP16' 2024-06-26T06:49:33.7594166Z Entering 'third_party/FXdiv' 2024-06-26T06:49:33.7647113Z Entering 'third_party/NNPACK' 2024-06-26T06:49:33.7698993Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T06:49:33.7749642Z Entering 'third_party/XNNPACK' 2024-06-26T06:49:33.7821736Z Entering 'third_party/benchmark' 2024-06-26T06:49:33.7873144Z Entering 'third_party/cpp-httplib' 2024-06-26T06:49:33.7923396Z Entering 'third_party/cpuinfo' 2024-06-26T06:49:33.7975697Z Entering 'third_party/cudnn_frontend' 2024-06-26T06:49:33.8026658Z Entering 'third_party/cutlass' 2024-06-26T06:49:33.8085340Z Entering 'third_party/eigen' 2024-06-26T06:49:33.8138532Z Entering 'third_party/fbgemm' 2024-06-26T06:49:33.8191377Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T06:49:33.8240021Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T06:49:33.8290681Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T06:49:33.8348567Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T06:49:33.8401102Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T06:49:33.8453114Z Entering 'third_party/flatbuffers' 2024-06-26T06:49:33.8508795Z Entering 'third_party/fmt' 2024-06-26T06:49:33.8560765Z Entering 'third_party/foxi' 2024-06-26T06:49:33.8611522Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T06:49:33.8663931Z Entering 'third_party/gloo' 2024-06-26T06:49:33.8715565Z Entering 'third_party/googletest' 2024-06-26T06:49:33.8768944Z Entering 'third_party/ideep' 2024-06-26T06:49:33.8818245Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T06:49:33.8877974Z Entering 'third_party/ittapi' 2024-06-26T06:49:33.8928765Z Entering 'third_party/kineto' 2024-06-26T06:49:33.8978555Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T06:49:33.9028024Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T06:49:33.9080931Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T06:49:33.9131054Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T06:49:33.9182331Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T06:49:33.9231396Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T06:49:33.9284462Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T06:49:33.9336076Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T06:49:33.9388679Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T06:49:33.9439087Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T06:49:33.9491136Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T06:49:33.9542151Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T06:49:33.9595113Z Entering 'third_party/mimalloc' 2024-06-26T06:49:33.9648671Z Entering 'third_party/nccl/nccl' 2024-06-26T06:49:33.9700311Z Entering 'third_party/nlohmann' 2024-06-26T06:49:33.9751187Z Entering 'third_party/onnx' 2024-06-26T06:49:33.9821697Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T06:49:33.9872111Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T06:49:33.9924606Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T06:49:33.9977574Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T06:49:34.0027369Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T06:49:34.0077903Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T06:49:34.0127651Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T06:49:34.0180119Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T06:49:34.0230336Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T06:49:34.0280596Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T06:49:34.0330216Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T06:49:34.0383569Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T06:49:34.0435847Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T06:49:34.0508993Z Entering 'third_party/pocketfft' 2024-06-26T06:49:34.0559397Z Entering 'third_party/protobuf' 2024-06-26T06:49:34.0613656Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T06:49:34.0663376Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T06:49:34.0715791Z Entering 'third_party/psimd' 2024-06-26T06:49:34.0766158Z Entering 'third_party/pthreadpool' 2024-06-26T06:49:34.0817243Z Entering 'third_party/pybind11' 2024-06-26T06:49:34.0868057Z Entering 'third_party/python-peachpy' 2024-06-26T06:49:34.0919412Z Entering 'third_party/sleef' 2024-06-26T06:49:34.0970307Z Entering 'third_party/tensorpipe' 2024-06-26T06:49:34.1021687Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T06:49:34.1072096Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T06:49:34.1121489Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T06:49:34.1172267Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T06:49:34.1221724Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T06:49:34.1293631Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T06:49:34.1320167Z http.https://github.com/.extraheader 2024-06-26T06:49:34.1332198Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-06-26T06:49:34.1368426Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T06:49:34.1676659Z Entering 'android/libs/fbjni' 2024-06-26T06:49:34.1709341Z http.https://github.com/.extraheader 2024-06-26T06:49:34.1744400Z Entering 'third_party/FP16' 2024-06-26T06:49:34.1777463Z http.https://github.com/.extraheader 2024-06-26T06:49:34.1811284Z Entering 'third_party/FXdiv' 2024-06-26T06:49:34.1844178Z http.https://github.com/.extraheader 2024-06-26T06:49:34.1878209Z Entering 'third_party/NNPACK' 2024-06-26T06:49:34.1910725Z http.https://github.com/.extraheader 2024-06-26T06:49:34.1945664Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T06:49:34.1978157Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2011468Z Entering 'third_party/XNNPACK' 2024-06-26T06:49:34.2044461Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2098048Z Entering 'third_party/benchmark' 2024-06-26T06:49:34.2130182Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2163880Z Entering 'third_party/cpp-httplib' 2024-06-26T06:49:34.2197358Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2231112Z Entering 'third_party/cpuinfo' 2024-06-26T06:49:34.2263994Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2298293Z Entering 'third_party/cudnn_frontend' 2024-06-26T06:49:34.2330423Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2364361Z Entering 'third_party/cutlass' 2024-06-26T06:49:34.2397749Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2439402Z Entering 'third_party/eigen' 2024-06-26T06:49:34.2472363Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2507571Z Entering 'third_party/fbgemm' 2024-06-26T06:49:34.2540169Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2574002Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T06:49:34.2605460Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2639226Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T06:49:34.2671568Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2704902Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T06:49:34.2736815Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2778015Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T06:49:34.2809494Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2843401Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T06:49:34.2875490Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2908979Z Entering 'third_party/flatbuffers' 2024-06-26T06:49:34.2942052Z http.https://github.com/.extraheader 2024-06-26T06:49:34.2978455Z Entering 'third_party/fmt' 2024-06-26T06:49:34.3010224Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3044491Z Entering 'third_party/foxi' 2024-06-26T06:49:34.3076885Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3110868Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T06:49:34.3143356Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3177685Z Entering 'third_party/gloo' 2024-06-26T06:49:34.3209666Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3243805Z Entering 'third_party/googletest' 2024-06-26T06:49:34.3276671Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3310150Z Entering 'third_party/ideep' 2024-06-26T06:49:34.3342498Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3374906Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T06:49:34.3406150Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3446676Z Entering 'third_party/ittapi' 2024-06-26T06:49:34.3479394Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3512632Z Entering 'third_party/kineto' 2024-06-26T06:49:34.3545087Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3578763Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T06:49:34.3610219Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3644203Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T06:49:34.3676970Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3712604Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T06:49:34.3745427Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3781163Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T06:49:34.3813189Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3848128Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T06:49:34.3880855Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3914017Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T06:49:34.3947357Z http.https://github.com/.extraheader 2024-06-26T06:49:34.3983821Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T06:49:34.4015771Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4049976Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T06:49:34.4082103Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4116433Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T06:49:34.4149459Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4184612Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T06:49:34.4216627Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4252742Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T06:49:34.4285085Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4318608Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T06:49:34.4350079Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4385393Z Entering 'third_party/mimalloc' 2024-06-26T06:49:34.4417938Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4451891Z Entering 'third_party/nccl/nccl' 2024-06-26T06:49:34.4483880Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4517379Z Entering 'third_party/nlohmann' 2024-06-26T06:49:34.4550412Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4586403Z Entering 'third_party/onnx' 2024-06-26T06:49:34.4618406Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4671188Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T06:49:34.4704715Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4738681Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T06:49:34.4770810Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4807791Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T06:49:34.4841241Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4877963Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T06:49:34.4910499Z http.https://github.com/.extraheader 2024-06-26T06:49:34.4944718Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T06:49:34.4976776Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5011228Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T06:49:34.5043284Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5076671Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T06:49:34.5108481Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5144054Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T06:49:34.5175783Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5208829Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T06:49:34.5240622Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5274761Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T06:49:34.5306851Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5339520Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T06:49:34.5371214Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5407024Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T06:49:34.5439430Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5474644Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T06:49:34.5506464Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5562177Z Entering 'third_party/pocketfft' 2024-06-26T06:49:34.5595643Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5630054Z Entering 'third_party/protobuf' 2024-06-26T06:49:34.5663817Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5700682Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T06:49:34.5733144Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5766866Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T06:49:34.5800279Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5835515Z Entering 'third_party/psimd' 2024-06-26T06:49:34.5868440Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5901856Z Entering 'third_party/pthreadpool' 2024-06-26T06:49:34.5934580Z http.https://github.com/.extraheader 2024-06-26T06:49:34.5967660Z Entering 'third_party/pybind11' 2024-06-26T06:49:34.6000398Z http.https://github.com/.extraheader 2024-06-26T06:49:34.6034916Z Entering 'third_party/python-peachpy' 2024-06-26T06:49:34.6067716Z http.https://github.com/.extraheader 2024-06-26T06:49:34.6101031Z Entering 'third_party/sleef' 2024-06-26T06:49:34.6133295Z http.https://github.com/.extraheader 2024-06-26T06:49:34.6166906Z Entering 'third_party/tensorpipe' 2024-06-26T06:49:34.6199473Z http.https://github.com/.extraheader 2024-06-26T06:49:34.6233090Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T06:49:34.6264875Z http.https://github.com/.extraheader 2024-06-26T06:49:34.6299037Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T06:49:34.6331031Z http.https://github.com/.extraheader 2024-06-26T06:49:34.6364671Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T06:49:34.6397238Z http.https://github.com/.extraheader 2024-06-26T06:49:34.6430695Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T06:49:34.6462643Z http.https://github.com/.extraheader 2024-06-26T06:49:34.6495250Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T06:49:34.6526712Z http.https://github.com/.extraheader 2024-06-26T06:49:34.6665240Z A job completed hook has been configured by the self-hosted runner administrator 2024-06-26T06:49:34.6688621Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T06:49:34.6696294Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:49:34.6696951Z ##[endgroup] 2024-06-26T06:49:35.9312404Z Cleaning up orphan processes