2024-08-20T22:09:57.3621929Z Current runner version: '2.319.1' 2024-08-20T22:09:57.3628040Z Runner name: 'i-021b0327168a01512' 2024-08-20T22:09:57.3628841Z Runner group name: 'Default' 2024-08-20T22:09:57.3629743Z Machine name: 'ip-10-0-25-243' 2024-08-20T22:09:57.3645896Z Testing runner upgrade compatibility 2024-08-20T22:09:57.5395254Z ##[group]GITHUB_TOKEN Permissions 2024-08-20T22:09:57.5397425Z Actions: read 2024-08-20T22:09:57.5397923Z Attestations: read 2024-08-20T22:09:57.5398473Z Checks: read 2024-08-20T22:09:57.5399012Z Contents: read 2024-08-20T22:09:57.5399542Z Deployments: read 2024-08-20T22:09:57.5400029Z Discussions: read 2024-08-20T22:09:57.5400589Z Issues: read 2024-08-20T22:09:57.5401092Z Metadata: read 2024-08-20T22:09:57.5401560Z Packages: read 2024-08-20T22:09:57.5402110Z Pages: read 2024-08-20T22:09:57.5402617Z PullRequests: read 2024-08-20T22:09:57.5403172Z RepositoryProjects: read 2024-08-20T22:09:57.5403790Z SecurityEvents: read 2024-08-20T22:09:57.5404357Z Statuses: read 2024-08-20T22:09:57.5405099Z ##[endgroup] 2024-08-20T22:09:57.5408150Z Secret source: Actions 2024-08-20T22:09:57.5408925Z Prepare workflow directory 2024-08-20T22:09:57.6327288Z Prepare all required actions 2024-08-20T22:09:57.6491769Z Getting action download info 2024-08-20T22:09:57.7952368Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-20T22:09:58.2764894Z Download action repository 'pytorch/pytorch@main' (SHA:1ae5d5bb62141d7c9b1b0b66c66a462b4e10b1f2) 2024-08-20T22:10:02.3943024Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-20T22:10:02.5898942Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-20T22:10:02.9411749Z Getting action download info 2024-08-20T22:10:03.0423498Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-20T22:10:03.2425808Z Getting action download info 2024-08-20T22:10:03.3371388Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-20T22:10:03.5893670Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/133712 (40ec5f6ddd9787aca0449b24128343ff4c4a88b3) 2024-08-20T22:10:03.5895904Z ##[group] Inputs 2024-08-20T22:10:03.5896385Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:10:03.5905727Z test-matrix: {"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:10:03.5915773Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:10:03.5916932Z sync-tag: 2024-08-20T22:10:03.5917731Z timeout-minutes: 240 2024-08-20T22:10:03.5918085Z use-gha: 2024-08-20T22:10:03.5918376Z dashboard-tag: 2024-08-20T22:10:03.5918709Z s3-bucket: gha-artifacts 2024-08-20T22:10:03.5919064Z aws-role-to-assume: 2024-08-20T22:10:03.5919394Z ##[endgroup] 2024-08-20T22:10:03.5920242Z Complete job name: cuda12.1-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:10:03.6659419Z A job started hook has been configured by the self-hosted runner administrator 2024-08-20T22:10:03.6800010Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-20T22:10:03.6811793Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:10:03.6812319Z ##[endgroup] 2024-08-20T22:10:04.8312801Z Runner Type: amz2023.linux.g5.4xlarge.nvidia.gpu 2024-08-20T22:10:04.8313362Z Instance Type: g5.4xlarge 2024-08-20T22:10:04.8314085Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-20T22:10:04.8314628Z AMI ID: ami-06c68f701d8090592 2024-08-20T22:10:10.5935951Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-20T22:10:10.5936558Z with: 2024-08-20T22:10:10.5937272Z github-secret: *** 2024-08-20T22:10:10.5938203Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:10:10.5939197Z activate-with-label: false 2024-08-20T22:10:10.5939566Z label: with-ssh 2024-08-20T22:10:10.5939906Z remove-existing-keys: true 2024-08-20T22:10:10.5940272Z fail-silently: true 2024-08-20T22:10:10.5940601Z env: 2024-08-20T22:10:10.5940888Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:10:10.5941242Z ##[endgroup] 2024-08-20T22:10:10.6834982Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-20T22:10:10.6837686Z ciflow reference detected, attempting to extract PR number 2024-08-20T22:10:10.9971189Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-08-20T22:10:11.0840127Z No SSH keys found for user pytorch-bot[bot] 2024-08-20T22:10:11.0841220Z Grabbing public ssh keys from https://github.com/XuehaiPan.keys 2024-08-20T22:10:11.2178610Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-20T22:10:11.2195140Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-20T22:10:11.2224138Z Login using: ssh ec2-user@ec2-44-199-217-221.compute-1.amazonaws.com 2024-08-20T22:10:11.2225013Z All testing is done inside the container, to start an interactive session run: 2024-08-20T22:10:11.2225831Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:10:11.2357686Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-20T22:10:11.2358253Z with: 2024-08-20T22:10:11.2358551Z submodules: recursive 2024-08-20T22:10:11.2358891Z fetch-depth: 0 2024-08-20T22:10:11.2359191Z env: 2024-08-20T22:10:11.2359475Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:10:11.2359820Z ##[endgroup] 2024-08-20T22:10:11.2550416Z ##[group]Run retry () { 2024-08-20T22:10:11.2550824Z retry () { 2024-08-20T22:10:11.2551566Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-20T22:10:11.2552179Z } 2024-08-20T22:10:11.2552494Z echo "${GITHUB_WORKSPACE}" 2024-08-20T22:10:11.2553297Z if [ -z "${NO_SUDO}" ]; then 2024-08-20T22:10:11.2553787Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:10:11.2554252Z else 2024-08-20T22:10:11.2554599Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:10:11.2555030Z fi 2024-08-20T22:10:11.2555379Z mkdir "${GITHUB_WORKSPACE}" 2024-08-20T22:10:11.2569082Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:10:11.2569602Z env: 2024-08-20T22:10:11.2569898Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:10:11.2570255Z NO_SUDO: 2024-08-20T22:10:11.2570547Z ##[endgroup] 2024-08-20T22:10:11.2602975Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:10:11.3972272Z ##[group]Run malfet/checkout@silent-checkout 2024-08-20T22:10:11.3972756Z with: 2024-08-20T22:10:11.3973103Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:10:11.3973553Z fetch-depth: 0 2024-08-20T22:10:11.3973878Z submodules: recursive 2024-08-20T22:10:11.3974223Z quiet-checkout: true 2024-08-20T22:10:11.3974579Z repository: pytorch/pytorch 2024-08-20T22:10:11.3975188Z token: *** 2024-08-20T22:10:11.3975491Z ssh-strict: true 2024-08-20T22:10:11.3975822Z persist-credentials: true 2024-08-20T22:10:11.3976185Z clean: true 2024-08-20T22:10:11.3976526Z sparse-checkout-cone-mode: true 2024-08-20T22:10:11.3976917Z lfs: false 2024-08-20T22:10:11.3977220Z set-safe-directory: true 2024-08-20T22:10:11.3977563Z env: 2024-08-20T22:10:11.3977846Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:10:11.3978185Z ##[endgroup] 2024-08-20T22:10:11.4955003Z Syncing repository: pytorch/pytorch 2024-08-20T22:10:11.4956579Z ##[group]Getting Git version info 2024-08-20T22:10:11.4957426Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:10:11.4958322Z [command]/usr/bin/git version 2024-08-20T22:10:11.4958690Z git version 2.40.1 2024-08-20T22:10:11.4963757Z ##[endgroup] 2024-08-20T22:10:11.4978428Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/5b3609a9-2580-4f92-b986-01d26db28ce8' before making global git config changes 2024-08-20T22:10:11.4979714Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T22:10:11.4983822Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:10:11.5034264Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:10:11.5038293Z ##[group]Initializing the repository 2024-08-20T22:10:11.5041222Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:10:11.5086942Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-20T22:10:11.5087952Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-20T22:10:11.5088824Z hint: of your new repositories, which will suppress this warning, call: 2024-08-20T22:10:11.5089427Z hint: 2024-08-20T22:10:11.5090101Z hint: git config --global init.defaultBranch 2024-08-20T22:10:11.5090589Z hint: 2024-08-20T22:10:11.5091129Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-20T22:10:11.5092021Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-20T22:10:11.5092925Z hint: 2024-08-20T22:10:11.5093272Z hint: git branch -m 2024-08-20T22:10:11.5094045Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-20T22:10:11.5100569Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-20T22:10:11.5143533Z ##[endgroup] 2024-08-20T22:10:11.5144144Z ##[group]Disabling automatic garbage collection 2024-08-20T22:10:11.5146593Z [command]/usr/bin/git config --local gc.auto 0 2024-08-20T22:10:11.5188603Z ##[endgroup] 2024-08-20T22:10:11.5189153Z ##[group]Setting up auth 2024-08-20T22:10:11.5194946Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T22:10:11.5237263Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T22:10:11.5609787Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T22:10:11.5650429Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T22:10:11.6005583Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:10:11.6059599Z ##[endgroup] 2024-08-20T22:10:11.6060352Z ##[group]Fetching the repository 2024-08-20T22:10:11.6065943Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-20T22:10:14.6226006Z remote: Enumerating objects: 1008518 2024-08-20T22:10:14.6226574Z remote: Enumerating objects: 1010229, done. 2024-08-20T22:10:14.6228606Z remote: Counting objects: 0% (1/1711) 2024-08-20T22:10:14.6229194Z remote: Counting objects: 1% (18/1711) 2024-08-20T22:10:14.6229901Z remote: Counting objects: 2% (35/1711) 2024-08-20T22:10:14.6230517Z remote: Counting objects: 3% (52/1711) 2024-08-20T22:10:14.6231033Z remote: Counting objects: 4% (69/1711) 2024-08-20T22:10:14.6231581Z remote: Counting objects: 5% (86/1711) 2024-08-20T22:10:14.6232280Z remote: Counting objects: 6% (103/1711) 2024-08-20T22:10:14.6232970Z remote: Counting objects: 7% (120/1711) 2024-08-20T22:10:14.6233672Z remote: Counting objects: 8% (137/1711) 2024-08-20T22:10:14.6234357Z remote: Counting objects: 9% (154/1711) 2024-08-20T22:10:14.6235038Z remote: Counting objects: 10% (172/1711) 2024-08-20T22:10:14.6235722Z remote: Counting objects: 11% (189/1711) 2024-08-20T22:10:14.6236388Z remote: Counting objects: 12% (206/1711) 2024-08-20T22:10:14.6237079Z remote: Counting objects: 13% (223/1711) 2024-08-20T22:10:14.6237695Z remote: Counting objects: 14% (240/1711) 2024-08-20T22:10:14.6238328Z remote: Counting objects: 15% (257/1711) 2024-08-20T22:10:14.6239014Z remote: Counting objects: 16% (274/1711) 2024-08-20T22:10:14.6239686Z remote: Counting objects: 17% (291/1711) 2024-08-20T22:10:14.6240202Z remote: Counting objects: 18% (308/1711) 2024-08-20T22:10:14.6240722Z remote: Counting objects: 19% (326/1711) 2024-08-20T22:10:14.6241405Z remote: Counting objects: 20% (343/1711) 2024-08-20T22:10:14.6242087Z remote: Counting objects: 21% (360/1711) 2024-08-20T22:10:14.6242820Z remote: Counting objects: 22% (377/1711) 2024-08-20T22:10:14.6243493Z remote: Counting objects: 23% (394/1711) 2024-08-20T22:10:14.6244159Z remote: Counting objects: 24% (411/1711) 2024-08-20T22:10:14.6244846Z remote: Counting objects: 25% (428/1711) 2024-08-20T22:10:14.6245520Z remote: Counting objects: 26% (445/1711) 2024-08-20T22:10:14.6246193Z remote: Counting objects: 27% (462/1711) 2024-08-20T22:10:14.6247584Z remote: Counting objects: 28% (480/1711) 2024-08-20T22:10:14.6248283Z remote: Counting objects: 29% (497/1711) 2024-08-20T22:10:14.6248964Z remote: Counting objects: 30% (514/1711) 2024-08-20T22:10:14.6249635Z remote: Counting objects: 31% (531/1711) 2024-08-20T22:10:14.6250308Z remote: Counting objects: 32% (548/1711) 2024-08-20T22:10:14.6250980Z remote: Counting objects: 33% (565/1711) 2024-08-20T22:10:14.6251853Z remote: Counting objects: 34% (582/1711) 2024-08-20T22:10:14.6252531Z remote: Counting objects: 35% (599/1711) 2024-08-20T22:10:14.6253490Z remote: Counting objects: 36% (616/1711) 2024-08-20T22:10:14.6254172Z remote: Counting objects: 37% (634/1711) 2024-08-20T22:10:14.6254784Z remote: Counting objects: 38% (651/1711) 2024-08-20T22:10:14.6255397Z remote: Counting objects: 39% (668/1711) 2024-08-20T22:10:14.6255909Z remote: Counting objects: 40% (685/1711) 2024-08-20T22:10:14.6256438Z remote: Counting objects: 41% (702/1711) 2024-08-20T22:10:14.6256965Z remote: Counting objects: 42% (719/1711) 2024-08-20T22:10:14.6257487Z remote: Counting objects: 43% (736/1711) 2024-08-20T22:10:14.6257999Z remote: Counting objects: 44% (753/1711) 2024-08-20T22:10:14.6258520Z remote: Counting objects: 45% (770/1711) 2024-08-20T22:10:14.6259043Z remote: Counting objects: 46% (788/1711) 2024-08-20T22:10:14.6259566Z remote: Counting objects: 47% (805/1711) 2024-08-20T22:10:14.6260071Z remote: Counting objects: 48% (822/1711) 2024-08-20T22:10:14.6260587Z remote: Counting objects: 49% (839/1711) 2024-08-20T22:10:14.6261108Z remote: Counting objects: 50% (856/1711) 2024-08-20T22:10:14.6261630Z remote: Counting objects: 51% (873/1711) 2024-08-20T22:10:14.6262140Z remote: Counting objects: 52% (890/1711) 2024-08-20T22:10:14.6262661Z remote: Counting objects: 53% (907/1711) 2024-08-20T22:10:14.6263183Z remote: Counting objects: 54% (924/1711) 2024-08-20T22:10:14.6263700Z remote: Counting objects: 55% (942/1711) 2024-08-20T22:10:14.6264207Z remote: Counting objects: 56% (959/1711) 2024-08-20T22:10:14.6264730Z remote: Counting objects: 57% (976/1711) 2024-08-20T22:10:14.6265248Z remote: Counting objects: 58% (993/1711) 2024-08-20T22:10:14.6265779Z remote: Counting objects: 59% (1010/1711) 2024-08-20T22:10:14.6266323Z remote: Counting objects: 60% (1027/1711) 2024-08-20T22:10:14.6266869Z remote: Counting objects: 61% (1044/1711) 2024-08-20T22:10:14.6267417Z remote: Counting objects: 62% (1061/1711) 2024-08-20T22:10:14.6267973Z remote: Counting objects: 63% (1078/1711) 2024-08-20T22:10:14.6268519Z remote: Counting objects: 64% (1096/1711) 2024-08-20T22:10:14.6269053Z remote: Counting objects: 65% (1113/1711) 2024-08-20T22:10:14.6269603Z remote: Counting objects: 66% (1130/1711) 2024-08-20T22:10:14.6270151Z remote: Counting objects: 67% (1147/1711) 2024-08-20T22:10:14.6270699Z remote: Counting objects: 68% (1164/1711) 2024-08-20T22:10:14.6271237Z remote: Counting objects: 69% (1181/1711) 2024-08-20T22:10:14.6271776Z remote: Counting objects: 70% (1198/1711) 2024-08-20T22:10:14.6272312Z remote: Counting objects: 71% (1215/1711) 2024-08-20T22:10:14.6272862Z remote: Counting objects: 72% (1232/1711) 2024-08-20T22:10:14.6273464Z remote: Counting objects: 73% (1250/1711) 2024-08-20T22:10:14.6273998Z remote: Counting objects: 74% (1267/1711) 2024-08-20T22:10:14.6274537Z remote: Counting objects: 75% (1284/1711) 2024-08-20T22:10:14.6275077Z remote: Counting objects: 76% (1301/1711) 2024-08-20T22:10:14.6275609Z remote: Counting objects: 77% (1318/1711) 2024-08-20T22:10:14.6276152Z remote: Counting objects: 78% (1335/1711) 2024-08-20T22:10:14.6276856Z remote: Counting objects: 79% (1352/1711) 2024-08-20T22:10:14.6277395Z remote: Counting objects: 80% (1369/1711) 2024-08-20T22:10:14.6277930Z remote: Counting objects: 81% (1386/1711) 2024-08-20T22:10:14.6278464Z remote: Counting objects: 82% (1404/1711) 2024-08-20T22:10:14.6278996Z remote: Counting objects: 83% (1421/1711) 2024-08-20T22:10:14.6279706Z remote: Counting objects: 84% (1438/1711) 2024-08-20T22:10:14.6280405Z remote: Counting objects: 85% (1455/1711) 2024-08-20T22:10:14.6281116Z remote: Counting objects: 86% (1472/1711) 2024-08-20T22:10:14.6281645Z remote: Counting objects: 87% (1489/1711) 2024-08-20T22:10:14.6282175Z remote: Counting objects: 88% (1506/1711) 2024-08-20T22:10:14.6282698Z remote: Counting objects: 89% (1523/1711) 2024-08-20T22:10:14.6283237Z remote: Counting objects: 90% (1540/1711) 2024-08-20T22:10:14.6283819Z remote: Counting objects: 91% (1558/1711) 2024-08-20T22:10:14.6284360Z remote: Counting objects: 92% (1575/1711) 2024-08-20T22:10:14.6284883Z remote: Counting objects: 93% (1592/1711) 2024-08-20T22:10:14.6285417Z remote: Counting objects: 94% (1609/1711) 2024-08-20T22:10:14.6285952Z remote: Counting objects: 95% (1626/1711) 2024-08-20T22:10:14.6286480Z remote: Counting objects: 96% (1643/1711) 2024-08-20T22:10:14.6287006Z remote: Counting objects: 97% (1660/1711) 2024-08-20T22:10:14.6287557Z remote: Counting objects: 98% (1677/1711) 2024-08-20T22:10:14.6288095Z remote: Counting objects: 99% (1694/1711) 2024-08-20T22:10:14.6288626Z remote: Counting objects: 100% (1711/1711) 2024-08-20T22:10:14.6289192Z remote: Counting objects: 100% (1711/1711), done. 2024-08-20T22:10:14.6350030Z remote: Compressing objects: 0% (1/885) 2024-08-20T22:10:14.6664590Z remote: Compressing objects: 1% (9/885) 2024-08-20T22:10:14.6837179Z remote: Compressing objects: 2% (18/885) 2024-08-20T22:10:14.7330334Z remote: Compressing objects: 3% (27/885) 2024-08-20T22:10:14.7890145Z remote: Compressing objects: 4% (36/885) 2024-08-20T22:10:14.8565642Z remote: Compressing objects: 5% (45/885) 2024-08-20T22:10:14.8995129Z remote: Compressing objects: 6% (54/885) 2024-08-20T22:10:14.9414683Z remote: Compressing objects: 7% (62/885) 2024-08-20T22:10:14.9847540Z remote: Compressing objects: 8% (71/885) 2024-08-20T22:10:15.0150038Z remote: Compressing objects: 9% (80/885) 2024-08-20T22:10:15.0420908Z remote: Compressing objects: 10% (89/885) 2024-08-20T22:10:15.0723385Z remote: Compressing objects: 11% (98/885) 2024-08-20T22:10:15.0945552Z remote: Compressing objects: 12% (107/885) 2024-08-20T22:10:15.1114734Z remote: Compressing objects: 13% (116/885) 2024-08-20T22:10:15.1281586Z remote: Compressing objects: 14% (124/885) 2024-08-20T22:10:15.1411666Z remote: Compressing objects: 15% (133/885) 2024-08-20T22:10:15.1493632Z remote: Compressing objects: 16% (142/885) 2024-08-20T22:10:15.1586410Z remote: Compressing objects: 17% (151/885) 2024-08-20T22:10:15.1651080Z remote: Compressing objects: 18% (160/885) 2024-08-20T22:10:15.1699833Z remote: Compressing objects: 19% (169/885) 2024-08-20T22:10:15.1742342Z remote: Compressing objects: 20% (177/885) 2024-08-20T22:10:15.1769393Z remote: Compressing objects: 21% (186/885) 2024-08-20T22:10:15.1787201Z remote: Compressing objects: 22% (195/885) 2024-08-20T22:10:15.1790226Z remote: Compressing objects: 23% (204/885) 2024-08-20T22:10:15.1798348Z remote: Compressing objects: 24% (213/885) 2024-08-20T22:10:15.1806532Z remote: Compressing objects: 25% (222/885) 2024-08-20T22:10:15.1812118Z remote: Compressing objects: 26% (231/885) 2024-08-20T22:10:15.1819792Z remote: Compressing objects: 27% (239/885) 2024-08-20T22:10:15.1833298Z remote: Compressing objects: 28% (248/885) 2024-08-20T22:10:15.1849829Z remote: Compressing objects: 29% (257/885) 2024-08-20T22:10:15.1862125Z remote: Compressing objects: 30% (266/885) 2024-08-20T22:10:15.1873585Z remote: Compressing objects: 31% (275/885) 2024-08-20T22:10:15.1883091Z remote: Compressing objects: 32% (284/885) 2024-08-20T22:10:15.1889747Z remote: Compressing objects: 33% (293/885) 2024-08-20T22:10:15.1896852Z remote: Compressing objects: 34% (301/885) 2024-08-20T22:10:15.1907282Z remote: Compressing objects: 35% (310/885) 2024-08-20T22:10:15.1916716Z remote: Compressing objects: 36% (319/885) 2024-08-20T22:10:15.1928992Z remote: Compressing objects: 37% (328/885) 2024-08-20T22:10:15.1939224Z remote: Compressing objects: 38% (337/885) 2024-08-20T22:10:15.1955542Z remote: Compressing objects: 39% (346/885) 2024-08-20T22:10:15.1968806Z remote: Compressing objects: 40% (354/885) 2024-08-20T22:10:15.1970775Z remote: Compressing objects: 41% (363/885) 2024-08-20T22:10:15.1972060Z remote: Compressing objects: 42% (372/885) 2024-08-20T22:10:15.1980240Z remote: Compressing objects: 43% (381/885) 2024-08-20T22:10:15.1988493Z remote: Compressing objects: 44% (390/885) 2024-08-20T22:10:15.1996189Z remote: Compressing objects: 45% (399/885) 2024-08-20T22:10:15.2000107Z remote: Compressing objects: 46% (408/885) 2024-08-20T22:10:15.2006263Z remote: Compressing objects: 47% (416/885) 2024-08-20T22:10:15.2010582Z remote: Compressing objects: 48% (425/885) 2024-08-20T22:10:15.2020055Z remote: Compressing objects: 49% (434/885) 2024-08-20T22:10:15.2022801Z remote: Compressing objects: 50% (443/885) 2024-08-20T22:10:15.2023560Z remote: Compressing objects: 51% (452/885) 2024-08-20T22:10:15.2026293Z remote: Compressing objects: 52% (461/885) 2024-08-20T22:10:15.2029173Z remote: Compressing objects: 53% (470/885) 2024-08-20T22:10:15.2031417Z remote: Compressing objects: 54% (478/885) 2024-08-20T22:10:15.2033563Z remote: Compressing objects: 55% (487/885) 2024-08-20T22:10:15.2035313Z remote: Compressing objects: 56% (496/885) 2024-08-20T22:10:15.2037074Z remote: Compressing objects: 57% (505/885) 2024-08-20T22:10:15.2037902Z remote: Compressing objects: 58% (514/885) 2024-08-20T22:10:15.2039195Z remote: Compressing objects: 59% (523/885) 2024-08-20T22:10:15.2039938Z remote: Compressing objects: 60% (531/885) 2024-08-20T22:10:15.2040638Z remote: Compressing objects: 61% (540/885) 2024-08-20T22:10:15.2041357Z remote: Compressing objects: 62% (549/885) 2024-08-20T22:10:15.2042065Z remote: Compressing objects: 63% (558/885) 2024-08-20T22:10:15.2048060Z remote: Compressing objects: 64% (567/885) 2024-08-20T22:10:15.2058114Z remote: Compressing objects: 65% (576/885) 2024-08-20T22:10:15.2064849Z remote: Compressing objects: 66% (585/885) 2024-08-20T22:10:15.2070605Z remote: Compressing objects: 67% (593/885) 2024-08-20T22:10:15.2077193Z remote: Compressing objects: 68% (602/885) 2024-08-20T22:10:15.2082521Z remote: Compressing objects: 69% (611/885) 2024-08-20T22:10:15.2088712Z remote: Compressing objects: 70% (620/885) 2024-08-20T22:10:15.2093295Z remote: Compressing objects: 71% (629/885) 2024-08-20T22:10:15.2098212Z remote: Compressing objects: 72% (638/885) 2024-08-20T22:10:15.2104482Z remote: Compressing objects: 73% (647/885) 2024-08-20T22:10:15.2106891Z remote: Compressing objects: 74% (655/885) 2024-08-20T22:10:15.2110542Z remote: Compressing objects: 75% (664/885) 2024-08-20T22:10:15.2114293Z remote: Compressing objects: 76% (673/885) 2024-08-20T22:10:15.2118521Z remote: Compressing objects: 77% (682/885) 2024-08-20T22:10:15.2121251Z remote: Compressing objects: 78% (691/885) 2024-08-20T22:10:15.2123793Z remote: Compressing objects: 79% (700/885) 2024-08-20T22:10:15.2127011Z remote: Compressing objects: 80% (708/885) 2024-08-20T22:10:15.2129962Z remote: Compressing objects: 81% (717/885) 2024-08-20T22:10:15.2132737Z remote: Compressing objects: 82% (726/885) 2024-08-20T22:10:15.2135742Z remote: Compressing objects: 83% (735/885) 2024-08-20T22:10:15.2137486Z remote: Compressing objects: 84% (744/885) 2024-08-20T22:10:15.2140411Z remote: Compressing objects: 85% (753/885) 2024-08-20T22:10:15.2143154Z remote: Compressing objects: 86% (762/885) 2024-08-20T22:10:15.2145410Z remote: Compressing objects: 87% (770/885) 2024-08-20T22:10:15.2148632Z remote: Compressing objects: 88% (779/885) 2024-08-20T22:10:15.2151325Z remote: Compressing objects: 89% (788/885) 2024-08-20T22:10:15.2152938Z remote: Compressing objects: 90% (797/885) 2024-08-20T22:10:15.2154467Z remote: Compressing objects: 91% (806/885) 2024-08-20T22:10:15.2155654Z remote: Compressing objects: 92% (815/885) 2024-08-20T22:10:15.2156387Z remote: Compressing objects: 93% (824/885) 2024-08-20T22:10:15.2158255Z remote: Compressing objects: 94% (832/885) 2024-08-20T22:10:15.2160341Z remote: Compressing objects: 95% (841/885) 2024-08-20T22:10:15.2161788Z remote: Compressing objects: 96% (850/885) 2024-08-20T22:10:15.2163832Z remote: Compressing objects: 97% (859/885) 2024-08-20T22:10:15.2165010Z remote: Compressing objects: 98% (868/885) 2024-08-20T22:10:15.2168647Z remote: Compressing objects: 99% (877/885) 2024-08-20T22:10:15.2169381Z remote: Compressing objects: 100% (885/885) 2024-08-20T22:10:15.2170059Z remote: Compressing objects: 100% (885/885), done. 2024-08-20T22:10:35.6673108Z remote: Total 1010229 (delta 1101), reused 1314 (delta 823), pack-reused 1008518 (from 1) 2024-08-20T22:10:56.7675635Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-20T22:10:56.7715618Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:10:56.7721411Z ##[endgroup] 2024-08-20T22:10:56.7726228Z ##[group]Determining the checkout info 2024-08-20T22:10:56.7726927Z ##[endgroup] 2024-08-20T22:10:56.7727471Z ##[group]Checking out the ref 2024-08-20T22:10:56.7728439Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:10:58.5041333Z ##[endgroup] 2024-08-20T22:10:58.5041956Z ##[group]Setting up auth for fetching submodules 2024-08-20T22:10:58.5048787Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:10:58.5109749Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-20T22:10:58.5149482Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-20T22:10:58.5189170Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-20T22:10:58.5226968Z ##[endgroup] 2024-08-20T22:10:58.5227547Z ##[group]Fetching submodules 2024-08-20T22:10:58.5229809Z [command]/usr/bin/git submodule sync --recursive 2024-08-20T22:10:58.5607830Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-20T22:10:58.5973757Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-20T22:10:58.5975627Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-20T22:10:58.5979148Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-20T22:10:58.5982176Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-20T22:10:58.5986168Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-20T22:10:58.5988468Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-20T22:10:58.5992307Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-20T22:10:58.5995778Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-20T22:10:58.5999257Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-20T22:10:58.6002934Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-20T22:10:58.6006541Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-20T22:10:58.6010166Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-20T22:10:58.6013989Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-20T22:10:58.6018106Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-20T22:10:58.6021902Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-20T22:10:58.6027599Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:10:58.6034839Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-20T22:10:58.6039037Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-20T22:10:58.6043225Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-20T22:10:58.6047751Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-20T22:10:58.6052098Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-20T22:10:58.6057499Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-20T22:10:58.6061716Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-20T22:10:58.6066309Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-20T22:10:58.6070813Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-20T22:10:58.6075831Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-20T22:10:58.6080385Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-20T22:10:58.6085281Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-20T22:10:58.6090019Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-20T22:10:58.6095077Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-20T22:10:58.6100347Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-20T22:10:58.6105728Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-20T22:10:58.6110547Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-20T22:10:58.6119044Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-20T22:10:58.6153867Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-20T22:10:58.8871803Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-20T22:10:59.0586372Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-20T22:10:59.2362601Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-20T22:10:59.4775967Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-20T22:11:01.4631796Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-20T22:11:12.7053483Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-20T22:11:13.1181855Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-20T22:11:13.5568631Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-20T22:11:14.1408788Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-20T22:11:15.3357846Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-20T22:11:17.4648439Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-20T22:11:23.3186346Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-20T22:11:24.7343609Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-20T22:11:26.4054681Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-20T22:11:27.5673875Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-20T22:11:28.0063432Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-20T22:11:28.3208123Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-20T22:11:29.3824629Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-20T22:11:29.7531386Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-20T22:11:30.0140504Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-20T22:11:31.7035741Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-20T22:11:32.4860683Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-20T22:11:33.2223247Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-20T22:11:38.8741590Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-20T22:11:40.7628751Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-20T22:11:44.9615727Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-20T22:11:45.2178139Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-20T22:11:53.9385609Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-20T22:11:54.1023992Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-20T22:11:54.3264202Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-20T22:11:55.2641830Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-20T22:11:55.5455029Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-20T22:11:56.1526822Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-20T22:11:56.6587800Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-20T22:11:56.6738939Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-20T22:11:56.6857260Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-20T22:11:56.7171634Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-20T22:11:56.7633894Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-20T22:11:57.9012713Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-20T22:11:57.9299689Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-20T22:11:57.9824826Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-20T22:11:58.0945243Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-20T22:11:58.1347065Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-20T22:11:58.7575909Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-20T22:11:59.0425939Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-20T22:11:59.1363460Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-20T22:11:59.1386199Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:11:59.1389537Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:11:59.1392930Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:11:59.1396406Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:11:59.1400078Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:11:59.1434137Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-20T22:12:00.2042600Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-20T22:12:00.8069920Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-20T22:12:02.7511755Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-20T22:12:03.7778087Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-20T22:12:04.1320750Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-20T22:12:04.2404796Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-20T22:12:04.7459817Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-20T22:12:04.8165966Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-20T22:12:04.8314516Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-20T22:12:04.9805096Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-20T22:12:05.0262920Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-20T22:12:05.0721081Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-20T22:12:05.1036240Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-20T22:12:05.1556413Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:12:05.1712945Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-20T22:12:05.1732340Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-20T22:12:05.1763736Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-20T22:12:17.8205744Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-20T22:12:17.8422069Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-20T22:12:17.9422595Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-20T22:12:17.9444058Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:12:17.9447541Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:12:17.9451607Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:12:17.9484787Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-20T22:12:18.4709280Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-20T22:12:19.6853576Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-20T22:12:20.8191157Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-20T22:12:20.8209980Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:12:20.8213277Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:12:20.8216798Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:12:20.8221012Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:12:20.8224631Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:12:20.8228428Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:12:20.8232156Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:12:20.8235917Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:12:20.8269338Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-20T22:12:21.6474437Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-20T22:12:22.0143939Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-20T22:12:23.1793966Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-20T22:12:23.4806255Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-20T22:12:24.0016756Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-20T22:12:25.0458334Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-20T22:12:32.3544367Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-20T22:12:32.8467829Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-20T22:12:32.8699707Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-20T22:12:32.9139342Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-20T22:12:32.9315855Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-20T22:12:32.9336597Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:12:32.9369913Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-20T22:12:33.3250219Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-20T22:12:33.3477108Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-20T22:12:33.3956208Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-20T22:12:33.5143600Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-20T22:12:33.5343126Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-20T22:12:33.5791980Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-20T22:12:33.6449565Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-20T22:12:33.6896024Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-20T22:12:33.7194543Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-20T22:12:33.8452300Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-20T22:12:34.3577328Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-20T22:12:34.3616824Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-20T22:12:34.3619367Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-20T22:12:34.3650795Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-20T22:12:34.8107964Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-20T22:12:35.8334804Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-20T22:12:35.8743929Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-20T22:12:35.9633398Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-20T22:12:35.9654724Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:12:35.9657980Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:12:35.9661276Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:12:35.9665082Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:12:35.9668747Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:12:35.9672288Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:12:35.9675926Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:12:35.9679522Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:12:35.9712083Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-20T22:12:36.3698868Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-20T22:12:37.4600465Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-20T22:12:37.7544893Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-20T22:12:43.9061840Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-20T22:12:44.2021547Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-20T22:12:44.3931872Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-20T22:12:44.6946741Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-20T22:12:51.4424094Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-20T22:12:51.4904359Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-20T22:12:51.5090768Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-20T22:12:51.6321857Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-20T22:12:51.6481782Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-20T22:12:51.6670714Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-20T22:12:51.6870855Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-20T22:12:51.6889965Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:12:51.6893225Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:12:51.6926563Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-20T22:12:53.5692872Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-20T22:12:54.9015027Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-20T22:12:54.9549650Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:12:55.5986404Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-20T22:12:55.6133225Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-20T22:12:55.9289687Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-20T22:12:55.9315067Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:12:55.9318107Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-20T22:12:55.9350511Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-20T22:12:56.3609902Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-20T22:12:57.4111451Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-20T22:12:57.4929255Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-20T22:12:57.5049111Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-20T22:12:57.5201180Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-20T22:12:57.5648021Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-20T22:12:57.5993115Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-20T22:12:57.6480196Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-20T22:12:57.6818686Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-20T22:12:57.6839978Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:12:57.6843021Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:12:57.6846584Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:12:57.6849937Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:12:57.6881578Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-20T22:12:58.7127492Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-20T22:12:58.9482081Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-20T22:13:00.1488932Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-20T22:13:01.1374074Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-20T22:13:01.1568376Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-20T22:13:01.2358255Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-20T22:13:01.2709888Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-20T22:13:01.2726426Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:13:01.2758179Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-20T22:13:01.5167424Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-20T22:13:01.5218966Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-20T22:13:01.5585931Z Entering 'android/libs/fbjni' 2024-08-20T22:13:01.5638766Z Entering 'third_party/FP16' 2024-08-20T22:13:01.5692142Z Entering 'third_party/FXdiv' 2024-08-20T22:13:01.5744622Z Entering 'third_party/NNPACK' 2024-08-20T22:13:01.5798641Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:13:01.5850750Z Entering 'third_party/XNNPACK' 2024-08-20T22:13:01.5923730Z Entering 'third_party/benchmark' 2024-08-20T22:13:01.5976661Z Entering 'third_party/cpp-httplib' 2024-08-20T22:13:01.6029263Z Entering 'third_party/cpuinfo' 2024-08-20T22:13:01.6082592Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:13:01.6135782Z Entering 'third_party/cutlass' 2024-08-20T22:13:01.6196918Z Entering 'third_party/eigen' 2024-08-20T22:13:01.6250965Z Entering 'third_party/fbgemm' 2024-08-20T22:13:01.6303173Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:13:01.6354066Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:13:01.6405408Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:13:01.6464393Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:13:01.6514541Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:13:01.6567490Z Entering 'third_party/flatbuffers' 2024-08-20T22:13:01.6623570Z Entering 'third_party/fmt' 2024-08-20T22:13:01.6676861Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:13:01.6729674Z Entering 'third_party/gloo' 2024-08-20T22:13:01.6783482Z Entering 'third_party/googletest' 2024-08-20T22:13:01.6836154Z Entering 'third_party/ideep' 2024-08-20T22:13:01.6886514Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:13:01.6945364Z Entering 'third_party/ittapi' 2024-08-20T22:13:01.6998610Z Entering 'third_party/kineto' 2024-08-20T22:13:01.7049876Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:13:01.7103802Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:13:01.7158417Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:13:01.7210124Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:13:01.7262528Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:13:01.7312722Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:13:01.7370300Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:13:01.7422636Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:13:01.7475408Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:13:01.7528577Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:13:01.7584154Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:13:01.7635344Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:13:01.7690032Z Entering 'third_party/mimalloc' 2024-08-20T22:13:01.7742548Z Entering 'third_party/nccl/nccl' 2024-08-20T22:13:01.7795868Z Entering 'third_party/nlohmann' 2024-08-20T22:13:01.7849393Z Entering 'third_party/onnx' 2024-08-20T22:13:01.7920654Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:13:01.7974423Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:13:01.8029448Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:13:01.8082763Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:13:01.8133431Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:13:01.8183681Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:13:01.8233807Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:13:01.8286915Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:13:01.8336887Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:13:01.8387357Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:13:01.8436603Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:13:01.8490908Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:13:01.8543700Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:13:01.8616509Z Entering 'third_party/pocketfft' 2024-08-20T22:13:01.8668385Z Entering 'third_party/protobuf' 2024-08-20T22:13:01.8722910Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:13:01.8772406Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:13:01.8829540Z Entering 'third_party/psimd' 2024-08-20T22:13:01.8881889Z Entering 'third_party/pthreadpool' 2024-08-20T22:13:01.8933625Z Entering 'third_party/pybind11' 2024-08-20T22:13:01.8986021Z Entering 'third_party/python-peachpy' 2024-08-20T22:13:01.9039114Z Entering 'third_party/sleef' 2024-08-20T22:13:01.9093628Z Entering 'third_party/tensorpipe' 2024-08-20T22:13:01.9145803Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:13:01.9197232Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:13:01.9247711Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:13:01.9300008Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:13:01.9348463Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:13:01.9422987Z ##[endgroup] 2024-08-20T22:13:01.9425373Z ##[group]Persisting credentials for submodules 2024-08-20T22:13:01.9428263Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-20T22:13:01.9789161Z Entering 'android/libs/fbjni' 2024-08-20T22:13:01.9854915Z Entering 'third_party/FP16' 2024-08-20T22:13:01.9922658Z Entering 'third_party/FXdiv' 2024-08-20T22:13:01.9989978Z Entering 'third_party/NNPACK' 2024-08-20T22:13:02.0057838Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:13:02.0125193Z Entering 'third_party/XNNPACK' 2024-08-20T22:13:02.0212069Z Entering 'third_party/benchmark' 2024-08-20T22:13:02.0284343Z Entering 'third_party/cpp-httplib' 2024-08-20T22:13:02.0351142Z Entering 'third_party/cpuinfo' 2024-08-20T22:13:02.0420617Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:13:02.0487784Z Entering 'third_party/cutlass' 2024-08-20T22:13:02.0562517Z Entering 'third_party/eigen' 2024-08-20T22:13:02.0630901Z Entering 'third_party/fbgemm' 2024-08-20T22:13:02.0697222Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:13:02.0764039Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:13:02.0830251Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:13:02.0903669Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:13:02.0970444Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:13:02.1038825Z Entering 'third_party/flatbuffers' 2024-08-20T22:13:02.1110961Z Entering 'third_party/fmt' 2024-08-20T22:13:02.1177725Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:13:02.1245256Z Entering 'third_party/gloo' 2024-08-20T22:13:02.1313333Z Entering 'third_party/googletest' 2024-08-20T22:13:02.1380292Z Entering 'third_party/ideep' 2024-08-20T22:13:02.1444683Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:13:02.1519993Z Entering 'third_party/ittapi' 2024-08-20T22:13:02.1587522Z Entering 'third_party/kineto' 2024-08-20T22:13:02.1653520Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:13:02.1721155Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:13:02.1794247Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:13:02.1861677Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:13:02.1933114Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:13:02.1998345Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:13:02.2068897Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:13:02.2136009Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:13:02.2203379Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:13:02.2270842Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:13:02.2340525Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:13:02.2408006Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:13:02.2477228Z Entering 'third_party/mimalloc' 2024-08-20T22:13:02.2544310Z Entering 'third_party/nccl/nccl' 2024-08-20T22:13:02.2612277Z Entering 'third_party/nlohmann' 2024-08-20T22:13:02.2680004Z Entering 'third_party/onnx' 2024-08-20T22:13:02.2762611Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:13:02.2829935Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:13:02.2901837Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:13:02.2968719Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:13:02.3035565Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:13:02.3102483Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:13:02.3169333Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:13:02.3237032Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:13:02.3304717Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:13:02.3371475Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:13:02.3436299Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:13:02.3505463Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:13:02.3574931Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:13:02.3664051Z Entering 'third_party/pocketfft' 2024-08-20T22:13:02.3729828Z Entering 'third_party/protobuf' 2024-08-20T22:13:02.3803789Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:13:02.3869960Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:13:02.3939806Z Entering 'third_party/psimd' 2024-08-20T22:13:02.4006766Z Entering 'third_party/pthreadpool' 2024-08-20T22:13:02.4075302Z Entering 'third_party/pybind11' 2024-08-20T22:13:02.4142049Z Entering 'third_party/python-peachpy' 2024-08-20T22:13:02.4209186Z Entering 'third_party/sleef' 2024-08-20T22:13:02.4278611Z Entering 'third_party/tensorpipe' 2024-08-20T22:13:02.4343178Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:13:02.4409828Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:13:02.4477572Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:13:02.4542604Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:13:02.4605939Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:13:02.4698885Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-20T22:13:02.5066788Z Entering 'android/libs/fbjni' 2024-08-20T22:13:02.5129102Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-20T22:13:02.5151119Z Entering 'third_party/FP16' 2024-08-20T22:13:02.5214127Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-20T22:13:02.5235335Z Entering 'third_party/FXdiv' 2024-08-20T22:13:02.5298038Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-20T22:13:02.5320940Z Entering 'third_party/NNPACK' 2024-08-20T22:13:02.5383188Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-20T22:13:02.5405347Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:13:02.5467241Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-20T22:13:02.5489660Z Entering 'third_party/XNNPACK' 2024-08-20T22:13:02.5551846Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-20T22:13:02.5590086Z Entering 'third_party/benchmark' 2024-08-20T22:13:02.5652164Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:13:02.5674294Z Entering 'third_party/cpp-httplib' 2024-08-20T22:13:02.5736202Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-20T22:13:02.5757639Z Entering 'third_party/cpuinfo' 2024-08-20T22:13:02.5818892Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:13:02.5840830Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:13:02.5903008Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-20T22:13:02.5926091Z Entering 'third_party/cutlass' 2024-08-20T22:13:02.5988686Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:13:02.6017950Z Entering 'third_party/eigen' 2024-08-20T22:13:02.6079575Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-20T22:13:02.6102816Z Entering 'third_party/fbgemm' 2024-08-20T22:13:02.6165807Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-20T22:13:02.6185778Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:13:02.6251215Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-20T22:13:02.6271988Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:13:02.6333678Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:13:02.6354747Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:13:02.6415675Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:13:02.6443221Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:13:02.6505469Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:13:02.6525822Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:13:02.6589399Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-20T22:13:02.6615392Z Entering 'third_party/flatbuffers' 2024-08-20T22:13:02.6678523Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-20T22:13:02.6702298Z Entering 'third_party/fmt' 2024-08-20T22:13:02.6763978Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:13:02.6785745Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:13:02.6846975Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-20T22:13:02.6868719Z Entering 'third_party/gloo' 2024-08-20T22:13:02.6929634Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-20T22:13:02.6951186Z Entering 'third_party/googletest' 2024-08-20T22:13:02.7012977Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:13:02.7035044Z Entering 'third_party/ideep' 2024-08-20T22:13:02.7096678Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-20T22:13:02.7115457Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:13:02.7176150Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-20T22:13:02.7205754Z Entering 'third_party/ittapi' 2024-08-20T22:13:02.7267187Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-20T22:13:02.7288600Z Entering 'third_party/kineto' 2024-08-20T22:13:02.7351917Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-20T22:13:02.7372309Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:13:02.7438505Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-20T22:13:02.7457762Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:13:02.7523557Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-20T22:13:02.7544811Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:13:02.7607758Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-20T22:13:02.7629543Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:13:02.7695725Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:13:02.7714522Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:13:02.7777092Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-20T22:13:02.7795522Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:13:02.7859378Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-20T22:13:02.7883446Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:13:02.7948686Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-20T22:13:02.7969893Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:13:02.8031685Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:13:02.8052352Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:13:02.8115473Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-20T22:13:02.8136799Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:13:02.8199754Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-20T22:13:02.8222817Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:13:02.8285449Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-20T22:13:02.8306763Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:13:02.8369335Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-20T22:13:02.8392249Z Entering 'third_party/mimalloc' 2024-08-20T22:13:02.8454892Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-20T22:13:02.8476822Z Entering 'third_party/nccl/nccl' 2024-08-20T22:13:02.8539252Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-20T22:13:02.8563076Z Entering 'third_party/nlohmann' 2024-08-20T22:13:02.8624531Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-20T22:13:02.8647580Z Entering 'third_party/onnx' 2024-08-20T22:13:02.8716904Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-20T22:13:02.8750562Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:13:02.8812429Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:13:02.8832890Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:13:02.8893732Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:13:02.8919748Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:13:02.8988174Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-20T22:13:02.9009716Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:13:02.9071380Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:13:02.9091608Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:13:02.9152296Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:13:02.9172598Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:13:02.9234021Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-20T22:13:02.9253970Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:13:02.9320071Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-20T22:13:02.9341216Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:13:02.9404016Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-20T22:13:02.9424638Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:13:02.9485940Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-20T22:13:02.9505550Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:13:02.9567587Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-20T22:13:02.9585581Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:13:02.9647526Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-20T22:13:02.9669795Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:13:02.9730940Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-20T22:13:02.9755272Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:13:02.9814987Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-20T22:13:02.9856006Z Entering 'third_party/pocketfft' 2024-08-20T22:13:02.9918494Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-20T22:13:02.9939854Z Entering 'third_party/protobuf' 2024-08-20T22:13:03.0006523Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-20T22:13:03.0029622Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:13:03.0090516Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:13:03.0110825Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:13:03.0172134Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:13:03.0196692Z Entering 'third_party/psimd' 2024-08-20T22:13:03.0259265Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-20T22:13:03.0279675Z Entering 'third_party/pthreadpool' 2024-08-20T22:13:03.0341258Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-20T22:13:03.0363035Z Entering 'third_party/pybind11' 2024-08-20T22:13:03.0424264Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:13:03.0446117Z Entering 'third_party/python-peachpy' 2024-08-20T22:13:03.0507167Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-20T22:13:03.0528681Z Entering 'third_party/sleef' 2024-08-20T22:13:03.0590306Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-20T22:13:03.0611810Z Entering 'third_party/tensorpipe' 2024-08-20T22:13:03.0674350Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-20T22:13:03.0694138Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:13:03.0760648Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:13:03.0780629Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:13:03.0842261Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-20T22:13:03.0862425Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:13:03.0928230Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-20T22:13:03.0947906Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:13:03.1009312Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:13:03.1027396Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:13:03.1092862Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-20T22:13:03.1782860Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-20T22:13:03.2150112Z Entering 'android/libs/fbjni' 2024-08-20T22:13:03.2203267Z Entering 'third_party/FP16' 2024-08-20T22:13:03.2256054Z Entering 'third_party/FXdiv' 2024-08-20T22:13:03.2308656Z Entering 'third_party/NNPACK' 2024-08-20T22:13:03.2362264Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:13:03.2415802Z Entering 'third_party/XNNPACK' 2024-08-20T22:13:03.2486112Z Entering 'third_party/benchmark' 2024-08-20T22:13:03.2540306Z Entering 'third_party/cpp-httplib' 2024-08-20T22:13:03.2593363Z Entering 'third_party/cpuinfo' 2024-08-20T22:13:03.2647134Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:13:03.2697999Z Entering 'third_party/cutlass' 2024-08-20T22:13:03.2757715Z Entering 'third_party/eigen' 2024-08-20T22:13:03.2810125Z Entering 'third_party/fbgemm' 2024-08-20T22:13:03.2862845Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:13:03.2914371Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:13:03.2966539Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:13:03.3023189Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:13:03.3077198Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:13:03.3130586Z Entering 'third_party/flatbuffers' 2024-08-20T22:13:03.3186474Z Entering 'third_party/fmt' 2024-08-20T22:13:03.3237503Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:13:03.3289883Z Entering 'third_party/gloo' 2024-08-20T22:13:03.3341501Z Entering 'third_party/googletest' 2024-08-20T22:13:03.3392709Z Entering 'third_party/ideep' 2024-08-20T22:13:03.3441661Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:13:03.3503505Z Entering 'third_party/ittapi' 2024-08-20T22:13:03.3557465Z Entering 'third_party/kineto' 2024-08-20T22:13:03.3607378Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:13:03.3657401Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:13:03.3710068Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:13:03.3761182Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:13:03.3810860Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:13:03.3859685Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:13:03.3918565Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:13:03.3970722Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:13:03.4021477Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:13:03.4074351Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:13:03.4126830Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:13:03.4180227Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:13:03.4234515Z Entering 'third_party/mimalloc' 2024-08-20T22:13:03.4286373Z Entering 'third_party/nccl/nccl' 2024-08-20T22:13:03.4339108Z Entering 'third_party/nlohmann' 2024-08-20T22:13:03.4394458Z Entering 'third_party/onnx' 2024-08-20T22:13:03.4459812Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:13:03.4515953Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:13:03.4572431Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:13:03.4624733Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:13:03.4676287Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:13:03.4726598Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:13:03.4776474Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:13:03.4829482Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:13:03.4883841Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:13:03.4932782Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:13:03.4984269Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:13:03.5036405Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:13:03.5090217Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:13:03.5161257Z Entering 'third_party/pocketfft' 2024-08-20T22:13:03.5212603Z Entering 'third_party/protobuf' 2024-08-20T22:13:03.5265645Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:13:03.5315978Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:13:03.5370396Z Entering 'third_party/psimd' 2024-08-20T22:13:03.5422944Z Entering 'third_party/pthreadpool' 2024-08-20T22:13:03.5477962Z Entering 'third_party/pybind11' 2024-08-20T22:13:03.5530306Z Entering 'third_party/python-peachpy' 2024-08-20T22:13:03.5582940Z Entering 'third_party/sleef' 2024-08-20T22:13:03.5635369Z Entering 'third_party/tensorpipe' 2024-08-20T22:13:03.5687068Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:13:03.5737103Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:13:03.5786521Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:13:03.5835656Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:13:03.5884787Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:13:03.5960880Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-20T22:13:03.6328267Z Entering 'android/libs/fbjni' 2024-08-20T22:13:03.6380397Z Entering 'third_party/FP16' 2024-08-20T22:13:03.6432176Z Entering 'third_party/FXdiv' 2024-08-20T22:13:03.6485667Z Entering 'third_party/NNPACK' 2024-08-20T22:13:03.6538735Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:13:03.6593747Z Entering 'third_party/XNNPACK' 2024-08-20T22:13:03.6659888Z Entering 'third_party/benchmark' 2024-08-20T22:13:03.6710443Z Entering 'third_party/cpp-httplib' 2024-08-20T22:13:03.6761391Z Entering 'third_party/cpuinfo' 2024-08-20T22:13:03.6812922Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:13:03.6864423Z Entering 'third_party/cutlass' 2024-08-20T22:13:03.6923252Z Entering 'third_party/eigen' 2024-08-20T22:13:03.6977031Z Entering 'third_party/fbgemm' 2024-08-20T22:13:03.7028219Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:13:03.7078210Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:13:03.7129386Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:13:03.7187074Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:13:03.7237201Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:13:03.7290736Z Entering 'third_party/flatbuffers' 2024-08-20T22:13:03.7346257Z Entering 'third_party/fmt' 2024-08-20T22:13:03.7399693Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:13:03.7452408Z Entering 'third_party/gloo' 2024-08-20T22:13:03.7504316Z Entering 'third_party/googletest' 2024-08-20T22:13:03.7557656Z Entering 'third_party/ideep' 2024-08-20T22:13:03.7606844Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:13:03.7665712Z Entering 'third_party/ittapi' 2024-08-20T22:13:03.7716866Z Entering 'third_party/kineto' 2024-08-20T22:13:03.7768113Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:13:03.7817746Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:13:03.7870968Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:13:03.7921451Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:13:03.7972454Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:13:03.8021506Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:13:03.8080775Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:13:03.8131300Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:13:03.8181609Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:13:03.8233427Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:13:03.8287984Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:13:03.8338396Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:13:03.8392068Z Entering 'third_party/mimalloc' 2024-08-20T22:13:03.8446105Z Entering 'third_party/nccl/nccl' 2024-08-20T22:13:03.8497706Z Entering 'third_party/nlohmann' 2024-08-20T22:13:03.8551086Z Entering 'third_party/onnx' 2024-08-20T22:13:03.8620196Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:13:03.8676107Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:13:03.8731684Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:13:03.8789384Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:13:03.8843970Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:13:03.8894927Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:13:03.8944580Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:13:03.8999613Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:13:03.9049117Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:13:03.9100663Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:13:03.9148838Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:13:03.9202352Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:13:03.9256358Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:13:03.9330266Z Entering 'third_party/pocketfft' 2024-08-20T22:13:03.9383509Z Entering 'third_party/protobuf' 2024-08-20T22:13:03.9436468Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:13:03.9487254Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:13:03.9541173Z Entering 'third_party/psimd' 2024-08-20T22:13:03.9593420Z Entering 'third_party/pthreadpool' 2024-08-20T22:13:03.9644143Z Entering 'third_party/pybind11' 2024-08-20T22:13:03.9697906Z Entering 'third_party/python-peachpy' 2024-08-20T22:13:03.9748185Z Entering 'third_party/sleef' 2024-08-20T22:13:03.9800054Z Entering 'third_party/tensorpipe' 2024-08-20T22:13:03.9850301Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:13:03.9902453Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:13:03.9952577Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:13:04.0003237Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:13:04.0055828Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:13:04.0129219Z ##[endgroup] 2024-08-20T22:13:04.0179487Z [command]/usr/bin/git log -1 --format='%H' 2024-08-20T22:13:04.0215581Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-20T22:13:04.0441027Z Prepare all required actions 2024-08-20T22:13:04.0441490Z Getting action download info 2024-08-20T22:13:04.2181215Z ##[group]Run ./.github/actions/setup-linux 2024-08-20T22:13:04.2181678Z env: 2024-08-20T22:13:04.2181989Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:04.2182332Z ##[endgroup] 2024-08-20T22:13:04.2261173Z ##[group]Run set -euo pipefail 2024-08-20T22:13:04.2261627Z set -euo pipefail 2024-08-20T22:13:04.2262031Z function get_ec2_metadata() { 2024-08-20T22:13:04.2262579Z  # Pulled from instance metadata endpoint for EC2 2024-08-20T22:13:04.2263467Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-20T22:13:04.2264233Z  category=$1 2024-08-20T22:13:04.2264794Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-20T22:13:04.2265446Z  runner_name_str=i-021b0327168a01512 2024-08-20T22:13:04.2265951Z  if [[ -f /.inarc ]]; then 2024-08-20T22:13:04.2266462Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-20T22:13:04.2267052Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-20T22:13:04.2267747Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-20T22:13:04.2268360Z  else 2024-08-20T22:13:04.2268873Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-20T22:13:04.2269469Z  fi 2024-08-20T22:13:04.2269779Z } 2024-08-20T22:13:04.2270152Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-20T22:13:04.2270762Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-20T22:13:04.2271458Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-20T22:13:04.2272077Z echo "system info $(uname -a)" 2024-08-20T22:13:04.2283905Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:04.2284414Z env: 2024-08-20T22:13:04.2284710Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:04.2285070Z ##[endgroup] 2024-08-20T22:13:04.2387909Z ami-id: ami-06c68f701d8090592 2024-08-20T22:13:04.2448897Z instance-id: i-021b0327168a01512 2024-08-20T22:13:04.2503008Z instance-type: g5.4xlarge 2024-08-20T22:13:04.2517108Z system info Linux ip-10-0-25-243.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-20T22:13:04.2541430Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:13:04.2542404Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:13:04.2551918Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:04.2552596Z env: 2024-08-20T22:13:04.2553160Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:04.2553529Z ##[endgroup] 2024-08-20T22:13:04.2629474Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-20T22:13:04.2630107Z if systemctl is-active --quiet docker; then 2024-08-20T22:13:04.2630668Z  echo "Docker daemon is running..."; 2024-08-20T22:13:04.2631135Z else 2024-08-20T22:13:04.2631636Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-20T22:13:04.2632242Z fi 2024-08-20T22:13:04.2640762Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:04.2641264Z env: 2024-08-20T22:13:04.2641551Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:04.2641909Z ##[endgroup] 2024-08-20T22:13:04.2732586Z Docker daemon is running... 2024-08-20T22:13:04.2784699Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:13:04.2785269Z with: 2024-08-20T22:13:04.2785572Z shell: bash 2024-08-20T22:13:04.2785874Z timeout_minutes: 5 2024-08-20T22:13:04.2786210Z max_attempts: 3 2024-08-20T22:13:04.2786541Z retry_wait_seconds: 30 2024-08-20T22:13:04.2789630Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-20T22:13:04.2792583Z polling_interval_seconds: 1 2024-08-20T22:13:04.2792964Z warning_on_retry: true 2024-08-20T22:13:04.2793327Z continue_on_error: false 2024-08-20T22:13:04.2793684Z env: 2024-08-20T22:13:04.2793979Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:04.2794343Z AWS_RETRY_MODE: standard 2024-08-20T22:13:04.2794705Z AWS_MAX_ATTEMPTS: 5 2024-08-20T22:13:04.2795064Z AWS_DEFAULT_REGION: us-east-1 2024-08-20T22:13:04.2795457Z ##[endgroup] 2024-08-20T22:13:05.4336947Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:13:05.4337779Z Configure a credential helper to remove this warning. See 2024-08-20T22:13:05.4338637Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:13:05.4339158Z 2024-08-20T22:13:05.4339286Z Login Succeeded 2024-08-20T22:13:06.3360375Z Command completed after 1 attempt(s). 2024-08-20T22:13:06.3425025Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:13:06.3425757Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:13:06.3426421Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:13:06.3435955Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:06.3436457Z env: 2024-08-20T22:13:06.3436754Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:06.3437114Z ##[endgroup] 2024-08-20T22:13:06.3531748Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:13:06.3532692Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:13:06.3533335Z # shellcheck disable=SC2046 2024-08-20T22:13:06.3533808Z docker stop $(docker ps -q) || true 2024-08-20T22:13:06.3534283Z # Prune all of the docker images 2024-08-20T22:13:06.3534854Z docker system prune -af 2024-08-20T22:13:06.3544090Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:06.3544587Z env: 2024-08-20T22:13:06.3544859Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:06.3545206Z ##[endgroup] 2024-08-20T22:13:06.3860764Z "docker stop" requires at least 1 argument. 2024-08-20T22:13:06.3861490Z See 'docker stop --help'. 2024-08-20T22:13:06.3862133Z 2024-08-20T22:13:06.3862361Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-20T22:13:06.3863522Z 2024-08-20T22:13:06.3863683Z Stop one or more running containers 2024-08-20T22:13:06.4303426Z Total reclaimed space: 0B 2024-08-20T22:13:06.4359015Z ##[group]Run set +e 2024-08-20T22:13:06.4359408Z set +e 2024-08-20T22:13:06.4359723Z set -x 2024-08-20T22:13:06.4360032Z  2024-08-20T22:13:06.4360370Z PT_DOMAIN=download.pytorch.org 2024-08-20T22:13:06.4361170Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-20T22:13:06.4362256Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-20T22:13:06.4363013Z # one is returned at random 2024-08-20T22:13:06.4363566Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-20T22:13:06.4364083Z  2024-08-20T22:13:06.4364423Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:13:06.4365049Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-20T22:13:06.4366016Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-20T22:13:06.4366585Z  2024-08-20T22:13:06.4366930Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:13:06.4367496Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-20T22:13:06.4368019Z  exit 1 2024-08-20T22:13:06.4368342Z  fi 2024-08-20T22:13:06.4368640Z fi 2024-08-20T22:13:06.4368929Z  2024-08-20T22:13:06.4369301Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-20T22:13:06.4369819Z  # Clean up any old records first 2024-08-20T22:13:06.4370340Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-20T22:13:06.4370805Z fi 2024-08-20T22:13:06.4371089Z  2024-08-20T22:13:06.4371532Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-20T22:13:06.4372094Z cat /etc/hosts 2024-08-20T22:13:06.4384023Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:06.4384761Z env: 2024-08-20T22:13:06.4385187Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:06.4385690Z ##[endgroup] 2024-08-20T22:13:06.4416752Z + PT_DOMAIN=download.pytorch.org 2024-08-20T22:13:06.4423169Z ++ dig -4 +short download.pytorch.org 2024-08-20T22:13:06.4423649Z ++ tail -n1 2024-08-20T22:13:06.4825884Z + RESOLVED_IP=18.160.10.22 2024-08-20T22:13:06.4826347Z + '[' -z 18.160.10.22 ']' 2024-08-20T22:13:06.4826800Z + grep -r download.pytorch.org /etc/hosts 2024-08-20T22:13:06.4845047Z + echo '18.160.10.22 download.pytorch.org' 2024-08-20T22:13:06.4846076Z + sudo tee -a /etc/hosts 2024-08-20T22:13:06.7435183Z 18.160.10.22 download.pytorch.org 2024-08-20T22:13:06.7457920Z + cat /etc/hosts 2024-08-20T22:13:06.7468685Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-20T22:13:06.7480702Z ::1 localhost6 localhost6.localdomain6 2024-08-20T22:13:06.7481450Z 18.160.10.22 download.pytorch.org 2024-08-20T22:13:06.7635678Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-20T22:13:06.7636285Z with: 2024-08-20T22:13:06.7637379Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:06.7638602Z docker-build-dir: .ci/docker 2024-08-20T22:13:06.7638984Z working-directory: . 2024-08-20T22:13:06.7639450Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:06.7639976Z force-push: false 2024-08-20T22:13:06.7640278Z env: 2024-08-20T22:13:06.7640554Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:06.7640890Z ##[endgroup] 2024-08-20T22:13:06.7661203Z ##[group]Run set -ex 2024-08-20T22:13:06.7661581Z set -ex 2024-08-20T22:13:06.7661889Z  2024-08-20T22:13:06.7662624Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-20T22:13:06.7663640Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-20T22:13:06.7664447Z # job could then download the pre-built image as usual 2024-08-20T22:13:06.7665199Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-20T22:13:06.7665882Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:06.7666531Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:06.7667102Z  2024-08-20T22:13:06.7667621Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-20T22:13:06.7668253Z  exit 0 2024-08-20T22:13:06.7668569Z else 2024-08-20T22:13:06.7668945Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:06.7669409Z fi 2024-08-20T22:13:06.7669709Z  2024-08-20T22:13:06.7670195Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-20T22:13:06.7671040Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-20T22:13:06.7671825Z  # use it as it is, but first let's extract the tag 2024-08-20T22:13:06.7672539Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-20T22:13:06.7673298Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:06.7674066Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:06.7674682Z else 2024-08-20T22:13:06.7675156Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-20T22:13:06.7675887Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:06.7676884Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:06.7677726Z fi 2024-08-20T22:13:06.7688353Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:06.7688860Z env: 2024-08-20T22:13:06.7689151Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:06.7689519Z REPO_NAME: pytorch 2024-08-20T22:13:06.7690660Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:06.7691864Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:13:06.7692372Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:06.7692934Z ##[endgroup] 2024-08-20T22:13:06.7722693Z + [[ ! -d .ci/docker ]] 2024-08-20T22:13:06.7723419Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-20T22:13:06.7724099Z + echo skip=false 2024-08-20T22:13:06.7726261Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-20T22:13:06.7729565Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:06.7731306Z ++ awk -F '[:,]' '{print $2}' 2024-08-20T22:13:06.7758512Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:06.7759610Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:06.7761489Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:06.7794884Z ##[group]Run set +e 2024-08-20T22:13:06.7795360Z set +e 2024-08-20T22:13:06.7795805Z set -x 2024-08-20T22:13:06.7796301Z  2024-08-20T22:13:06.7796797Z login() { 2024-08-20T22:13:06.7797700Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:13:06.7798580Z } 2024-08-20T22:13:06.7798994Z  2024-08-20T22:13:06.7799437Z retry () { 2024-08-20T22:13:06.7799908Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:13:06.7800489Z } 2024-08-20T22:13:06.7800924Z  2024-08-20T22:13:06.7801377Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:13:06.7801871Z  2024-08-20T22:13:06.7802506Z # Check if image already exists, if it does then skip building it 2024-08-20T22:13:06.7803469Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-20T22:13:06.7804078Z  exit 0 2024-08-20T22:13:06.7804502Z fi 2024-08-20T22:13:06.7804918Z  2024-08-20T22:13:06.7805533Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-20T22:13:06.7806507Z # be empty. The default action would be to continue rebuild the image 2024-08-20T22:13:06.7807403Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-20T22:13:06.7808195Z  # if we're on the base branch then use the parent commit 2024-08-20T22:13:06.7808980Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-20T22:13:06.7809526Z else 2024-08-20T22:13:06.7810067Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-20T22:13:06.7811017Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-20T22:13:06.7811641Z fi 2024-08-20T22:13:06.7812059Z  2024-08-20T22:13:06.7812512Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-20T22:13:06.7813135Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:06.7813738Z  2024-08-20T22:13:06.7814832Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-20T22:13:06.7815679Z  exit 0 2024-08-20T22:13:06.7816125Z fi 2024-08-20T22:13:06.7816587Z  2024-08-20T22:13:06.7817106Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-20T22:13:06.7818215Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-20T22:13:06.7819262Z  exit 1 2024-08-20T22:13:06.7819705Z fi 2024-08-20T22:13:06.7820056Z  2024-08-20T22:13:06.7820703Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-20T22:13:06.7821789Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-20T22:13:06.7822818Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-20T22:13:06.7823900Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-20T22:13:06.7825099Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-20T22:13:06.7826073Z fi 2024-08-20T22:13:06.7826505Z  2024-08-20T22:13:06.7826973Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:06.7835492Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:06.7836096Z env: 2024-08-20T22:13:06.7836557Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:06.7837063Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:13:06.7837601Z BASE_REVISION: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:13:06.7839052Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:06.7840406Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:06.7841110Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:06.7841850Z ##[endgroup] 2024-08-20T22:13:06.7872121Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:06.7873102Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:06.7874362Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:13:06.7876856Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:07.3238482Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:13:07.3239450Z Configure a credential helper to remove this warning. See 2024-08-20T22:13:07.3240510Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:13:07.3241057Z 2024-08-20T22:13:07.3241517Z Login Succeeded 2024-08-20T22:13:07.3264972Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:07.5325754Z { 2024-08-20T22:13:07.5326440Z "schemaVersion": 2, 2024-08-20T22:13:07.5327497Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-20T22:13:07.5328462Z "config": { 2024-08-20T22:13:07.5328990Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-20T22:13:07.5329733Z "size": 48711, 2024-08-20T22:13:07.5330451Z "digest": "sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb" 2024-08-20T22:13:07.5331870Z }, 2024-08-20T22:13:07.5332560Z "layers": [ 2024-08-20T22:13:07.5340605Z { 2024-08-20T22:13:07.5341253Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5342278Z "size": 28580681, 2024-08-20T22:13:07.5343217Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-08-20T22:13:07.5344173Z }, 2024-08-20T22:13:07.5345358Z { 2024-08-20T22:13:07.5345926Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5346554Z "size": 7943451, 2024-08-20T22:13:07.5348009Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-08-20T22:13:07.5349659Z }, 2024-08-20T22:13:07.5350485Z { 2024-08-20T22:13:07.5351210Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5352177Z "size": 55728572, 2024-08-20T22:13:07.5353275Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-08-20T22:13:07.5354175Z }, 2024-08-20T22:13:07.5354498Z { 2024-08-20T22:13:07.5355145Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5355875Z "size": 186, 2024-08-20T22:13:07.5356490Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-08-20T22:13:07.5357279Z }, 2024-08-20T22:13:07.5357878Z { 2024-08-20T22:13:07.5358429Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5359129Z "size": 6886, 2024-08-20T22:13:07.5359780Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-08-20T22:13:07.5360515Z }, 2024-08-20T22:13:07.5361238Z { 2024-08-20T22:13:07.5361739Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5362449Z "size": 1329001756, 2024-08-20T22:13:07.5363201Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-08-20T22:13:07.5363952Z }, 2024-08-20T22:13:07.5364274Z { 2024-08-20T22:13:07.5364925Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5365589Z "size": 62501, 2024-08-20T22:13:07.5366196Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-08-20T22:13:07.5366946Z }, 2024-08-20T22:13:07.5367321Z { 2024-08-20T22:13:07.5367900Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5368501Z "size": 1684, 2024-08-20T22:13:07.5369143Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-08-20T22:13:07.5370049Z }, 2024-08-20T22:13:07.5370421Z { 2024-08-20T22:13:07.5370919Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5371624Z "size": 1523, 2024-08-20T22:13:07.5372382Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-08-20T22:13:07.5373110Z }, 2024-08-20T22:13:07.5373477Z { 2024-08-20T22:13:07.5374034Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5374780Z "size": 2528295403, 2024-08-20T22:13:07.5375471Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-08-20T22:13:07.5376212Z }, 2024-08-20T22:13:07.5376559Z { 2024-08-20T22:13:07.5377163Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5377772Z "size": 86016, 2024-08-20T22:13:07.5378421Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-08-20T22:13:07.5379291Z }, 2024-08-20T22:13:07.5379638Z { 2024-08-20T22:13:07.5380138Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5380879Z "size": 1822, 2024-08-20T22:13:07.5381890Z "digest": "sha256:ac23c5fcc1ef310e9cd0d138b180281188e20a3e28c51cd14751622e3c7d92ea" 2024-08-20T22:13:07.5382623Z }, 2024-08-20T22:13:07.5383001Z { 2024-08-20T22:13:07.5383535Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5384211Z "size": 246761563, 2024-08-20T22:13:07.5384893Z "digest": "sha256:ca426ffc9d2bd051ad1ff5c4f02fd5d9be161e70953cc37882c815080c708f4d" 2024-08-20T22:13:07.5385613Z }, 2024-08-20T22:13:07.5385985Z { 2024-08-20T22:13:07.5386627Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5387241Z "size": 544, 2024-08-20T22:13:07.5387886Z "digest": "sha256:4373aa743caf6660c671295b837048c8350a2165f2336dc3b7b6c96462579e50" 2024-08-20T22:13:07.5388634Z }, 2024-08-20T22:13:07.5389005Z { 2024-08-20T22:13:07.5389502Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5390205Z "size": 1258, 2024-08-20T22:13:07.5390856Z "digest": "sha256:75b2abb16c685854c017d0cf75c626526c956734c656d9e78fdd4520317e9b3c" 2024-08-20T22:13:07.5391514Z }, 2024-08-20T22:13:07.5391907Z { 2024-08-20T22:13:07.5392503Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5393276Z "size": 484, 2024-08-20T22:13:07.5393860Z "digest": "sha256:ef1e3ef1e85f0124270b8506721219150e230184181f8207d64463bf5c67e656" 2024-08-20T22:13:07.5394580Z }, 2024-08-20T22:13:07.5394970Z { 2024-08-20T22:13:07.5395525Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5396137Z "size": 91711766, 2024-08-20T22:13:07.5396835Z "digest": "sha256:0bf61114d37b2d09065aa6852b1e78a32580a1ff011dd5d23f03b0f5c4e710af" 2024-08-20T22:13:07.5397558Z }, 2024-08-20T22:13:07.5397868Z { 2024-08-20T22:13:07.5398449Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5399116Z "size": 3390, 2024-08-20T22:13:07.5399932Z "digest": "sha256:2df849f72ba4258df10c9c45ace21a66d93b73196e78e96f12c13098822f60e2" 2024-08-20T22:13:07.5400692Z }, 2024-08-20T22:13:07.5401043Z { 2024-08-20T22:13:07.5401571Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5402291Z "size": 1909, 2024-08-20T22:13:07.5402889Z "digest": "sha256:34c47d63c5de27418739aa048b4dd781c3006d65691e9b903c95304d0509d89c" 2024-08-20T22:13:07.5403581Z }, 2024-08-20T22:13:07.5404002Z { 2024-08-20T22:13:07.5404530Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5405137Z "size": 701, 2024-08-20T22:13:07.5405842Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:13:07.5406552Z }, 2024-08-20T22:13:07.5406960Z { 2024-08-20T22:13:07.5407531Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5408271Z "size": 2864254879, 2024-08-20T22:13:07.5408959Z "digest": "sha256:6efb96fc87ea1bca54876d85331e5585e28f325a1d60ee1aef7811eabb74f933" 2024-08-20T22:13:07.5409686Z }, 2024-08-20T22:13:07.5410061Z { 2024-08-20T22:13:07.5410601Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5411295Z "size": 380, 2024-08-20T22:13:07.5411911Z "digest": "sha256:b119c4fc6c58d9e72744201ff4850b34184a94eb3566031b7d36e87ecce950a6" 2024-08-20T22:13:07.5412623Z }, 2024-08-20T22:13:07.5413022Z { 2024-08-20T22:13:07.5413608Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5414257Z "size": 12900, 2024-08-20T22:13:07.5415035Z "digest": "sha256:33b639226eeefa41d7bb26a8ecc20d271ceef45571dabfd80aaee3634926381b" 2024-08-20T22:13:07.5415782Z }, 2024-08-20T22:13:07.5416195Z { 2024-08-20T22:13:07.5416756Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5417378Z "size": 803, 2024-08-20T22:13:07.5418063Z "digest": "sha256:d3547bd43146930aa830d54562c98aa678e355cf5f84527e9717eb72fffe8ca3" 2024-08-20T22:13:07.5418777Z }, 2024-08-20T22:13:07.5419119Z { 2024-08-20T22:13:07.5419665Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5420407Z "size": 106, 2024-08-20T22:13:07.5421034Z "digest": "sha256:5645520256fa6d7f2270608eaed53849a131d03fbf4c5d4472c9cfde1a439e47" 2024-08-20T22:13:07.5421797Z }, 2024-08-20T22:13:07.5422108Z { 2024-08-20T22:13:07.5422640Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5423364Z "size": 503, 2024-08-20T22:13:07.5423962Z "digest": "sha256:829ed7d91e3818d0e11834c71551117c7aeec0a51d798e93d641faa0c02582cb" 2024-08-20T22:13:07.5424668Z }, 2024-08-20T22:13:07.5425093Z { 2024-08-20T22:13:07.5425633Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5426254Z "size": 121477267, 2024-08-20T22:13:07.5426993Z "digest": "sha256:385d0421d67c533d628985c453a186028a61f6ad4209ae519ec3d4ecea661d3e" 2024-08-20T22:13:07.5427769Z }, 2024-08-20T22:13:07.5428137Z { 2024-08-20T22:13:07.5428681Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5429321Z "size": 109, 2024-08-20T22:13:07.5429987Z "digest": "sha256:7cbfce999d76851e07dd71056a6d852629d789ccfd46be44cc405b451fb8bb58" 2024-08-20T22:13:07.5430744Z }, 2024-08-20T22:13:07.5431060Z { 2024-08-20T22:13:07.5431619Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5432317Z "size": 490, 2024-08-20T22:13:07.5432914Z "digest": "sha256:6658e558ebe496f0d628947534b0ea1e44808f84f2dfd8a9a704e8aa5329d791" 2024-08-20T22:13:07.5433645Z }, 2024-08-20T22:13:07.5434108Z { 2024-08-20T22:13:07.5434660Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5435266Z "size": 297, 2024-08-20T22:13:07.5435939Z "digest": "sha256:33444a5a7e01a3010fd1cc16b32a66892a08600d7f700747d23faf3c36624f3a" 2024-08-20T22:13:07.5436675Z }, 2024-08-20T22:13:07.5437193Z { 2024-08-20T22:13:07.5437690Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5438364Z "size": 103, 2024-08-20T22:13:07.5439049Z "digest": "sha256:7c47122f4e09b47e891b4ae9e499a35a4eb46faacae87d0e946615adda6e0706" 2024-08-20T22:13:07.5439754Z }, 2024-08-20T22:13:07.5440110Z { 2024-08-20T22:13:07.5440687Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5441438Z "size": 1473, 2024-08-20T22:13:07.5442039Z "digest": "sha256:1811ff4d3556ac255604f346d879528aff2aadd7652c2d39baf4fe6eaa60aec3" 2024-08-20T22:13:07.5442847Z }, 2024-08-20T22:13:07.5443221Z { 2024-08-20T22:13:07.5443758Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5444416Z "size": 594729516, 2024-08-20T22:13:07.5445087Z "digest": "sha256:7754ee068b33f760d9c320c26a1a3935247ace0c8c463a976105c1586a43fb1c" 2024-08-20T22:13:07.5445898Z }, 2024-08-20T22:13:07.5446288Z { 2024-08-20T22:13:07.5446830Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5447479Z "size": 164, 2024-08-20T22:13:07.5448268Z "digest": "sha256:0479fbac3361713c075bf788e778117917dcacdd1ff71ed79a789dd2016a1714" 2024-08-20T22:13:07.5448945Z }, 2024-08-20T22:13:07.5449318Z { 2024-08-20T22:13:07.5449906Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5450558Z "size": 564, 2024-08-20T22:13:07.5451173Z "digest": "sha256:f1d6874dee9c34d154278d2b6935d04447f07fa89d25d4a09398234040d8a7e8" 2024-08-20T22:13:07.5451919Z }, 2024-08-20T22:13:07.5452260Z { 2024-08-20T22:13:07.5453140Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5453894Z "size": 35874203, 2024-08-20T22:13:07.5454700Z "digest": "sha256:afc429e87b593a64f3e0a1be51e09e966ea1aa71e6e8a0bec4edc9f2f7bc1c05" 2024-08-20T22:13:07.5455733Z }, 2024-08-20T22:13:07.5456184Z { 2024-08-20T22:13:07.5456880Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5457650Z "size": 104, 2024-08-20T22:13:07.5458345Z "digest": "sha256:019620471c2592cd22437fe8ed1a94c784114cf7445c5c2ab36dd0bae727fc66" 2024-08-20T22:13:07.5459036Z }, 2024-08-20T22:13:07.5459390Z { 2024-08-20T22:13:07.5459991Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5460657Z "size": 425, 2024-08-20T22:13:07.5461275Z "digest": "sha256:27aeb1b2cd5e28d8f0e54fc1090baf88abc29ebf5fce088b493fcae99c9550d1" 2024-08-20T22:13:07.5462067Z }, 2024-08-20T22:13:07.5462488Z { 2024-08-20T22:13:07.5462991Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5463713Z "size": 20262183, 2024-08-20T22:13:07.5464368Z "digest": "sha256:829c0e19f4b7797e9120a86172fabfd710f9b3971ffd714a616931a3fadc7192" 2024-08-20T22:13:07.5465119Z }, 2024-08-20T22:13:07.5465459Z { 2024-08-20T22:13:07.5465988Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5466708Z "size": 438, 2024-08-20T22:13:07.5467342Z "digest": "sha256:560cd6122a3868e7e82e75eb103d182edf6512114d61ccdfae099af00cb16b4c" 2024-08-20T22:13:07.5468016Z }, 2024-08-20T22:13:07.5468447Z { 2024-08-20T22:13:07.5468981Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5469694Z "size": 701, 2024-08-20T22:13:07.5470374Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:13:07.5471090Z }, 2024-08-20T22:13:07.5471442Z { 2024-08-20T22:13:07.5472003Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5472649Z "size": 141, 2024-08-20T22:13:07.5473311Z "digest": "sha256:5a4eb19d72198a01f3babd4711c40fcf7c6e5cb9d5cfedc507b0061408d47097" 2024-08-20T22:13:07.5474077Z }, 2024-08-20T22:13:07.5474398Z { 2024-08-20T22:13:07.5474966Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5475673Z "size": 135, 2024-08-20T22:13:07.5476570Z "digest": "sha256:da6a838a66c72b5de9a5590c233779880ea901c1264940a1c618825d2501ffd1" 2024-08-20T22:13:07.5477249Z }, 2024-08-20T22:13:07.5477657Z { 2024-08-20T22:13:07.5478213Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5478830Z "size": 4838068375, 2024-08-20T22:13:07.5479550Z "digest": "sha256:cbb79a7053537bed2d80b77198ea552b985563b11d3f8616593c6ab575a3a090" 2024-08-20T22:13:07.5480288Z }, 2024-08-20T22:13:07.5480645Z { 2024-08-20T22:13:07.5481189Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5481864Z "size": 194, 2024-08-20T22:13:07.5482498Z "digest": "sha256:4d4fa6802ab3e6b36d8d4adb650046e9370c4a16a19876970559ef8a312ca855" 2024-08-20T22:13:07.5483287Z }, 2024-08-20T22:13:07.5483812Z + exit 0 2024-08-20T22:13:07.5484233Z { 2024-08-20T22:13:07.5484888Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5485543Z "size": 564, 2024-08-20T22:13:07.5486262Z "digest": "sha256:0f085d4fa279366fcd52d303b7ad8abddc5962a8872174b8109f3f5d5ca5f2cf" 2024-08-20T22:13:07.5486975Z }, 2024-08-20T22:13:07.5487285Z { 2024-08-20T22:13:07.5487887Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5488542Z "size": 43162921, 2024-08-20T22:13:07.5489155Z "digest": "sha256:4a0412373305517489d891a4749e6d2b80b4aeb7df3b8adca0ed0abb2f3e5300" 2024-08-20T22:13:07.5489949Z }, 2024-08-20T22:13:07.5490378Z { 2024-08-20T22:13:07.5490914Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5491587Z "size": 106, 2024-08-20T22:13:07.5492215Z "digest": "sha256:302c8c1779444e54919f83e91c6d5e1fdea653fc899be802b3a2a34d34d99d5d" 2024-08-20T22:13:07.5492991Z }, 2024-08-20T22:13:07.5493387Z { 2024-08-20T22:13:07.5493876Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5494677Z "size": 1380, 2024-08-20T22:13:07.5495560Z "digest": "sha256:02970347dff622ee45629bc28788716d00528464f71f88ec7b74ae5f44a486e2" 2024-08-20T22:13:07.5496487Z }, 2024-08-20T22:13:07.5496909Z { 2024-08-20T22:13:07.5497730Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5498447Z "size": 701, 2024-08-20T22:13:07.5499069Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:13:07.5499837Z }, 2024-08-20T22:13:07.5500213Z { 2024-08-20T22:13:07.5500777Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5501483Z "size": 139, 2024-08-20T22:13:07.5502133Z "digest": "sha256:916c5e8580164926a3aec87e88a5d1c2d02756e49fdeed4198e1e6304efa2fd3" 2024-08-20T22:13:07.5502832Z }, 2024-08-20T22:13:07.5503229Z { 2024-08-20T22:13:07.5503740Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5504387Z "size": 118, 2024-08-20T22:13:07.5505134Z "digest": "sha256:c42b3602a1fb4df5111224650f15adffb025979ae9c614807379ea84e189533b" 2024-08-20T22:13:07.5505816Z }, 2024-08-20T22:13:07.5506165Z { 2024-08-20T22:13:07.5506773Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5507431Z "size": 4324672728, 2024-08-20T22:13:07.5508069Z "digest": "sha256:11fad972ef6cb9d2dc236504b0ee5b6e5ea996e4576ece0ed3262fbb9ee2de60" 2024-08-20T22:13:07.5508864Z }, 2024-08-20T22:13:07.5509216Z { 2024-08-20T22:13:07.5509804Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5510435Z "size": 174, 2024-08-20T22:13:07.5511054Z "digest": "sha256:f957c16244819132b478272aaf24231699c4fcbd495c229a574f94daab8d6b86" 2024-08-20T22:13:07.5511864Z }, 2024-08-20T22:13:07.5512202Z { 2024-08-20T22:13:07.5512740Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5513463Z "size": 907, 2024-08-20T22:13:07.5514117Z "digest": "sha256:768c31dca016b80bbdc973c8ee818034bdbd17b7ea9bdadbf0ec45ea51005720" 2024-08-20T22:13:07.5514902Z }, 2024-08-20T22:13:07.5515336Z { 2024-08-20T22:13:07.5515865Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5516530Z "size": 701, 2024-08-20T22:13:07.5517185Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:13:07.5517893Z }, 2024-08-20T22:13:07.5518329Z { 2024-08-20T22:13:07.5518914Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5519518Z "size": 134, 2024-08-20T22:13:07.5520172Z "digest": "sha256:8c32d0ecebbe2054c96a3a5c1f3d6c8b87c2e826686b369e5242cd55d0f01c2d" 2024-08-20T22:13:07.5520928Z }, 2024-08-20T22:13:07.5521238Z { 2024-08-20T22:13:07.5521779Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5522491Z "size": 32, 2024-08-20T22:13:07.5523162Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:13:07.5523921Z }, 2024-08-20T22:13:07.5524315Z { 2024-08-20T22:13:07.5524859Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5525570Z "size": 158, 2024-08-20T22:13:07.5526213Z "digest": "sha256:78ee77eddef23f20afc78e2924f4467041cadb754634ee3df431c68211cf676d" 2024-08-20T22:13:07.5526940Z }, 2024-08-20T22:13:07.5527279Z { 2024-08-20T22:13:07.5527813Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5528470Z "size": 1841, 2024-08-20T22:13:07.5529097Z "digest": "sha256:37513e9b43d7123e7ddee09100f62a72b57af7b3833eaecd515e247e2f63e13f" 2024-08-20T22:13:07.5529860Z }, 2024-08-20T22:13:07.5530167Z { 2024-08-20T22:13:07.5530693Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5531401Z "size": 7529776, 2024-08-20T22:13:07.5532120Z "digest": "sha256:a7a5392e2bf6b73d9be292cc22a99ad4637636c15733beb7afbe6403b5ba21a8" 2024-08-20T22:13:07.5532798Z }, 2024-08-20T22:13:07.5533212Z { 2024-08-20T22:13:07.5533743Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5534545Z "size": 163, 2024-08-20T22:13:07.5535308Z "digest": "sha256:b10de02255c021b2366b85bf12ddf593aac89f88964201c455b71a145bc5ca2e" 2024-08-20T22:13:07.5536206Z }, 2024-08-20T22:13:07.5536723Z { 2024-08-20T22:13:07.5537325Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5537973Z "size": 7943, 2024-08-20T22:13:07.5538681Z "digest": "sha256:e56eeff2db93ffe5002a2bb764795b7333321c6802cc28bef0801a619fe6a07e" 2024-08-20T22:13:07.5539387Z }, 2024-08-20T22:13:07.5539765Z { 2024-08-20T22:13:07.5540353Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5540996Z "size": 8068, 2024-08-20T22:13:07.5541656Z "digest": "sha256:70ac3e38d0fd1f0f46ad710c3fc485fd01406606054e8cf84a63af1570bc9f3a" 2024-08-20T22:13:07.5542374Z }, 2024-08-20T22:13:07.5542722Z { 2024-08-20T22:13:07.5543272Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5543927Z "size": 302, 2024-08-20T22:13:07.5544550Z "digest": "sha256:eb09cbf3e4168749cc8605e3188ed35070f6037d13930e443c38d8941734488f" 2024-08-20T22:13:07.5545269Z }, 2024-08-20T22:13:07.5545657Z { 2024-08-20T22:13:07.5546222Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5546887Z "size": 7629638, 2024-08-20T22:13:07.5547561Z "digest": "sha256:ee1273153675e6543b58f5be50321abee5e91f7351701d5d501a6837eaf676ef" 2024-08-20T22:13:07.5548272Z }, 2024-08-20T22:13:07.5548578Z { 2024-08-20T22:13:07.5549140Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5549805Z "size": 108, 2024-08-20T22:13:07.5550436Z "digest": "sha256:e22979f9e0eedfa16f98eb4fa26f30a50c237f5ffd2f6e5b898f2b908058496d" 2024-08-20T22:13:07.5551161Z }, 2024-08-20T22:13:07.5551528Z { 2024-08-20T22:13:07.5552055Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5553213Z "size": 54145661, 2024-08-20T22:13:07.5553988Z "digest": "sha256:6c1f0aabce7c84f9b0b9ab3b96325778dba1c1465b7463c5b4c6be1cf52b74db" 2024-08-20T22:13:07.5554695Z }, 2024-08-20T22:13:07.5555134Z { 2024-08-20T22:13:07.5555626Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5556262Z "size": 473, 2024-08-20T22:13:07.5556965Z "digest": "sha256:7ebe49c3a23e71b289ce012b48dad35791a2f2aa7fcee2326a64d763b0d14df8" 2024-08-20T22:13:07.5557671Z }, 2024-08-20T22:13:07.5557982Z { 2024-08-20T22:13:07.5558578Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5559221Z "size": 1374858766, 2024-08-20T22:13:07.5559944Z "digest": "sha256:5d115b29b5d53e5418762bc41ae975dd245bcffc3d058b8902c58146d505cb37" 2024-08-20T22:13:07.5560676Z }, 2024-08-20T22:13:07.5561019Z { 2024-08-20T22:13:07.5561761Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5562367Z "size": 106, 2024-08-20T22:13:07.5563039Z "digest": "sha256:7f2d7a04fbe65bbe1e895138e0cfeeef53ad926c890bab8e7e132ad1e655e48f" 2024-08-20T22:13:07.5563803Z }, 2024-08-20T22:13:07.5564146Z { 2024-08-20T22:13:07.5564652Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5565339Z "size": 559, 2024-08-20T22:13:07.5565958Z "digest": "sha256:d79fd7d526989e554455eda7d459f2c88444030f61710101e52ec8c7a49b7b79" 2024-08-20T22:13:07.5566671Z }, 2024-08-20T22:13:07.5567110Z { 2024-08-20T22:13:07.5567630Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5568294Z "size": 46248488, 2024-08-20T22:13:07.5568945Z "digest": "sha256:9752d6fbbcab4e7403f26777c5b65911320de6389c81b833a47ba45c4f3354b4" 2024-08-20T22:13:07.5569664Z }, 2024-08-20T22:13:07.5570006Z { 2024-08-20T22:13:07.5570569Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5571209Z "size": 111, 2024-08-20T22:13:07.5571838Z "digest": "sha256:83808b2eff4d71ed46db7c29f66fe918f328405c25b314225dfad01b4868d1f4" 2024-08-20T22:13:07.5572649Z }, 2024-08-20T22:13:07.5573077Z { 2024-08-20T22:13:07.5573621Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5574329Z "size": 32, 2024-08-20T22:13:07.5575150Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:13:07.5576147Z }, 2024-08-20T22:13:07.5576555Z { 2024-08-20T22:13:07.5577210Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5577923Z "size": 32, 2024-08-20T22:13:07.5578524Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:13:07.5579221Z }, 2024-08-20T22:13:07.5579632Z { 2024-08-20T22:13:07.5580142Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5580744Z "size": 32, 2024-08-20T22:13:07.5581526Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:13:07.5582217Z }, 2024-08-20T22:13:07.5582575Z { 2024-08-20T22:13:07.5583139Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:07.5583766Z "size": 32, 2024-08-20T22:13:07.5584376Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:13:07.5585133Z } 2024-08-20T22:13:07.5585432Z ] 2024-08-20T22:13:07.5585783Z } 2024-08-20T22:13:07.5880318Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:13:07.5880795Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:13:07.5881339Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-20T22:13:07.5890931Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:07.5891430Z env: 2024-08-20T22:13:07.5891714Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:07.5892855Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:07.5894025Z ##[endgroup] 2024-08-20T22:13:07.5925503Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:07.5976520Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-20T22:13:07.5977105Z with: 2024-08-20T22:13:07.5978171Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:07.5979512Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:07.5980029Z env: 2024-08-20T22:13:07.5980301Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:07.5980646Z ##[endgroup] 2024-08-20T22:13:07.5997333Z ##[group]Run set -x 2024-08-20T22:13:07.5997827Z set -x 2024-08-20T22:13:07.5998133Z set +e 2024-08-20T22:13:07.5998431Z  2024-08-20T22:13:07.5998708Z login() { 2024-08-20T22:13:07.5999369Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:13:07.6000084Z } 2024-08-20T22:13:07.6000367Z  2024-08-20T22:13:07.6000674Z retry () { 2024-08-20T22:13:07.6001074Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:13:07.6001518Z } 2024-08-20T22:13:07.6001797Z  2024-08-20T22:13:07.6002118Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:13:07.6002530Z  2024-08-20T22:13:07.6002805Z set -e 2024-08-20T22:13:07.6003277Z # ignore output since only exit code is used for conditional 2024-08-20T22:13:07.6003975Z # only pull docker image if it's not available locally 2024-08-20T22:13:07.6004729Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-20T22:13:07.6005423Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-20T22:13:07.6005866Z fi 2024-08-20T22:13:07.6013861Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:07.6014464Z env: 2024-08-20T22:13:07.6014745Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:07.6015885Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:07.6017173Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:07.6017681Z ##[endgroup] 2024-08-20T22:13:07.6044296Z + set +e 2024-08-20T22:13:07.6044936Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:07.6045606Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:07.6047791Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:13:07.6049827Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:08.1492965Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:13:08.1493803Z Configure a credential helper to remove this warning. See 2024-08-20T22:13:08.1494758Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:13:08.1495295Z 2024-08-20T22:13:08.1495432Z Login Succeeded 2024-08-20T22:13:08.1517494Z + set -e 2024-08-20T22:13:08.1518816Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:08.1677783Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:08.1679929Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:08.3921962Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-08-20T22:13:08.3924520Z 7a2c55901189: Pulling fs layer 2024-08-20T22:13:08.3925085Z 224fe954d725: Pulling fs layer 2024-08-20T22:13:08.3925897Z 75722010b82e: Pulling fs layer 2024-08-20T22:13:08.3926444Z d527cbbb87e3: Pulling fs layer 2024-08-20T22:13:08.3926974Z b57676e46aee: Pulling fs layer 2024-08-20T22:13:08.3927471Z a8c1e85b5e14: Pulling fs layer 2024-08-20T22:13:08.3927973Z a41a8d1c11c8: Pulling fs layer 2024-08-20T22:13:08.3928478Z 0c1227890755: Pulling fs layer 2024-08-20T22:13:08.3928968Z d8d1234baab3: Pulling fs layer 2024-08-20T22:13:08.3929472Z 7ed32bc8e469: Pulling fs layer 2024-08-20T22:13:08.3929946Z ec1e7978c1fe: Pulling fs layer 2024-08-20T22:13:08.3930338Z ac23c5fcc1ef: Pulling fs layer 2024-08-20T22:13:08.3930814Z ca426ffc9d2b: Pulling fs layer 2024-08-20T22:13:08.3931449Z 4373aa743caf: Pulling fs layer 2024-08-20T22:13:08.3931892Z 75b2abb16c68: Pulling fs layer 2024-08-20T22:13:08.3932298Z ef1e3ef1e85f: Pulling fs layer 2024-08-20T22:13:08.3932733Z 0bf61114d37b: Pulling fs layer 2024-08-20T22:13:08.3933125Z 2df849f72ba4: Pulling fs layer 2024-08-20T22:13:08.3933574Z 34c47d63c5de: Pulling fs layer 2024-08-20T22:13:08.3933976Z 1fef4bd19792: Pulling fs layer 2024-08-20T22:13:08.3934611Z 6efb96fc87ea: Pulling fs layer 2024-08-20T22:13:08.3935005Z b119c4fc6c58: Pulling fs layer 2024-08-20T22:13:08.3935474Z 33b639226eee: Pulling fs layer 2024-08-20T22:13:08.3935908Z d3547bd43146: Pulling fs layer 2024-08-20T22:13:08.3936332Z 5645520256fa: Pulling fs layer 2024-08-20T22:13:08.3936799Z 829ed7d91e38: Pulling fs layer 2024-08-20T22:13:08.3937266Z 385d0421d67c: Pulling fs layer 2024-08-20T22:13:08.3937732Z 7cbfce999d76: Pulling fs layer 2024-08-20T22:13:08.3938196Z 6658e558ebe4: Pulling fs layer 2024-08-20T22:13:08.3938579Z 33444a5a7e01: Pulling fs layer 2024-08-20T22:13:08.3939033Z 7c47122f4e09: Pulling fs layer 2024-08-20T22:13:08.3939414Z 1811ff4d3556: Pulling fs layer 2024-08-20T22:13:08.3939795Z 7754ee068b33: Pulling fs layer 2024-08-20T22:13:08.3940162Z a8c1e85b5e14: Waiting 2024-08-20T22:13:08.3940510Z 0479fbac3361: Pulling fs layer 2024-08-20T22:13:08.3940887Z f1d6874dee9c: Pulling fs layer 2024-08-20T22:13:08.3941278Z afc429e87b59: Pulling fs layer 2024-08-20T22:13:08.3941660Z 019620471c25: Pulling fs layer 2024-08-20T22:13:08.3942041Z 27aeb1b2cd5e: Pulling fs layer 2024-08-20T22:13:08.3942424Z 829c0e19f4b7: Pulling fs layer 2024-08-20T22:13:08.3942858Z 560cd6122a38: Pulling fs layer 2024-08-20T22:13:08.3943222Z a41a8d1c11c8: Waiting 2024-08-20T22:13:08.3943548Z 33b639226eee: Waiting 2024-08-20T22:13:08.3943966Z ac23c5fcc1ef: Waiting 2024-08-20T22:13:08.3944377Z ca426ffc9d2b: Waiting 2024-08-20T22:13:08.3944747Z 0c1227890755: Waiting 2024-08-20T22:13:08.3945119Z 4373aa743caf: Waiting 2024-08-20T22:13:08.3945445Z d8d1234baab3: Waiting 2024-08-20T22:13:08.3945778Z 7ed32bc8e469: Waiting 2024-08-20T22:13:08.3946107Z 2df849f72ba4: Waiting 2024-08-20T22:13:08.3946450Z 5a4eb19d7219: Pulling fs layer 2024-08-20T22:13:08.3946837Z da6a838a66c7: Pulling fs layer 2024-08-20T22:13:08.3947346Z cbb79a705353: Pulling fs layer 2024-08-20T22:13:08.3947731Z 4d4fa6802ab3: Pulling fs layer 2024-08-20T22:13:08.3948110Z 6efb96fc87ea: Waiting 2024-08-20T22:13:08.3948462Z 0f085d4fa279: Pulling fs layer 2024-08-20T22:13:08.3948844Z 4a0412373305: Pulling fs layer 2024-08-20T22:13:08.3949206Z 1fef4bd19792: Waiting 2024-08-20T22:13:08.3949550Z 302c8c177944: Pulling fs layer 2024-08-20T22:13:08.3949929Z 02970347dff6: Pulling fs layer 2024-08-20T22:13:08.3950287Z 7c47122f4e09: Waiting 2024-08-20T22:13:08.3950628Z 916c5e858016: Pulling fs layer 2024-08-20T22:13:08.3950995Z 75b2abb16c68: Waiting 2024-08-20T22:13:08.3951325Z ec1e7978c1fe: Waiting 2024-08-20T22:13:08.3951671Z c42b3602a1fb: Pulling fs layer 2024-08-20T22:13:08.3952061Z 11fad972ef6c: Pulling fs layer 2024-08-20T22:13:08.3952446Z 34c47d63c5de: Waiting 2024-08-20T22:13:08.3953057Z ef1e3ef1e85f: Waiting 2024-08-20T22:13:08.3953391Z 27aeb1b2cd5e: Waiting 2024-08-20T22:13:08.3953794Z f957c1624481: Pulling fs layer 2024-08-20T22:13:08.3954178Z 768c31dca016: Pulling fs layer 2024-08-20T22:13:08.3954593Z 829c0e19f4b7: Waiting 2024-08-20T22:13:08.3954917Z 560cd6122a38: Waiting 2024-08-20T22:13:08.3955520Z afc429e87b59: Waiting 2024-08-20T22:13:08.3955878Z 8c32d0ecebbe: Pulling fs layer 2024-08-20T22:13:08.3956240Z 5645520256fa: Waiting 2024-08-20T22:13:08.3956561Z b57676e46aee: Waiting 2024-08-20T22:13:08.3956884Z 019620471c25: Waiting 2024-08-20T22:13:08.3957203Z d3547bd43146: Waiting 2024-08-20T22:13:08.3957544Z 916c5e858016: Waiting 2024-08-20T22:13:08.3957913Z 7cbfce999d76: Waiting 2024-08-20T22:13:08.3958258Z 4f4fb700ef54: Pulling fs layer 2024-08-20T22:13:08.3958620Z c42b3602a1fb: Waiting 2024-08-20T22:13:08.3958960Z 78ee77eddef2: Pulling fs layer 2024-08-20T22:13:08.3959342Z 37513e9b43d7: Pulling fs layer 2024-08-20T22:13:08.3959874Z cbb79a705353: Waiting 2024-08-20T22:13:08.3960283Z a7a5392e2bf6: Pulling fs layer 2024-08-20T22:13:08.3960681Z b10de02255c0: Pulling fs layer 2024-08-20T22:13:08.3961071Z e56eeff2db93: Pulling fs layer 2024-08-20T22:13:08.3961433Z 5a4eb19d7219: Waiting 2024-08-20T22:13:08.3961760Z da6a838a66c7: Waiting 2024-08-20T22:13:08.3962101Z 70ac3e38d0fd: Pulling fs layer 2024-08-20T22:13:08.3962497Z eb09cbf3e416: Pulling fs layer 2024-08-20T22:13:08.3962873Z ee1273153675: Pulling fs layer 2024-08-20T22:13:08.3963237Z 4d4fa6802ab3: Waiting 2024-08-20T22:13:08.3963577Z e22979f9e0ee: Pulling fs layer 2024-08-20T22:13:08.3963946Z d527cbbb87e3: Waiting 2024-08-20T22:13:08.3964264Z 02970347dff6: Waiting 2024-08-20T22:13:08.3964605Z 6c1f0aabce7c: Pulling fs layer 2024-08-20T22:13:08.3965054Z 7ebe49c3a23e: Pulling fs layer 2024-08-20T22:13:08.3965433Z 5d115b29b5d5: Pulling fs layer 2024-08-20T22:13:08.3965804Z 6658e558ebe4: Waiting 2024-08-20T22:13:08.3966153Z 7f2d7a04fbe6: Pulling fs layer 2024-08-20T22:13:08.3966521Z 33444a5a7e01: Waiting 2024-08-20T22:13:08.3966851Z 8c32d0ecebbe: Waiting 2024-08-20T22:13:08.3967207Z f1d6874dee9c: Waiting 2024-08-20T22:13:08.3967607Z d79fd7d52698: Pulling fs layer 2024-08-20T22:13:08.3967993Z 9752d6fbbcab: Pulling fs layer 2024-08-20T22:13:08.3968355Z f957c1624481: Waiting 2024-08-20T22:13:08.3968693Z 83808b2eff4d: Pulling fs layer 2024-08-20T22:13:08.3969067Z 78ee77eddef2: Waiting 2024-08-20T22:13:08.3969397Z 4f4fb700ef54: Waiting 2024-08-20T22:13:08.3969717Z 1811ff4d3556: Waiting 2024-08-20T22:13:08.3970039Z 385d0421d67c: Waiting 2024-08-20T22:13:08.3970366Z 7ebe49c3a23e: Waiting 2024-08-20T22:13:08.3970685Z 5d115b29b5d5: Waiting 2024-08-20T22:13:08.3971009Z a7a5392e2bf6: Waiting 2024-08-20T22:13:08.3971418Z 4a0412373305: Waiting 2024-08-20T22:13:08.3971741Z 9752d6fbbcab: Waiting 2024-08-20T22:13:08.3972061Z d79fd7d52698: Waiting 2024-08-20T22:13:08.3972389Z 83808b2eff4d: Waiting 2024-08-20T22:13:08.3972762Z b10de02255c0: Waiting 2024-08-20T22:13:08.3973079Z 768c31dca016: Waiting 2024-08-20T22:13:08.3973407Z 70ac3e38d0fd: Waiting 2024-08-20T22:13:08.3973734Z e56eeff2db93: Waiting 2024-08-20T22:13:08.3974061Z eb09cbf3e416: Waiting 2024-08-20T22:13:08.3974478Z ee1273153675: Waiting 2024-08-20T22:13:08.3974853Z 0f085d4fa279: Waiting 2024-08-20T22:13:08.3975178Z e22979f9e0ee: Waiting 2024-08-20T22:13:08.3975546Z 6c1f0aabce7c: Waiting 2024-08-20T22:13:08.3975887Z 302c8c177944: Waiting 2024-08-20T22:13:08.5241367Z 224fe954d725: Verifying Checksum 2024-08-20T22:13:08.5241818Z 224fe954d725: Download complete 2024-08-20T22:13:08.5920585Z d527cbbb87e3: Verifying Checksum 2024-08-20T22:13:08.5921040Z d527cbbb87e3: Download complete 2024-08-20T22:13:08.6678924Z b57676e46aee: Download complete 2024-08-20T22:13:08.7301139Z 7a2c55901189: Verifying Checksum 2024-08-20T22:13:08.7301575Z 7a2c55901189: Download complete 2024-08-20T22:13:08.8142028Z a41a8d1c11c8: Download complete 2024-08-20T22:13:08.8750541Z 0c1227890755: Verifying Checksum 2024-08-20T22:13:08.8751101Z 0c1227890755: Download complete 2024-08-20T22:13:08.9522683Z d8d1234baab3: Verifying Checksum 2024-08-20T22:13:08.9523190Z d8d1234baab3: Download complete 2024-08-20T22:13:08.9945562Z 75722010b82e: Verifying Checksum 2024-08-20T22:13:08.9946039Z 75722010b82e: Download complete 2024-08-20T22:13:09.0946071Z ec1e7978c1fe: Verifying Checksum 2024-08-20T22:13:09.0946545Z ec1e7978c1fe: Download complete 2024-08-20T22:13:09.1686267Z ac23c5fcc1ef: Download complete 2024-08-20T22:13:09.8939648Z 7a2c55901189: Pull complete 2024-08-20T22:13:10.2129537Z 224fe954d725: Pull complete 2024-08-20T22:13:11.1361405Z 75722010b82e: Pull complete 2024-08-20T22:13:11.1562067Z d527cbbb87e3: Pull complete 2024-08-20T22:13:11.1764189Z b57676e46aee: Pull complete 2024-08-20T22:13:11.6903334Z ca426ffc9d2b: Verifying Checksum 2024-08-20T22:13:11.6903801Z ca426ffc9d2b: Download complete 2024-08-20T22:13:11.8006948Z 4373aa743caf: Download complete 2024-08-20T22:13:11.8706444Z 75b2abb16c68: Verifying Checksum 2024-08-20T22:13:11.8707083Z 75b2abb16c68: Download complete 2024-08-20T22:13:12.0116900Z ef1e3ef1e85f: Verifying Checksum 2024-08-20T22:13:12.0117521Z ef1e3ef1e85f: Download complete 2024-08-20T22:13:12.9770823Z 0bf61114d37b: Verifying Checksum 2024-08-20T22:13:12.9771304Z 0bf61114d37b: Download complete 2024-08-20T22:13:13.0557017Z 2df849f72ba4: Verifying Checksum 2024-08-20T22:13:13.0557615Z 2df849f72ba4: Download complete 2024-08-20T22:13:13.1195540Z 34c47d63c5de: Verifying Checksum 2024-08-20T22:13:13.1196151Z 34c47d63c5de: Download complete 2024-08-20T22:13:13.2036381Z 1fef4bd19792: Verifying Checksum 2024-08-20T22:13:13.2037014Z 1fef4bd19792: Download complete 2024-08-20T22:13:22.0276905Z a8c1e85b5e14: Verifying Checksum 2024-08-20T22:13:22.0277529Z a8c1e85b5e14: Download complete 2024-08-20T22:13:22.0925154Z b119c4fc6c58: Verifying Checksum 2024-08-20T22:13:22.0925705Z b119c4fc6c58: Download complete 2024-08-20T22:13:22.1637384Z 33b639226eee: Download complete 2024-08-20T22:13:22.2361872Z d3547bd43146: Download complete 2024-08-20T22:13:22.3170707Z 5645520256fa: Verifying Checksum 2024-08-20T22:13:22.3171353Z 5645520256fa: Download complete 2024-08-20T22:13:22.3793835Z 829ed7d91e38: Verifying Checksum 2024-08-20T22:13:22.3794478Z 829ed7d91e38: Download complete 2024-08-20T22:13:23.6375416Z 385d0421d67c: Verifying Checksum 2024-08-20T22:13:23.6376069Z 385d0421d67c: Download complete 2024-08-20T22:13:23.7033643Z 7cbfce999d76: Verifying Checksum 2024-08-20T22:13:23.7034355Z 7cbfce999d76: Download complete 2024-08-20T22:13:23.7678601Z 6658e558ebe4: Download complete 2024-08-20T22:13:23.8412564Z 33444a5a7e01: Verifying Checksum 2024-08-20T22:13:23.8413097Z 33444a5a7e01: Download complete 2024-08-20T22:13:23.9068448Z 7c47122f4e09: Verifying Checksum 2024-08-20T22:13:23.9069113Z 7c47122f4e09: Download complete 2024-08-20T22:13:23.9821803Z 1811ff4d3556: Verifying Checksum 2024-08-20T22:13:23.9822398Z 1811ff4d3556: Download complete 2024-08-20T22:13:30.0500276Z 7754ee068b33: Verifying Checksum 2024-08-20T22:13:30.0500755Z 7754ee068b33: Download complete 2024-08-20T22:13:30.1211471Z 0479fbac3361: Download complete 2024-08-20T22:13:30.1977813Z f1d6874dee9c: Download complete 2024-08-20T22:13:30.6011558Z afc429e87b59: Verifying Checksum 2024-08-20T22:13:30.6012240Z afc429e87b59: Download complete 2024-08-20T22:13:30.6763874Z 019620471c25: Verifying Checksum 2024-08-20T22:13:30.6764500Z 019620471c25: Download complete 2024-08-20T22:13:30.7391921Z 27aeb1b2cd5e: Verifying Checksum 2024-08-20T22:13:30.7392423Z 27aeb1b2cd5e: Download complete 2024-08-20T22:13:30.9856908Z 829c0e19f4b7: Verifying Checksum 2024-08-20T22:13:30.9857398Z 829c0e19f4b7: Download complete 2024-08-20T22:13:31.0550416Z 560cd6122a38: Verifying Checksum 2024-08-20T22:13:31.0551261Z 560cd6122a38: Download complete 2024-08-20T22:13:31.1508940Z 5a4eb19d7219: Verifying Checksum 2024-08-20T22:13:31.1509364Z 5a4eb19d7219: Download complete 2024-08-20T22:13:31.2435123Z da6a838a66c7: Download complete 2024-08-20T22:13:34.2810149Z 7ed32bc8e469: Verifying Checksum 2024-08-20T22:13:34.2810992Z 7ed32bc8e469: Download complete 2024-08-20T22:13:34.4398025Z 0f085d4fa279: Verifying Checksum 2024-08-20T22:13:34.4398604Z 0f085d4fa279: Download complete 2024-08-20T22:13:34.9914152Z 4a0412373305: Verifying Checksum 2024-08-20T22:13:34.9914746Z 4a0412373305: Download complete 2024-08-20T22:13:35.0822961Z 302c8c177944: Download complete 2024-08-20T22:13:35.1619614Z 02970347dff6: Download complete 2024-08-20T22:13:35.2189447Z a8c1e85b5e14: Pull complete 2024-08-20T22:13:35.2288155Z 916c5e858016: Verifying Checksum 2024-08-20T22:13:35.2288579Z 916c5e858016: Download complete 2024-08-20T22:13:35.3076845Z c42b3602a1fb: Download complete 2024-08-20T22:13:35.4060952Z a41a8d1c11c8: Pull complete 2024-08-20T22:13:35.5941343Z 0c1227890755: Pull complete 2024-08-20T22:13:35.7509845Z d8d1234baab3: Pull complete 2024-08-20T22:13:41.8971591Z 6efb96fc87ea: Verifying Checksum 2024-08-20T22:13:41.8972227Z 6efb96fc87ea: Download complete 2024-08-20T22:13:41.9668315Z f957c1624481: Verifying Checksum 2024-08-20T22:13:41.9668864Z f957c1624481: Download complete 2024-08-20T22:13:42.0322027Z 768c31dca016: Verifying Checksum 2024-08-20T22:13:42.1120424Z 768c31dca016: Download complete 2024-08-20T22:13:42.1120899Z 8c32d0ecebbe: Verifying Checksum 2024-08-20T22:13:42.1121395Z 8c32d0ecebbe: Download complete 2024-08-20T22:13:42.1206014Z 4f4fb700ef54: Verifying Checksum 2024-08-20T22:13:42.1206447Z 4f4fb700ef54: Download complete 2024-08-20T22:13:42.1923151Z 78ee77eddef2: Verifying Checksum 2024-08-20T22:13:42.1923575Z 78ee77eddef2: Download complete 2024-08-20T22:13:42.2618026Z 37513e9b43d7: Verifying Checksum 2024-08-20T22:13:42.2618493Z 37513e9b43d7: Download complete 2024-08-20T22:13:42.3855607Z a7a5392e2bf6: Verifying Checksum 2024-08-20T22:13:42.3856061Z a7a5392e2bf6: Download complete 2024-08-20T22:13:42.4641026Z b10de02255c0: Verifying Checksum 2024-08-20T22:13:42.4641606Z b10de02255c0: Download complete 2024-08-20T22:13:42.5477644Z e56eeff2db93: Download complete 2024-08-20T22:13:42.6137497Z 70ac3e38d0fd: Verifying Checksum 2024-08-20T22:13:42.6137943Z 70ac3e38d0fd: Download complete 2024-08-20T22:13:42.6878838Z eb09cbf3e416: Download complete 2024-08-20T22:13:42.8134972Z ee1273153675: Verifying Checksum 2024-08-20T22:13:42.8135571Z ee1273153675: Download complete 2024-08-20T22:13:42.8816337Z e22979f9e0ee: Verifying Checksum 2024-08-20T22:13:42.8816783Z e22979f9e0ee: Download complete 2024-08-20T22:13:43.4721830Z 6c1f0aabce7c: Verifying Checksum 2024-08-20T22:13:43.4722486Z 6c1f0aabce7c: Download complete 2024-08-20T22:13:43.5347223Z 7ebe49c3a23e: Verifying Checksum 2024-08-20T22:13:43.5347860Z 7ebe49c3a23e: Download complete 2024-08-20T22:14:09.0772959Z 5d115b29b5d5: Verifying Checksum 2024-08-20T22:14:09.0773419Z 5d115b29b5d5: Download complete 2024-08-20T22:14:09.1505789Z 7f2d7a04fbe6: Verifying Checksum 2024-08-20T22:14:09.1506343Z 7f2d7a04fbe6: Download complete 2024-08-20T22:14:09.2567561Z d79fd7d52698: Download complete 2024-08-20T22:14:10.4784750Z 9752d6fbbcab: Verifying Checksum 2024-08-20T22:14:10.4785233Z 9752d6fbbcab: Download complete 2024-08-20T22:14:10.5466115Z 83808b2eff4d: Verifying Checksum 2024-08-20T22:14:10.5466657Z 83808b2eff4d: Download complete 2024-08-20T22:14:48.6659817Z 11fad972ef6c: Verifying Checksum 2024-08-20T22:14:48.6660329Z 11fad972ef6c: Download complete 2024-08-20T22:14:50.7848009Z cbb79a705353: Verifying Checksum 2024-08-20T22:14:50.7848467Z cbb79a705353: Download complete 2024-08-20T22:15:05.1756748Z 7ed32bc8e469: Pull complete 2024-08-20T22:15:05.4128240Z ec1e7978c1fe: Pull complete 2024-08-20T22:15:05.6537813Z ac23c5fcc1ef: Pull complete 2024-08-20T22:15:14.1910322Z ca426ffc9d2b: Pull complete 2024-08-20T22:15:14.4080577Z 4373aa743caf: Pull complete 2024-08-20T22:15:14.6431040Z 75b2abb16c68: Pull complete 2024-08-20T22:15:14.8704668Z ef1e3ef1e85f: Pull complete 2024-08-20T22:15:17.5096241Z 0bf61114d37b: Pull complete 2024-08-20T22:15:17.6322888Z 2df849f72ba4: Pull complete 2024-08-20T22:15:17.7358484Z 34c47d63c5de: Pull complete 2024-08-20T22:15:17.8570632Z 1fef4bd19792: Pull complete 2024-08-20T22:16:16.2224333Z 6efb96fc87ea: Pull complete 2024-08-20T22:16:16.4558303Z b119c4fc6c58: Pull complete 2024-08-20T22:16:16.7073183Z 33b639226eee: Pull complete 2024-08-20T22:16:16.8359554Z d3547bd43146: Pull complete 2024-08-20T22:16:16.9928603Z 5645520256fa: Pull complete 2024-08-20T22:16:17.2259375Z 829ed7d91e38: Pull complete 2024-08-20T22:16:20.9283685Z 385d0421d67c: Pull complete 2024-08-20T22:16:21.0788085Z 7cbfce999d76: Pull complete 2024-08-20T22:16:21.2941574Z 6658e558ebe4: Pull complete 2024-08-20T22:16:21.5204406Z 33444a5a7e01: Pull complete 2024-08-20T22:16:21.6572063Z 7c47122f4e09: Pull complete 2024-08-20T22:16:21.8477444Z 1811ff4d3556: Pull complete 2024-08-20T22:16:32.5271467Z 7754ee068b33: Pull complete 2024-08-20T22:16:32.7561422Z 0479fbac3361: Pull complete 2024-08-20T22:16:32.9975302Z f1d6874dee9c: Pull complete 2024-08-20T22:16:33.9370928Z afc429e87b59: Pull complete 2024-08-20T22:16:34.0852925Z 019620471c25: Pull complete 2024-08-20T22:16:34.2312079Z 27aeb1b2cd5e: Pull complete 2024-08-20T22:16:34.6465241Z 829c0e19f4b7: Pull complete 2024-08-20T22:16:34.8070154Z 560cd6122a38: Pull complete 2024-08-20T22:16:35.0950163Z 5a4eb19d7219: Pull complete 2024-08-20T22:16:35.2698953Z da6a838a66c7: Pull complete 2024-08-20T22:17:13.8589496Z cbb79a705353: Pull complete 2024-08-20T22:17:14.0900906Z 4d4fa6802ab3: Pull complete 2024-08-20T22:17:14.3133456Z 0f085d4fa279: Pull complete 2024-08-20T22:17:16.7309172Z 4a0412373305: Pull complete 2024-08-20T22:17:16.7429256Z 302c8c177944: Pull complete 2024-08-20T22:17:16.7546302Z 02970347dff6: Pull complete 2024-08-20T22:17:16.7770880Z 916c5e858016: Pull complete 2024-08-20T22:17:16.7885710Z c42b3602a1fb: Pull complete 2024-08-20T22:18:38.6957511Z 11fad972ef6c: Pull complete 2024-08-20T22:18:38.9297542Z f957c1624481: Pull complete 2024-08-20T22:18:39.1622424Z 768c31dca016: Pull complete 2024-08-20T22:18:39.6008020Z 8c32d0ecebbe: Pull complete 2024-08-20T22:18:39.8224183Z 4f4fb700ef54: Pull complete 2024-08-20T22:18:40.0461331Z 78ee77eddef2: Pull complete 2024-08-20T22:18:40.2735257Z 37513e9b43d7: Pull complete 2024-08-20T22:18:40.6626175Z a7a5392e2bf6: Pull complete 2024-08-20T22:18:40.8806761Z b10de02255c0: Pull complete 2024-08-20T22:18:41.0833908Z e56eeff2db93: Pull complete 2024-08-20T22:18:41.3117566Z 70ac3e38d0fd: Pull complete 2024-08-20T22:18:41.5420568Z eb09cbf3e416: Pull complete 2024-08-20T22:18:42.8700168Z ee1273153675: Pull complete 2024-08-20T22:18:43.0920229Z e22979f9e0ee: Pull complete 2024-08-20T22:18:45.2760661Z 6c1f0aabce7c: Pull complete 2024-08-20T22:18:45.5165430Z 7ebe49c3a23e: Pull complete 2024-08-20T22:18:58.7841584Z 5d115b29b5d5: Pull complete 2024-08-20T22:18:58.8086351Z 7f2d7a04fbe6: Pull complete 2024-08-20T22:18:58.9629748Z d79fd7d52698: Pull complete 2024-08-20T22:18:59.7875692Z 9752d6fbbcab: Pull complete 2024-08-20T22:19:00.0008299Z 83808b2eff4d: Pull complete 2024-08-20T22:19:00.9336401Z Digest: sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-20T22:19:00.9743930Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:19:00.9967445Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:19:01.0060120Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:19:01.0061043Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:19:01.0071793Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:01.0072287Z env: 2024-08-20T22:19:01.0072580Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:01.0072938Z ##[endgroup] 2024-08-20T22:19:01.0263206Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-20T22:19:01.0263761Z with: 2024-08-20T22:19:01.0264053Z driver-version: 550.54.15 2024-08-20T22:19:01.0264395Z env: 2024-08-20T22:19:01.0264676Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:01.0265046Z ##[endgroup] 2024-08-20T22:19:01.0371423Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:19:01.0371988Z with: 2024-08-20T22:19:01.0372274Z timeout_minutes: 10 2024-08-20T22:19:01.0372607Z max_attempts: 3 2024-08-20T22:19:01.0405114Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-20T22:19:01.0438636Z retry_wait_seconds: 10 2024-08-20T22:19:01.0439015Z polling_interval_seconds: 1 2024-08-20T22:19:01.0439392Z warning_on_retry: true 2024-08-20T22:19:01.0439758Z continue_on_error: false 2024-08-20T22:19:01.0440106Z env: 2024-08-20T22:19:01.0440397Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:01.0440754Z DRIVER_VERSION: 550.54.15 2024-08-20T22:19:01.0441112Z ##[endgroup] 2024-08-20T22:19:01.1252928Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-20T22:19:01.1253620Z + pre_install_nvidia_driver_amzn2 2024-08-20T22:19:01.1256146Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-20T22:19:01.5816783Z No match for argument: nvidia-driver-latest-dkms 2024-08-20T22:19:01.5817314Z No packages marked for removal. 2024-08-20T22:19:01.5881926Z Dependencies resolved. 2024-08-20T22:19:01.5892120Z Nothing to do. 2024-08-20T22:19:01.5893048Z Complete! 2024-08-20T22:19:01.6644586Z + install_nvidia_driver_common 2024-08-20T22:19:01.6652407Z + echo 'Before installing NVIDIA driver' 2024-08-20T22:19:01.6653187Z Before installing NVIDIA driver 2024-08-20T22:19:01.6653798Z + lspci 2024-08-20T22:19:01.7431142Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:19:01.7431912Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:19:01.7432873Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:19:01.7433751Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:19:01.7434578Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:19:01.7435368Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:19:01.7436116Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:19:01.7436931Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:19:01.7437530Z + lsmod 2024-08-20T22:19:01.7478474Z Module Size Used by 2024-08-20T22:19:01.7479138Z xt_conntrack 16384 1 2024-08-20T22:19:01.7479597Z nft_chain_nat 16384 3 2024-08-20T22:19:01.7480056Z xt_MASQUERADE 20480 1 2024-08-20T22:19:01.7480682Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-20T22:19:01.7481386Z nf_conntrack_netlink 57344 0 2024-08-20T22:19:01.7482162Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:19:01.7482917Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:19:01.7483572Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:19:01.7484162Z xfrm_user 57344 1 2024-08-20T22:19:01.7484617Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:19:01.7485072Z xt_addrtype 16384 2 2024-08-20T22:19:01.7485465Z nft_compat 20480 4 2024-08-20T22:19:01.7485935Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:19:01.7486562Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:19:01.7487114Z br_netfilter 36864 0 2024-08-20T22:19:01.7487557Z bridge 307200 1 br_netfilter 2024-08-20T22:19:01.7488008Z stp 16384 1 bridge 2024-08-20T22:19:01.7488851Z llc 16384 2 bridge,stp 2024-08-20T22:19:01.7489348Z overlay 167936 0 2024-08-20T22:19:01.7489856Z tls 114688 0 2024-08-20T22:19:01.7490354Z nls_ascii 16384 1 2024-08-20T22:19:01.7490848Z nls_cp437 20480 1 2024-08-20T22:19:01.7491254Z vfat 24576 1 2024-08-20T22:19:01.7491632Z fat 86016 1 vfat 2024-08-20T22:19:01.7492033Z ena 167936 0 2024-08-20T22:19:01.7492412Z ptp 36864 1 ena 2024-08-20T22:19:01.7492826Z pps_core 24576 1 ptp 2024-08-20T22:19:01.7493236Z ghash_clmulni_intel 16384 0 2024-08-20T22:19:01.7493624Z aesni_intel 393216 0 2024-08-20T22:19:01.7494021Z i8042 45056 0 2024-08-20T22:19:01.7494460Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:19:01.7495045Z serio 28672 3 i8042 2024-08-20T22:19:01.7495538Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:19:01.7496061Z button 24576 0 2024-08-20T22:19:01.7496448Z sunrpc 692224 1 2024-08-20T22:19:01.7496832Z sch_fq_codel 20480 17 2024-08-20T22:19:01.7497210Z dm_mod 188416 0 2024-08-20T22:19:01.7497585Z fuse 163840 1 2024-08-20T22:19:01.7497970Z dax 45056 1 dm_mod 2024-08-20T22:19:01.7498379Z configfs 57344 1 2024-08-20T22:19:01.7498876Z loop 36864 0 2024-08-20T22:19:01.7499363Z dmi_sysfs 20480 0 2024-08-20T22:19:01.7499859Z crc32_pclmul 16384 0 2024-08-20T22:19:01.7500356Z crc32c_intel 24576 0 2024-08-20T22:19:01.7500765Z efivarfs 24576 1 2024-08-20T22:19:01.7501144Z + modinfo nvidia 2024-08-20T22:19:01.7502008Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:19:01.7502726Z alias: char-major-195-* 2024-08-20T22:19:01.7503110Z version: 550.54.15 2024-08-20T22:19:01.7503474Z supported: external 2024-08-20T22:19:01.7503827Z license: NVIDIA 2024-08-20T22:19:01.7504218Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:19:01.7504707Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:19:01.7505183Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:19:01.7505658Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:19:01.7506164Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:19:01.7506666Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:19:01.7507185Z depends: i2c-core,drm 2024-08-20T22:19:01.7507558Z retpoline: Y 2024-08-20T22:19:01.7507868Z name: nvidia 2024-08-20T22:19:01.7508634Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:19:01.7509352Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:19:01.7510013Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:19:01.7510618Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:19:01.7511073Z parm: NVreg_RmLogonRC:int 2024-08-20T22:19:01.7511518Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:19:01.7511985Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:19:01.7512424Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:19:01.7512876Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:19:01.7513416Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:19:01.7513992Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:19:01.7514526Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:19:01.7514966Z parm: NVreg_EnableMSI:int 2024-08-20T22:19:01.7515400Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:19:01.7515862Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:19:01.7516405Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:19:01.7516999Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:19:01.7517653Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:19:01.7518267Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:19:01.7518880Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:19:01.7519498Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:19:01.7520102Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:19:01.7520599Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:19:01.7521144Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:19:01.7521701Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:19:01.7522211Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:19:01.7522690Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:19:01.7523184Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:19:01.7523668Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:19:01.7524126Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:19:01.7524693Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:19:01.7525234Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:19:01.7525723Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:19:01.7526220Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:19:01.7526727Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:19:01.7527222Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:19:01.7527730Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:19:01.7528220Z parm: NVreg_RmMsg:charp 2024-08-20T22:19:01.7528634Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:19:01.7529111Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:19:01.7529598Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:19:01.7530063Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:19:01.7530552Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:19:01.7531038Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:19:01.7531507Z parm: rm_firmware_active:charp 2024-08-20T22:19:01.7531923Z + HAS_NVIDIA_DRIVER=0 2024-08-20T22:19:01.7532328Z ++ command -v nvidia-smi 2024-08-20T22:19:01.7532742Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-20T22:19:01.7533123Z + set +e 2024-08-20T22:19:01.7533636Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-20T22:19:04.0090913Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-20T22:19:04.0091353Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:19:04.0091954Z + '[' 0 -ne 0 ']' 2024-08-20T22:19:04.0092338Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-20T22:19:04.0092734Z + HAS_NVIDIA_DRIVER=1 2024-08-20T22:19:04.0093713Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-20T22:19:04.0094435Z + set -e 2024-08-20T22:19:04.0094834Z + '[' 1 -eq 0 ']' 2024-08-20T22:19:04.0095427Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-20T22:19:04.0096115Z + post_install_nvidia_driver_common 2024-08-20T22:19:04.0098762Z + sudo modprobe nvidia 2024-08-20T22:19:04.1448593Z + echo 'After installing NVIDIA driver' 2024-08-20T22:19:04.1449312Z + lspci 2024-08-20T22:19:04.1449731Z After installing NVIDIA driver 2024-08-20T22:19:04.1559158Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:19:04.1560708Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:19:04.1562732Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:19:04.1564206Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:19:04.1565655Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:19:04.1566602Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:19:04.1567342Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:19:04.1568172Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:19:04.1569096Z + lsmod 2024-08-20T22:19:04.1593213Z Module Size Used by 2024-08-20T22:19:04.1593792Z nvidia_uvm 4706304 0 2024-08-20T22:19:04.1594293Z nvidia 54071296 1 nvidia_uvm 2024-08-20T22:19:04.1594850Z drm 602112 1 nvidia 2024-08-20T22:19:04.1595418Z drm_panel_orientation_quirks 28672 1 drm 2024-08-20T22:19:04.1596027Z backlight 24576 1 drm 2024-08-20T22:19:04.1596467Z i2c_core 106496 2 nvidia,drm 2024-08-20T22:19:04.1596894Z xt_conntrack 16384 1 2024-08-20T22:19:04.1597283Z nft_chain_nat 16384 3 2024-08-20T22:19:04.1597694Z xt_MASQUERADE 20480 1 2024-08-20T22:19:04.1598166Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-20T22:19:04.1598662Z nf_conntrack_netlink 57344 0 2024-08-20T22:19:04.1599254Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:19:04.1599912Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:19:04.1600388Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:19:04.1600823Z xfrm_user 57344 1 2024-08-20T22:19:04.1601228Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:19:04.1601653Z xt_addrtype 16384 2 2024-08-20T22:19:04.1602037Z nft_compat 20480 4 2024-08-20T22:19:04.1602499Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:19:04.1603118Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:19:04.1603668Z br_netfilter 36864 0 2024-08-20T22:19:04.1604085Z bridge 307200 1 br_netfilter 2024-08-20T22:19:04.1604530Z stp 16384 1 bridge 2024-08-20T22:19:04.1604965Z llc 16384 2 bridge,stp 2024-08-20T22:19:04.1605441Z overlay 167936 0 2024-08-20T22:19:04.1605818Z tls 114688 0 2024-08-20T22:19:04.1606195Z nls_ascii 16384 1 2024-08-20T22:19:04.1606574Z nls_cp437 20480 1 2024-08-20T22:19:04.1606949Z vfat 24576 1 2024-08-20T22:19:04.1607317Z fat 86016 1 vfat 2024-08-20T22:19:04.1607714Z ena 167936 0 2024-08-20T22:19:04.1608094Z ptp 36864 1 ena 2024-08-20T22:19:04.1608497Z pps_core 24576 1 ptp 2024-08-20T22:19:04.1608901Z ghash_clmulni_intel 16384 0 2024-08-20T22:19:04.1609289Z aesni_intel 393216 0 2024-08-20T22:19:04.1609663Z i8042 45056 0 2024-08-20T22:19:04.1610064Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:19:04.1610502Z serio 28672 3 i8042 2024-08-20T22:19:04.1611283Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:19:04.1611990Z button 24576 0 2024-08-20T22:19:04.1612487Z sunrpc 692224 1 2024-08-20T22:19:04.1612943Z sch_fq_codel 20480 17 2024-08-20T22:19:04.1613456Z dm_mod 188416 0 2024-08-20T22:19:04.1613953Z fuse 163840 1 2024-08-20T22:19:04.1614337Z dax 45056 1 dm_mod 2024-08-20T22:19:04.1614861Z configfs 57344 1 2024-08-20T22:19:04.1615244Z loop 36864 0 2024-08-20T22:19:04.1615622Z dmi_sysfs 20480 0 2024-08-20T22:19:04.1615996Z crc32_pclmul 16384 0 2024-08-20T22:19:04.1616375Z crc32c_intel 24576 0 2024-08-20T22:19:04.1616752Z efivarfs 24576 1 2024-08-20T22:19:04.1617113Z + modinfo nvidia 2024-08-20T22:19:04.1618018Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:19:04.1618760Z alias: char-major-195-* 2024-08-20T22:19:04.1619155Z version: 550.54.15 2024-08-20T22:19:04.1619509Z supported: external 2024-08-20T22:19:04.1619852Z license: NVIDIA 2024-08-20T22:19:04.1620240Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:19:04.1620870Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:19:04.1621342Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:19:04.1621814Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:19:04.1622322Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:19:04.1622819Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:19:04.1623319Z depends: i2c-core,drm 2024-08-20T22:19:04.1623688Z retpoline: Y 2024-08-20T22:19:04.1624004Z name: nvidia 2024-08-20T22:19:04.1624633Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:19:04.1625339Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:19:04.1625997Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:19:04.1626593Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:19:04.1627040Z parm: NVreg_RmLogonRC:int 2024-08-20T22:19:04.1627474Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:19:04.1627945Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:19:04.1628391Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:19:04.1628835Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:19:04.1629366Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:19:04.1629944Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:19:04.1630428Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:19:04.1630866Z parm: NVreg_EnableMSI:int 2024-08-20T22:19:04.1631284Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:19:04.1631750Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:19:04.1632287Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:19:04.1632873Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:19:04.1633449Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:19:04.1634061Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:19:04.1634673Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:19:04.1635288Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:19:04.1635939Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:19:04.1636440Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:19:04.1636983Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:19:04.1637530Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:19:04.1638041Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:19:04.1638514Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:19:04.1639011Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:19:04.1639484Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:19:04.1640039Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:19:04.1640555Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:19:04.1641084Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:19:04.1641566Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:19:04.1642067Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:19:04.1642577Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:19:04.1643067Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:19:04.1643566Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:19:04.1644051Z parm: NVreg_RmMsg:charp 2024-08-20T22:19:04.1644469Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:19:04.1644948Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:19:04.1645418Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:19:04.1645885Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:19:04.1646377Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:19:04.1646853Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:19:04.1647313Z parm: rm_firmware_active:charp 2024-08-20T22:19:04.1647732Z + set +e 2024-08-20T22:19:04.1648056Z + nvidia-smi 2024-08-20T22:19:05.7141736Z Tue Aug 20 22:19:05 2024 2024-08-20T22:19:05.7143066Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:19:05.7143955Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:19:05.7144760Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:19:05.7145565Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:19:05.7146475Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:19:05.7147169Z | | | MIG M. | 2024-08-20T22:19:05.7147740Z |=========================================+========================+======================| 2024-08-20T22:19:05.7245655Z | 0 NVIDIA A10G Off | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:19:05.7246527Z | 0% 29C P0 66W / 300W | 0MiB / 23028MiB | 5% Default | 2024-08-20T22:19:05.7247183Z | | | N/A | 2024-08-20T22:19:05.7248016Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:19:05.7248781Z 2024-08-20T22:19:05.7249463Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:19:05.7250089Z | Processes: | 2024-08-20T22:19:05.7250808Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:19:05.7251523Z | ID ID Usage | 2024-08-20T22:19:05.7252105Z |=========================================================================================| 2024-08-20T22:19:05.7253026Z | No running processes found | 2024-08-20T22:19:05.7253812Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:19:06.3420733Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-20T22:19:07.9096155Z NVIDIA A10G 2024-08-20T22:19:08.3542788Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:19:08.3543413Z + '[' 0 -eq 0 ']' 2024-08-20T22:19:08.3543842Z + echo 'INFO: Ignoring allowed status 0' 2024-08-20T22:19:08.3544288Z + set -e 2024-08-20T22:19:08.3544600Z INFO: Ignoring allowed status 0 2024-08-20T22:19:08.3554004Z == Installing nvidia container toolkit for amzn2023 == 2024-08-20T22:19:08.3558215Z + sudo yum install -y yum-utils 2024-08-20T22:19:08.8019911Z Last metadata expiration check: 0:09:29 ago on Tue Aug 20 22:09:39 2024. 2024-08-20T22:19:08.8241948Z Package dnf-utils-4.3.0-13.amzn2023.0.4.noarch is already installed. 2024-08-20T22:19:08.8546484Z Dependencies resolved. 2024-08-20T22:19:08.8675337Z Nothing to do. 2024-08-20T22:19:08.8675717Z Complete! 2024-08-20T22:19:08.9630232Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2024-08-20T22:19:08.9631457Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:19:08.9633381Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:19:09.2236021Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:19:09.2961764Z + sudo yum install -y nvidia-docker2 2024-08-20T22:19:09.7633368Z nvidia-container-toolkit 10 kB/s | 833 B 00:00 2024-08-20T22:19:09.7856512Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2024-08-20T22:19:09.8160346Z Dependencies resolved. 2024-08-20T22:19:09.8285551Z Nothing to do. 2024-08-20T22:19:09.8285983Z Complete! 2024-08-20T22:19:09.9202147Z + sudo systemctl restart docker 2024-08-20T22:20:05.4111524Z Tue Aug 20 22:20:05 2024 2024-08-20T22:20:05.4112406Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:20:05.4113325Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:20:05.4114321Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:20:05.4115393Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:20:05.4116352Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:20:05.4117072Z | | | MIG M. | 2024-08-20T22:20:05.4117636Z |=========================================+========================+======================| 2024-08-20T22:20:05.4259785Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:20:05.4261784Z | 0% 29C P0 58W / 300W | 0MiB / 23028MiB | 5% Default | 2024-08-20T22:20:05.4262917Z | | | N/A | 2024-08-20T22:20:05.4264022Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:20:05.4264725Z 2024-08-20T22:20:05.4265408Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:20:05.4266067Z | Processes: | 2024-08-20T22:20:05.4266784Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:20:05.4267479Z | ID ID Usage | 2024-08-20T22:20:05.4268054Z |=========================================================================================| 2024-08-20T22:20:05.4268718Z | No running processes found | 2024-08-20T22:20:05.4269493Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:20:06.1665535Z Command completed after 1 attempt(s). 2024-08-20T22:20:06.1747450Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:20:06.1748232Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:20:06.1748936Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-20T22:20:06.1749594Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:20:06.1764897Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:06.1765393Z env: 2024-08-20T22:20:06.1765678Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:06.1766153Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:06.1766633Z ##[endgroup] 2024-08-20T22:20:06.4611576Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:20:06.8743936Z Collecting psutil==5.9.1 2024-08-20T22:20:06.9134298Z Downloading psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB) 2024-08-20T22:20:06.9702498Z Collecting nvidia-ml-py==11.525.84 2024-08-20T22:20:06.9743106Z Downloading nvidia_ml_py-11.525.84-py3-none-any.whl (34 kB) 2024-08-20T22:20:07.0566221Z Installing collected packages: psutil, nvidia-ml-py 2024-08-20T22:20:07.2128814Z Successfully installed nvidia-ml-py-11.525.84 psutil-5.9.1 2024-08-20T22:20:07.4123227Z Prepare all required actions 2024-08-20T22:20:07.4124012Z Getting action download info 2024-08-20T22:20:07.5040287Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-20T22:20:07.6904256Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-20T22:20:07.8127176Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-20T22:20:07.8127665Z with: 2024-08-20T22:20:07.8128010Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:20:07.8128466Z s3-bucket: gha-artifacts 2024-08-20T22:20:07.8128815Z env: 2024-08-20T22:20:07.8129098Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:07.8129551Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:07.8130029Z ##[endgroup] 2024-08-20T22:20:07.8164491Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:20:07.8164954Z with: 2024-08-20T22:20:07.8165328Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:20:07.8165789Z s3-bucket: gha-artifacts 2024-08-20T22:20:07.8166166Z region: us-east-1 2024-08-20T22:20:07.8166482Z env: 2024-08-20T22:20:07.8166767Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:07.8167242Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:07.8167756Z ##[endgroup] 2024-08-20T22:20:08.3044269Z (node:52532) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:20:08.3044960Z 2024-08-20T22:20:08.3045221Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:20:08.3045953Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:20:08.3046844Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:20:08.3887863Z Found 1 objects with prefix pytorch/pytorch/10479310725/linux-focal-cuda12.1-py3.10-gcc9-sm86/ 2024-08-20T22:20:08.3888939Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:20:25.0739173Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:20:25.0745754Z Artifact download has finished successfully 2024-08-20T22:20:25.1099567Z ##[group]Run unzip -o artifacts.zip 2024-08-20T22:20:25.1100040Z unzip -o artifacts.zip 2024-08-20T22:20:25.1108874Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:25.1109381Z env: 2024-08-20T22:20:25.1109682Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:25.1110155Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:25.1110657Z ##[endgroup] 2024-08-20T22:20:25.1179172Z Archive: artifacts.zip 2024-08-20T22:20:25.1180452Z creating: dist/ 2024-08-20T22:20:27.2487518Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:20:27.2488201Z creating: build/custom_test_artifacts/ 2024-08-20T22:20:27.2488812Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-20T22:20:27.2489570Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-20T22:20:27.2490427Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:20:27.2497276Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:20:27.2498327Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-20T22:20:27.2499290Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:20:27.2500299Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:20:27.2501288Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:20:27.2502422Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:20:27.2504074Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:20:27.2505115Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:20:27.2506125Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:20:27.2507617Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:20:27.2509199Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:20:27.2511111Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:20:27.2512271Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:20:27.2513843Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:20:27.2515328Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:20:27.2516385Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:20:27.2517401Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:20:27.2557480Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:20:27.2597158Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:20:27.2598621Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:20:27.2643857Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:20:27.2645291Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:20:27.2646738Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:20:27.2648219Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:20:27.2649701Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:20:27.2651094Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:20:27.2652502Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:20:27.2654067Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:20:27.2655513Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:20:27.2656813Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:20:27.2658066Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:20:27.2659304Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:20:27.2660550Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:20:27.2661763Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:20:27.2662992Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:20:27.2720134Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:20:27.2780351Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:20:27.2781553Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:20:27.2782904Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:20:27.2783794Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:20:27.2784702Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-20T22:20:27.2785640Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-20T22:20:27.2786677Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-20T22:20:27.2787853Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-20T22:20:27.2788960Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-20T22:20:27.2790059Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-20T22:20:27.2791161Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-20T22:20:27.2792257Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-20T22:20:27.2793353Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-20T22:20:27.2794431Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-20T22:20:27.2795509Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-20T22:20:27.2812549Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-20T22:20:27.2956894Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-20T22:20:27.2958729Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-20T22:20:27.2960139Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-20T22:20:27.2961359Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-20T22:20:27.2962533Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-20T22:20:27.2963634Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-20T22:20:27.2964766Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-20T22:20:27.2965928Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-20T22:20:27.2967100Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-20T22:20:27.2968236Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-20T22:20:27.2969426Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-20T22:20:27.2986083Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-20T22:20:27.3070321Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-20T22:20:27.3071525Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:20:27.3072623Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:20:27.3073788Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-20T22:20:27.3074714Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-20T22:20:27.3075639Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-20T22:20:27.3076678Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-20T22:20:27.3078824Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-20T22:20:27.3079636Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-20T22:20:27.3080457Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-20T22:20:27.3201272Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-20T22:20:27.3264451Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-20T22:20:27.3265418Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-20T22:20:27.3266148Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-20T22:20:27.3266991Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:20:27.3273629Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:20:27.3274824Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-20T22:20:27.3275851Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:20:27.3276841Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:20:27.3277910Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:20:27.3279029Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:20:27.3280314Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:20:27.3281373Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:20:27.3282366Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:20:27.3283612Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:20:27.3285094Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:20:27.3286957Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:20:27.3288187Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:20:27.3289761Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:20:27.3291131Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:20:27.3292275Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:20:27.3293284Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:20:27.3332902Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:20:27.3372933Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:20:27.3374493Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:20:27.3419685Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:20:27.3421181Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:20:27.3422927Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:20:27.3424488Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:20:27.3426435Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:20:27.3428211Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:20:27.3430065Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:20:27.3431818Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:20:27.3433484Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:20:27.3434961Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:20:27.3436218Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:20:27.3437464Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:20:27.3438705Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:20:27.3439910Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:20:27.3441127Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:20:27.3495064Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:20:27.3555431Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:20:27.3557014Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:20:27.3558333Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:20:27.3559489Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:20:27.3560695Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-20T22:20:27.3561950Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-20T22:20:27.3563297Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-20T22:20:27.3564881Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-20T22:20:27.3566340Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-20T22:20:27.3567448Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-20T22:20:27.3568568Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-20T22:20:27.3569694Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-20T22:20:27.3570813Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-20T22:20:27.3571935Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-20T22:20:27.3573049Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-20T22:20:27.3587683Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-20T22:20:27.3651976Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-20T22:20:27.3653784Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:20:27.3655284Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:20:27.3656796Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-20T22:20:27.3657859Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-20T22:20:27.3658778Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-20T22:20:27.3659848Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-20T22:20:27.3661004Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-20T22:20:27.3662073Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-20T22:20:27.3662922Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-20T22:20:27.3712077Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-20T22:20:27.3713095Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-20T22:20:27.3713906Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-20T22:20:27.3714819Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:20:27.3720864Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:20:27.3722209Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-20T22:20:27.3723378Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:20:27.3724461Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:20:27.3725817Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:20:27.3727246Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:20:27.3728652Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:20:27.3729760Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:20:27.3731140Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:20:27.3732657Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:20:27.3734134Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:20:27.3735653Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:20:27.3737165Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:20:27.3738778Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:20:27.3740197Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:20:27.3741315Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:20:27.3742405Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:20:27.3780585Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:20:27.3820050Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:20:27.3821829Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:20:27.3867356Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:20:27.3869500Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:20:27.3871511Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:20:27.3873472Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:20:27.3875302Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:20:27.3877278Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:20:27.3879242Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:20:27.3881027Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:20:27.3882751Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:20:27.3884122Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:20:27.3885452Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:20:27.3886746Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:20:27.3888060Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:20:27.3889347Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:20:27.3890663Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:20:27.3942462Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:20:27.4004164Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:20:27.4007201Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:20:27.4009678Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:20:27.4010791Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:20:27.4012125Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-20T22:20:27.4013493Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-20T22:20:27.4014925Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-20T22:20:27.4016222Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-20T22:20:27.4017451Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-20T22:20:27.4018913Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-20T22:20:27.4020231Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-20T22:20:27.4021611Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-20T22:20:27.4022828Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-20T22:20:27.4024029Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-20T22:20:27.4025352Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-20T22:20:27.4026621Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-20T22:20:27.4144990Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-20T22:20:27.4146592Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-20T22:20:27.4148207Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-20T22:20:27.4149927Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-20T22:20:27.4151653Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-20T22:20:27.4153370Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-20T22:20:27.4154652Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-20T22:20:27.4155918Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-20T22:20:27.4157195Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-20T22:20:27.4158449Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-20T22:20:27.4159764Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-20T22:20:27.4174881Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-20T22:20:27.4230118Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-20T22:20:27.4231843Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:20:27.4233309Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:20:27.4234534Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-20T22:20:27.4235530Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-20T22:20:27.4236508Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-20T22:20:27.4237481Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-20T22:20:27.4238760Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-20T22:20:27.4239969Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-20T22:20:27.4240976Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-20T22:20:27.4344853Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-20T22:20:27.4388078Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-20T22:20:27.4388766Z creating: build/lib/ 2024-08-20T22:20:27.4478217Z inflating: build/lib/libprotobuf-lite.a 2024-08-20T22:20:27.4933554Z inflating: build/lib/libprotobuf.a 2024-08-20T22:20:27.4943103Z inflating: build/lib/libpthreadpool.a 2024-08-20T22:20:27.4951154Z inflating: build/lib/libcpuinfo.a 2024-08-20T22:20:27.4959630Z inflating: build/lib/libcpuinfo_internals.a 2024-08-20T22:20:27.4960329Z inflating: build/lib/libclog.a 2024-08-20T22:20:27.4978910Z inflating: build/lib/libnnpack.a 2024-08-20T22:20:27.4981617Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-20T22:20:27.5046641Z inflating: build/lib/libgtest.a 2024-08-20T22:20:27.5121880Z inflating: build/lib/libbenchmark.a 2024-08-20T22:20:27.5185773Z inflating: build/lib/libasmjit.a 2024-08-20T22:20:27.5193467Z inflating: build/lib/libittnotify.a 2024-08-20T22:20:27.5221516Z inflating: build/lib/libtensorpipe_uv.a 2024-08-20T22:20:27.5349168Z inflating: build/lib/libgloo.a 2024-08-20T22:20:27.5370101Z inflating: build/lib/libfmt.a 2024-08-20T22:20:27.5465576Z inflating: build/lib/libc10.so 2024-08-20T22:20:27.5467026Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-20T22:20:27.5468630Z inflating: build/lib/libtorch_global_deps.so 2024-08-20T22:20:27.5487996Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-20T22:20:27.5991952Z inflating: build/lib/libprotoc.a 2024-08-20T22:20:27.6009917Z inflating: build/lib/libgmock.a 2024-08-20T22:20:27.6010503Z inflating: build/lib/libgtest_main.a 2024-08-20T22:20:27.6013071Z inflating: build/lib/libbenchmark_main.a 2024-08-20T22:20:27.6402205Z inflating: build/lib/libgloo_cuda.a 2024-08-20T22:20:28.6581188Z inflating: build/lib/libdnnl.a 2024-08-20T22:20:28.7154876Z inflating: build/lib/libtensorpipe.a 2024-08-20T22:20:28.7212634Z inflating: build/lib/libc10_cuda.so 2024-08-20T22:20:28.7213831Z inflating: build/lib/libgmock_main.a 2024-08-20T22:20:28.8474834Z inflating: build/lib/libfbgemm.a 2024-08-20T22:20:28.8731748Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-20T22:20:28.9238526Z inflating: build/lib/libkineto.a 2024-08-20T22:20:28.9423095Z inflating: build/lib/libXNNPACK.a 2024-08-20T22:20:28.9465980Z inflating: build/lib/libonnx_proto.a 2024-08-20T22:20:29.0159789Z inflating: build/lib/libonnx.a 2024-08-20T22:20:31.4615189Z inflating: build/lib/libtorch_cpu.so 2024-08-20T22:20:31.4620572Z inflating: build/lib/libunbox_lib.a 2024-08-20T22:20:31.4624670Z inflating: build/lib/libshm.so 2024-08-20T22:20:33.5270689Z inflating: build/lib/libtorch_cuda.so 2024-08-20T22:20:33.5272392Z inflating: build/lib/libtorch.so 2024-08-20T22:20:33.5275213Z inflating: build/lib/libc10d_cuda_test.so 2024-08-20T22:20:34.3474002Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-20T22:20:34.5468667Z inflating: build/lib/libtorch_python.so 2024-08-20T22:20:34.5541194Z inflating: build/lib/libtorchbind_test.so 2024-08-20T22:20:34.5562406Z inflating: build/lib/libjitbackend_test.so 2024-08-20T22:20:34.5588786Z inflating: build/lib/libaoti_custom_ops.so 2024-08-20T22:20:34.5614362Z inflating: build/lib/libbackend_with_compiler.so 2024-08-20T22:20:34.5649477Z inflating: build/lib/libnnapi_backend.so 2024-08-20T22:20:34.5650709Z creating: build/bin/ 2024-08-20T22:20:34.5700315Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-20T22:20:34.5753038Z inflating: build/bin/c10_DeviceGuard_test 2024-08-20T22:20:34.5803594Z inflating: build/bin/c10_Device_test 2024-08-20T22:20:34.5863353Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-20T22:20:34.5916949Z inflating: build/bin/c10_Scalar_test 2024-08-20T22:20:34.5966618Z inflating: build/bin/c10_StreamGuard_test 2024-08-20T22:20:34.6018857Z inflating: build/bin/c10_SymInt_test 2024-08-20T22:20:34.6073622Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-20T22:20:34.6129771Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-20T22:20:34.6186665Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-20T22:20:34.6257916Z inflating: build/bin/c10_cow_test 2024-08-20T22:20:34.6311236Z inflating: build/bin/c10_Bitset_test 2024-08-20T22:20:34.6361246Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-20T22:20:34.6410933Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-20T22:20:34.6461907Z inflating: build/bin/c10_Half_test 2024-08-20T22:20:34.6518667Z inflating: build/bin/c10_LeftRight_test 2024-08-20T22:20:34.6574017Z inflating: build/bin/c10_Metaprogramming_test 2024-08-20T22:20:34.6624346Z inflating: build/bin/c10_Synchronized_test 2024-08-20T22:20:34.6680738Z inflating: build/bin/c10_ThreadLocal_test 2024-08-20T22:20:34.6733008Z inflating: build/bin/c10_TypeIndex_test 2024-08-20T22:20:34.6784494Z inflating: build/bin/c10_TypeList_test 2024-08-20T22:20:34.6834482Z inflating: build/bin/c10_TypeTraits_test 2024-08-20T22:20:34.6886204Z inflating: build/bin/c10_accumulate_test 2024-08-20T22:20:34.6941978Z inflating: build/bin/c10_bfloat16_test 2024-08-20T22:20:34.6993525Z inflating: build/bin/c10_bit_cast_test 2024-08-20T22:20:34.7050842Z inflating: build/bin/c10_complex_math_test 2024-08-20T22:20:34.7106703Z inflating: build/bin/c10_complex_test 2024-08-20T22:20:34.7159931Z inflating: build/bin/c10_exception_test 2024-08-20T22:20:34.7210592Z inflating: build/bin/c10_flags_test 2024-08-20T22:20:34.7261073Z inflating: build/bin/c10_generic_math_test 2024-08-20T22:20:34.7425888Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-20T22:20:34.7477133Z inflating: build/bin/c10_irange_test 2024-08-20T22:20:34.7530872Z inflating: build/bin/c10_lazy_test 2024-08-20T22:20:34.7588580Z inflating: build/bin/c10_logging_test 2024-08-20T22:20:34.7651008Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-20T22:20:34.7726498Z inflating: build/bin/c10_optional_test 2024-08-20T22:20:34.7879219Z inflating: build/bin/c10_small_vector_test 2024-08-20T22:20:34.7932746Z inflating: build/bin/c10_registry_test 2024-08-20T22:20:34.7984782Z inflating: build/bin/c10_ssize_test 2024-08-20T22:20:34.8038195Z inflating: build/bin/c10_string_util_test 2024-08-20T22:20:34.8089033Z inflating: build/bin/c10_tempfile_test 2024-08-20T22:20:34.8148181Z inflating: build/bin/c10_string_view_test 2024-08-20T22:20:34.8197771Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-20T22:20:34.8254793Z inflating: build/bin/c10_typeid_test 2024-08-20T22:20:34.8702553Z inflating: build/bin/protoc-3.13.0.0 2024-08-20T22:20:34.9150296Z inflating: build/bin/protoc 2024-08-20T22:20:34.9204765Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-20T22:20:34.9257806Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-20T22:20:34.9310847Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-20T22:20:34.9363359Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-20T22:20:34.9416999Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-20T22:20:34.9470310Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-20T22:20:34.9519528Z inflating: build/bin/c10_cuda_CUDATest 2024-08-20T22:20:34.9572568Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-20T22:20:34.9904189Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-20T22:20:35.0249846Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-20T22:20:35.0610960Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-20T22:20:35.0664082Z inflating: build/bin/BackoffTest 2024-08-20T22:20:35.0717660Z inflating: build/bin/FileStoreTest 2024-08-20T22:20:35.0773882Z inflating: build/bin/TCPStoreTest 2024-08-20T22:20:35.0827318Z inflating: build/bin/HashStoreTest 2024-08-20T22:20:35.0841103Z inflating: build/bin/ProcessGroupMPITest 2024-08-20T22:20:35.0895567Z inflating: build/bin/test_edge_op_registration 2024-08-20T22:20:35.0899857Z inflating: build/bin/torch_shm_manager 2024-08-20T22:20:35.0902900Z inflating: build/bin/example_allreduce 2024-08-20T22:20:35.0958437Z inflating: build/bin/test_dist_autograd 2024-08-20T22:20:35.1027440Z inflating: build/bin/test_cpp_rpc 2024-08-20T22:20:35.1030154Z inflating: build/bin/parallel_benchmark 2024-08-20T22:20:35.1097823Z inflating: build/bin/test_mobile_nnc 2024-08-20T22:20:35.1107127Z inflating: build/bin/aot_model_compiler_test 2024-08-20T22:20:35.1454094Z inflating: build/bin/test_lazy 2024-08-20T22:20:35.2639778Z inflating: build/bin/test_api 2024-08-20T22:20:35.2714504Z inflating: build/bin/Dict_test 2024-08-20T22:20:35.2766530Z inflating: build/bin/Dimname_test 2024-08-20T22:20:35.2831440Z inflating: build/bin/MaybeOwned_test 2024-08-20T22:20:35.2890117Z inflating: build/bin/NamedTensor_test 2024-08-20T22:20:35.2948992Z inflating: build/bin/apply_utils_test 2024-08-20T22:20:35.3008739Z inflating: build/bin/atest 2024-08-20T22:20:35.3072603Z inflating: build/bin/basic 2024-08-20T22:20:35.3127897Z inflating: build/bin/broadcast_test 2024-08-20T22:20:35.3179805Z inflating: build/bin/cpu_allocator_test 2024-08-20T22:20:35.3238233Z inflating: build/bin/cpu_generator_test 2024-08-20T22:20:35.3292304Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-20T22:20:35.3385902Z inflating: build/bin/cpu_rng_test 2024-08-20T22:20:35.3436104Z inflating: build/bin/dispatch_key_set_test 2024-08-20T22:20:35.3487040Z inflating: build/bin/dlconvertor_test 2024-08-20T22:20:35.3545511Z inflating: build/bin/extension_backend_test 2024-08-20T22:20:35.3601849Z inflating: build/bin/half_test 2024-08-20T22:20:35.3698004Z inflating: build/bin/ivalue_test 2024-08-20T22:20:35.3747988Z inflating: build/bin/lazy_tensor_test 2024-08-20T22:20:35.3803761Z inflating: build/bin/math_kernel_test 2024-08-20T22:20:35.3857544Z inflating: build/bin/memory_format_test 2024-08-20T22:20:35.3910943Z inflating: build/bin/memory_overlapping_test 2024-08-20T22:20:35.3964478Z inflating: build/bin/mobile_memory_cleanup 2024-08-20T22:20:35.4020862Z inflating: build/bin/native_test 2024-08-20T22:20:35.4071834Z inflating: build/bin/operator_name_test 2024-08-20T22:20:35.4123747Z inflating: build/bin/operators_test 2024-08-20T22:20:35.4175841Z inflating: build/bin/packedtensoraccessor_test 2024-08-20T22:20:35.4243949Z inflating: build/bin/pow_test 2024-08-20T22:20:35.4301101Z inflating: build/bin/quantized_test 2024-08-20T22:20:35.4351511Z inflating: build/bin/reduce_ops_test 2024-08-20T22:20:35.4403174Z inflating: build/bin/reportMemoryUsage_test 2024-08-20T22:20:35.4460248Z inflating: build/bin/scalar_tensor_test 2024-08-20T22:20:35.4519202Z inflating: build/bin/scalar_test 2024-08-20T22:20:35.4571522Z inflating: build/bin/StorageUtils_test 2024-08-20T22:20:35.4624231Z inflating: build/bin/stride_properties_test 2024-08-20T22:20:35.4704364Z inflating: build/bin/tensor_iterator_test 2024-08-20T22:20:35.4759139Z inflating: build/bin/test_parallel 2024-08-20T22:20:35.4762344Z inflating: build/bin/thread_init_test 2024-08-20T22:20:35.4818473Z inflating: build/bin/type_ptr_test 2024-08-20T22:20:35.4878944Z inflating: build/bin/type_test 2024-08-20T22:20:35.4931873Z inflating: build/bin/undefined_tensor_test 2024-08-20T22:20:35.4932861Z inflating: build/bin/verify_api_visibility 2024-08-20T22:20:35.5003789Z inflating: build/bin/legacy_vmap_test 2024-08-20T22:20:35.5055499Z inflating: build/bin/weakref_test 2024-08-20T22:20:35.5107393Z inflating: build/bin/wrapdim_test 2024-08-20T22:20:35.5159636Z inflating: build/bin/xla_tensor_test 2024-08-20T22:20:35.5219823Z inflating: build/bin/IListRef_test 2024-08-20T22:20:35.5327420Z inflating: build/bin/List_test 2024-08-20T22:20:35.5394306Z inflating: build/bin/KernelFunction_test 2024-08-20T22:20:35.5516274Z inflating: build/bin/kernel_function_legacy_test 2024-08-20T22:20:35.5614006Z inflating: build/bin/kernel_function_test 2024-08-20T22:20:35.5743965Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-20T22:20:35.5847886Z inflating: build/bin/kernel_lambda_test 2024-08-20T22:20:35.5909688Z inflating: build/bin/kernel_stackbased_test 2024-08-20T22:20:35.6006759Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-20T22:20:35.6058152Z inflating: build/bin/CppSignature_test 2024-08-20T22:20:35.6114437Z inflating: build/bin/backend_fallback_test 2024-08-20T22:20:35.6163670Z inflating: build/bin/op_allowlist_test 2024-08-20T22:20:35.6467729Z inflating: build/bin/op_registration_test 2024-08-20T22:20:35.6532116Z inflating: build/bin/inline_container_test 2024-08-20T22:20:35.6585278Z inflating: build/bin/cuda_apply_test 2024-08-20T22:20:35.6637564Z inflating: build/bin/cuda_allocator_test 2024-08-20T22:20:35.6697338Z inflating: build/bin/cuda_atomic_ops_test 2024-08-20T22:20:35.6752423Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-20T22:20:35.6822994Z inflating: build/bin/cuda_complex_math_test 2024-08-20T22:20:35.6882900Z inflating: build/bin/cuda_complex_test 2024-08-20T22:20:35.6933011Z inflating: build/bin/cuda_device_test 2024-08-20T22:20:35.6991085Z inflating: build/bin/cuda_cub_test 2024-08-20T22:20:35.7042323Z inflating: build/bin/cuda_dlconvertor_test 2024-08-20T22:20:35.7107531Z inflating: build/bin/cuda_distributions_test 2024-08-20T22:20:35.7164791Z inflating: build/bin/cuda_generator_test 2024-08-20T22:20:35.7214948Z inflating: build/bin/cuda_half_test 2024-08-20T22:20:35.7266987Z inflating: build/bin/cuda_integer_divider_test 2024-08-20T22:20:35.7316604Z inflating: build/bin/cuda_optional_test 2024-08-20T22:20:35.7368701Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-20T22:20:35.7422324Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-20T22:20:35.7472307Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-20T22:20:35.7533269Z inflating: build/bin/cuda_stream_test 2024-08-20T22:20:35.7583132Z inflating: build/bin/cuda_cudnn_test 2024-08-20T22:20:35.7635511Z inflating: build/bin/cuda_vectorized_test 2024-08-20T22:20:35.7650076Z inflating: build/bin/tutorial_tensorexpr 2024-08-20T22:20:35.7717012Z inflating: build/bin/ProcessGroupGlooTest 2024-08-20T22:20:35.7774278Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-20T22:20:35.7838422Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-20T22:20:35.7901111Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-20T22:20:35.8730955Z inflating: build/bin/test_tensorexpr 2024-08-20T22:20:35.9314367Z inflating: build/bin/test_jit 2024-08-20T22:20:35.9314958Z creating: .additional_ci_files/ 2024-08-20T22:20:35.9375826Z inflating: .additional_ci_files/test-times.json 2024-08-20T22:20:35.9615901Z inflating: .additional_ci_files/test-class-times.json 2024-08-20T22:20:35.9651347Z ##[group]Run rm artifacts.zip 2024-08-20T22:20:35.9651759Z rm artifacts.zip 2024-08-20T22:20:35.9661095Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:35.9661633Z env: 2024-08-20T22:20:35.9661954Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:35.9662424Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:35.9662920Z ##[endgroup] 2024-08-20T22:20:36.0901384Z ##[group]Run df -H 2024-08-20T22:20:36.0901712Z df -H 2024-08-20T22:20:36.0910322Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:36.0910829Z env: 2024-08-20T22:20:36.0911126Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.0911631Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.0912268Z ##[endgroup] 2024-08-20T22:20:36.0962286Z Filesystem Size Used Avail Use% Mounted on 2024-08-20T22:20:36.0963016Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-20T22:20:36.0963533Z tmpfs 34G 0 34G 0% /dev/shm 2024-08-20T22:20:36.0964172Z tmpfs 14G 553k 14G 1% /run 2024-08-20T22:20:36.0964797Z /dev/nvme0n1p1 161G 53G 109G 33% / 2024-08-20T22:20:36.0965913Z tmpfs 34G 4.1k 34G 1% /tmp 2024-08-20T22:20:36.0966502Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-20T22:20:36.0967142Z tmpfs 6.7G 0 6.7G 0% /run/user/0 2024-08-20T22:20:36.1005907Z Prepare all required actions 2024-08-20T22:20:36.1006530Z Getting action download info 2024-08-20T22:20:36.2121193Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-20T22:20:36.2121680Z with: 2024-08-20T22:20:36.2121941Z env: 2024-08-20T22:20:36.2122223Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.2122684Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.2123162Z ##[endgroup] 2024-08-20T22:20:36.2156944Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:20:36.2157406Z with: 2024-08-20T22:20:36.2157689Z name: td_results 2024-08-20T22:20:36.2158016Z s3-bucket: gha-artifacts 2024-08-20T22:20:36.2158374Z region: us-east-1 2024-08-20T22:20:36.2158683Z env: 2024-08-20T22:20:36.2158965Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.2159419Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.2159910Z ##[endgroup] 2024-08-20T22:20:36.6875495Z (node:52557) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:20:36.6876269Z 2024-08-20T22:20:36.6876552Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:20:36.6877284Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:20:36.6878152Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:20:36.7429953Z Found 0 objects with prefix pytorch/pytorch/10479310725/td_results/ 2024-08-20T22:20:36.7436222Z Artifact download has finished successfully 2024-08-20T22:20:36.7756634Z ##[group]Run mkdir -p .additional_ci_files 2024-08-20T22:20:36.7757125Z mkdir -p .additional_ci_files 2024-08-20T22:20:36.7757700Z mv td_results.json .additional_ci_files/td_results.json 2024-08-20T22:20:36.7766736Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:36.7767241Z env: 2024-08-20T22:20:36.7767528Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.7768008Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.7768499Z ##[endgroup] 2024-08-20T22:20:36.7827348Z mv: cannot stat 'td_results.json': No such file or directory 2024-08-20T22:20:36.7842847Z ##[error]Process completed with exit code 1. 2024-08-20T22:20:36.7877618Z ##[group]Run .github/scripts/parse_ref.py 2024-08-20T22:20:36.7878099Z .github/scripts/parse_ref.py 2024-08-20T22:20:36.7886300Z shell: /usr/bin/bash -e {0} 2024-08-20T22:20:36.7886659Z env: 2024-08-20T22:20:36.7886941Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.7887401Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.7887884Z ##[endgroup] 2024-08-20T22:20:36.8172584Z Prepare all required actions 2024-08-20T22:20:36.8224493Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-20T22:20:36.8224950Z with: 2024-08-20T22:20:36.8225531Z github-token: *** 2024-08-20T22:20:36.8225860Z env: 2024-08-20T22:20:36.8226156Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.8226615Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.8227113Z ##[endgroup] 2024-08-20T22:20:36.8244020Z ##[group]Run set -eux 2024-08-20T22:20:36.8244400Z set -eux 2024-08-20T22:20:36.8245010Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-20T22:20:36.8254645Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:36.8255147Z env: 2024-08-20T22:20:36.8255430Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.8255893Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.8256534Z GITHUB_TOKEN: *** 2024-08-20T22:20:36.8256870Z ##[endgroup] 2024-08-20T22:20:36.8284520Z + python3 .github/scripts/get_workflow_job_id.py 10479310725 i-021b0327168a01512 2024-08-20T22:20:38.3965994Z setting job-id=29026528125 2024-08-20T22:20:38.3966884Z setting job-name=cuda12.1-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:20:38.4188652Z Prepare all required actions 2024-08-20T22:20:38.4189106Z Getting action download info 2024-08-20T22:20:38.5326125Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-20T22:20:38.5326590Z with: 2024-08-20T22:20:38.5327049Z github-token: *** 2024-08-20T22:20:38.5336035Z test-matrix: {"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:20:38.5345552Z job-name: cuda12.1-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:20:38.5346273Z env: 2024-08-20T22:20:38.5346560Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:38.5347014Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:38.5347505Z ##[endgroup] 2024-08-20T22:20:38.5390237Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:20:38.5390805Z with: 2024-08-20T22:20:38.5391113Z shell: bash 2024-08-20T22:20:38.5391471Z timeout_minutes: 10 2024-08-20T22:20:38.5391835Z max_attempts: 5 2024-08-20T22:20:38.5392188Z retry_wait_seconds: 30 2024-08-20T22:20:38.5393316Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:20:38.5394486Z polling_interval_seconds: 1 2024-08-20T22:20:38.5394884Z warning_on_retry: true 2024-08-20T22:20:38.5395239Z continue_on_error: false 2024-08-20T22:20:38.5395593Z env: 2024-08-20T22:20:38.5395880Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:38.5396337Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:38.5396984Z GITHUB_TOKEN: *** 2024-08-20T22:20:38.5397317Z ##[endgroup] 2024-08-20T22:20:38.6158071Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:20:38.8523261Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:20:38.9995236Z Collecting requests==2.27.1 2024-08-20T22:20:39.0372180Z Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB) 2024-08-20T22:20:39.2287033Z Collecting pyyaml==6.0.1 2024-08-20T22:20:39.2335107Z Downloading PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (738 kB) 2024-08-20T22:20:39.2757269Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-20T22:20:39.3213662Z Collecting certifi>=2017.4.17 2024-08-20T22:20:39.3258574Z Downloading certifi-2024.7.4-py3-none-any.whl (162 kB) 2024-08-20T22:20:39.3327179Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-20T22:20:39.6286788Z Collecting charset-normalizer~=2.0.0 2024-08-20T22:20:39.6336532Z Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 2024-08-20T22:20:39.7127812Z Installing collected packages: charset-normalizer, certifi, requests, pyyaml 2024-08-20T22:20:39.8274166Z Successfully installed certifi-2024.7.4 charset-normalizer-2.0.12 pyyaml-6.0.1 requests-2.27.1 2024-08-20T22:20:40.5936664Z Command completed after 1 attempt(s). 2024-08-20T22:20:40.5996010Z ##[group]Run set -x 2024-08-20T22:20:40.5996358Z set -x 2024-08-20T22:20:40.5996667Z  2024-08-20T22:20:40.5997224Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:20:40.5997896Z # in runner workspace 2024-08-20T22:20:40.5998439Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-20T22:20:40.6007481Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:40.6007993Z env: 2024-08-20T22:20:40.6008278Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:40.6008744Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:40.6009231Z ##[endgroup] 2024-08-20T22:20:40.6040613Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-20T22:20:40.6279452Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:20:40.6280021Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:20:40.6280516Z echo "Job name: ${JOB_NAME}" 2024-08-20T22:20:40.6280929Z  2024-08-20T22:20:40.6281494Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:20:40.6282187Z # in runner workspace 2024-08-20T22:20:40.6282777Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-20T22:20:40.6283471Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-20T22:20:40.6283940Z  --job-name "${JOB_NAME}" \ 2024-08-20T22:20:40.6293003Z  --test-matrix "{"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-08-20T22:20:40.6302435Z  --selected-test-configs "" \ 2024-08-20T22:20:40.6302938Z  --pr-number "${PR_NUMBER}" \ 2024-08-20T22:20:40.6303432Z  --tag "${TAG}" \ 2024-08-20T22:20:40.6303849Z  --event-name "${EVENT_NAME}" \ 2024-08-20T22:20:40.6304315Z  --schedule "${SCHEDULE}" \ 2024-08-20T22:20:40.6304764Z  --branch "${HEAD_BRANCH}" 2024-08-20T22:20:40.6313653Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:40.6314168Z env: 2024-08-20T22:20:40.6314455Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:40.6314928Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:40.6315623Z GITHUB_TOKEN: *** 2024-08-20T22:20:40.6316290Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:20:40.6317030Z PR_NUMBER: 2024-08-20T22:20:40.6317356Z TAG: ciflow/inductor/133712 2024-08-20T22:20:40.6317737Z EVENT_NAME: push 2024-08-20T22:20:40.6318068Z SCHEDULE: 2024-08-20T22:20:40.6318365Z HEAD_BRANCH: 2024-08-20T22:20:40.6318672Z ##[endgroup] 2024-08-20T22:20:40.6348728Z Workflow: inductor 2024-08-20T22:20:40.6349706Z Job name: cuda12.1-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:20:40.8708813Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-20T22:20:41.0403451Z ##[group]Run echo "Filtered matrix:" 2024-08-20T22:20:41.0403918Z echo "Filtered matrix:" 2024-08-20T22:20:41.0412925Z echo "{"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]}" 2024-08-20T22:20:41.0423403Z  2024-08-20T22:20:41.0423693Z echo 2024-08-20T22:20:41.0424085Z echo "Is the current job unstable? False" 2024-08-20T22:20:41.0424550Z  2024-08-20T22:20:41.0424832Z echo 2024-08-20T22:20:41.0425207Z echo "Is keep-going label set? False" 2024-08-20T22:20:41.0425658Z  2024-08-20T22:20:41.0425932Z echo 2024-08-20T22:20:41.0426274Z echo "Renabled issues? " 2024-08-20T22:20:41.0435077Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:41.0435584Z env: 2024-08-20T22:20:41.0435872Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:41.0436335Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:41.0436824Z ##[endgroup] 2024-08-20T22:20:41.0466534Z Filtered matrix: 2024-08-20T22:20:41.0477261Z {include: [{config: inductor, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_distributed, shard: 1, num_shards: 1, runner: amz2023.linux.g5.12xlarge.nvidia.gpu}, {config: inductor_huggingface, shard: 1, num_shards: 1, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_torchbench, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_torchbench, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_huggingface, shard: 1, num_shards: 1, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_timm, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_timm, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_torchbench, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_torchbench, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_huggingface, shard: 1, num_shards: 1, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_timm, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_timm, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_torchbench, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_torchbench, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_cpp_wrapper_abi_compatible, shard: 1, num_shards: 1, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}]} 2024-08-20T22:20:41.0486140Z 2024-08-20T22:20:41.0486302Z Is the current job unstable? False 2024-08-20T22:20:41.0486591Z 2024-08-20T22:20:41.0486954Z Is keep-going label set? False 2024-08-20T22:20:41.0487213Z 2024-08-20T22:20:41.0487345Z Renabled issues? 2024-08-20T22:20:41.0532344Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:20:41.0533064Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:20:41.0541966Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:41.0542479Z env: 2024-08-20T22:20:41.0542777Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:41.0543250Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:41.0543748Z JOB_TIMEOUT: 240 2024-08-20T22:20:41.0544073Z ##[endgroup] 2024-08-20T22:20:41.0640843Z ##[group]Run set -x 2024-08-20T22:20:41.0641241Z set -x 2024-08-20T22:20:41.0641543Z  2024-08-20T22:20:41.0641903Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-20T22:20:41.0642442Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-20T22:20:41.0643010Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-20T22:20:41.0643524Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-20T22:20:41.0643945Z else 2024-08-20T22:20:41.0644287Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:20:41.0644709Z fi 2024-08-20T22:20:41.0644990Z  2024-08-20T22:20:41.0645456Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-20T22:20:41.0646205Z # TODO: Stop building test binaries as part of the build phase 2024-08-20T22:20:41.0646874Z # Used for GPU_FLAG since that doesn't play nice 2024-08-20T22:20:41.0647447Z # shellcheck disable=SC2086,SC2090 2024-08-20T22:20:41.0647924Z container_name=$(docker run \ 2024-08-20T22:20:41.0648340Z  ${GPU_FLAG:-} \ 2024-08-20T22:20:41.0648717Z  -e BUILD_ENVIRONMENT \ 2024-08-20T22:20:41.0649118Z  -e PR_NUMBER \ 2024-08-20T22:20:41.0649482Z  -e GITHUB_ACTIONS \ 2024-08-20T22:20:41.0649878Z  -e GITHUB_REPOSITORY \ 2024-08-20T22:20:41.0650291Z  -e GITHUB_WORKFLOW \ 2024-08-20T22:20:41.0650677Z  -e GITHUB_JOB \ 2024-08-20T22:20:41.0651038Z  -e GITHUB_RUN_ID \ 2024-08-20T22:20:41.0651424Z  -e GITHUB_RUN_NUMBER \ 2024-08-20T22:20:41.0651832Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-20T22:20:41.0652228Z  -e JOB_ID \ 2024-08-20T22:20:41.0652568Z  -e JOB_NAME \ 2024-08-20T22:20:41.0653213Z  -e BASE_SHA \ 2024-08-20T22:20:41.0653562Z  -e BRANCH \ 2024-08-20T22:20:41.0653942Z  -e SHA1 \ 2024-08-20T22:20:41.0654367Z  -e AWS_DEFAULT_REGION \ 2024-08-20T22:20:41.0654780Z  -e IN_WHEEL_TEST \ 2024-08-20T22:20:41.0655163Z  -e SHARD_NUMBER \ 2024-08-20T22:20:41.0655533Z  -e TEST_CONFIG \ 2024-08-20T22:20:41.0655907Z  -e NUM_TEST_SHARDS \ 2024-08-20T22:20:41.0656302Z  -e REENABLED_ISSUES \ 2024-08-20T22:20:41.0656719Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-20T22:20:41.0657139Z  -e VERBOSE_TEST_LOGS \ 2024-08-20T22:20:41.0657544Z  -e TEST_SHOWLOCALS \ 2024-08-20T22:20:41.0657937Z  -e NO_TEST_TIMEOUT \ 2024-08-20T22:20:41.0658312Z  -e NO_TD \ 2024-08-20T22:20:41.0658655Z  -e TD_DISTRIBUTED \ 2024-08-20T22:20:41.0659038Z  -e PR_LABELS \ 2024-08-20T22:20:41.0659457Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-20T22:20:41.0659921Z  -e SCCACHE_BUCKET \ 2024-08-20T22:20:41.0660331Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-20T22:20:41.0660746Z  -e XLA_CUDA \ 2024-08-20T22:20:41.0661165Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-20T22:20:41.0661669Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-20T22:20:41.0662200Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-20T22:20:41.0662719Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-20T22:20:41.0663191Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-20T22:20:41.0663649Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-20T22:20:41.0664095Z  -e DASHBOARD_TAG \ 2024-08-20T22:20:41.0664577Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-20T22:20:41.0665140Z  --security-opt seccomp=unconfined \ 2024-08-20T22:20:41.0665614Z  --cap-add=SYS_PTRACE \ 2024-08-20T22:20:41.0666014Z  --ipc=host \ 2024-08-20T22:20:41.0666381Z  --shm-size="${SHM_SIZE}" \ 2024-08-20T22:20:41.0666782Z  --tty \ 2024-08-20T22:20:41.0667097Z  --detach \ 2024-08-20T22:20:41.0667643Z  --name="${container_name}" \ 2024-08-20T22:20:41.0668219Z  --user jenkins \ 2024-08-20T22:20:41.0668707Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-20T22:20:41.0669266Z  -w /var/lib/jenkins/workspace \ 2024-08-20T22:20:41.0669709Z  "${DOCKER_IMAGE}" 2024-08-20T22:20:41.0670065Z ) 2024-08-20T22:20:41.0670477Z # Propagate download.pytorch.org IP to container 2024-08-20T22:20:41.0671394Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-20T22:20:41.0672351Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-20T22:20:41.0673253Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-20T22:20:41.0683239Z shell: /usr/bin/bash -e {0} 2024-08-20T22:20:41.0683619Z env: 2024-08-20T22:20:41.0683915Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:41.0684391Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:41.0685013Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:20:41.0685516Z PR_NUMBER: 2024-08-20T22:20:41.0685855Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-20T22:20:41.0686282Z GITHUB_WORKFLOW: inductor 2024-08-20T22:20:41.0686643Z GITHUB_JOB: test 2024-08-20T22:20:41.0686975Z GITHUB_RUN_ID: 10479310725 2024-08-20T22:20:41.0687345Z GITHUB_RUN_NUMBER: 88082 2024-08-20T22:20:41.0687712Z GITHUB_RUN_ATTEMPT: 1 2024-08-20T22:20:41.0688057Z JOB_ID: 29026528125 2024-08-20T22:20:41.0688723Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:20:41.0689448Z BRANCH: 2024-08-20T22:20:41.0689800Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:20:41.0690335Z BASE_SHA: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:20:41.0690813Z TEST_CONFIG: inductor_timm 2024-08-20T22:20:41.0691181Z SHARD_NUMBER: 2 2024-08-20T22:20:41.0691508Z NUM_TEST_SHARDS: 2 2024-08-20T22:20:41.0691832Z REENABLED_ISSUES: 2024-08-20T22:20:41.0692189Z CONTINUE_THROUGH_ERROR: False 2024-08-20T22:20:41.0692587Z VERBOSE_TEST_LOGS: False 2024-08-20T22:20:41.0692958Z TEST_SHOWLOCALS: False 2024-08-20T22:20:41.0693308Z NO_TEST_TIMEOUT: False 2024-08-20T22:20:41.0693662Z NO_TD: False 2024-08-20T22:20:41.0693976Z TD_DISTRIBUTED: False 2024-08-20T22:20:41.0694453Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-20T22:20:41.0694950Z SCCACHE_S3_KEY_PREFIX: inductor 2024-08-20T22:20:41.0695337Z SHM_SIZE: 2g 2024-08-20T22:20:41.0696430Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:20:41.0697593Z XLA_CUDA: 2024-08-20T22:20:41.0698073Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:20:41.0698693Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-20T22:20:41.0699150Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-20T22:20:41.0699570Z DASHBOARD_TAG: 2024-08-20T22:20:41.0700087Z HUGGING_FACE_HUB_TOKEN: *** 2024-08-20T22:20:41.0700471Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-20T22:20:41.0700856Z ##[endgroup] 2024-08-20T22:20:41.0730781Z + [[ inductor_timm == \m\u\l\t\i\g\p\u ]] 2024-08-20T22:20:41.0731707Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]] 2024-08-20T22:20:41.0732240Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:20:41.0740112Z +++ nproc --ignore=2 2024-08-20T22:20:41.0771194Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479310725 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:20:52.4432880Z + container_name=ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T22:20:52.4434277Z + grep download.pytorch.org /etc/hosts 2024-08-20T22:20:52.4436477Z + docker exec -i ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-20T22:20:52.5517372Z + echo DOCKER_CONTAINER_ID=ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T22:20:52.5521209Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:20:52.5524159Z + docker exec -t ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-20T22:20:52.9472213Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-20T22:20:53.2723615Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-20T22:20:53.2727116Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-20T22:20:53.2729247Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-20T22:20:53.2731443Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-20T22:20:53.2733851Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-20T22:20:53.2739289Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.13.1) 2024-08-20T22:20:53.2755782Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-20T22:20:53.2770194Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-20T22:20:53.2787364Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.22.4) 2024-08-20T22:20:53.3161141Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-20T22:20:53.8819862Z Installing collected packages: torch 2024-08-20T22:21:03.8730659Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T22:21:03.8732023Z timm 1.0.9.dev0 requires torchvision, which is not installed. 2024-08-20T22:21:03.8733638Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-20T22:21:03.9563004Z + export TERM=vt100 2024-08-20T22:21:03.9563359Z + TERM=vt100 2024-08-20T22:21:03.9566647Z ++ dirname .ci/pytorch/test.sh 2024-08-20T22:21:03.9577956Z + source .ci/pytorch/common.sh 2024-08-20T22:21:03.9581749Z +++ dirname .ci/pytorch/common.sh 2024-08-20T22:21:03.9591550Z ++ source .ci/pytorch/common_utils.sh 2024-08-20T22:21:03.9593810Z +++ declare -f -t trap_add 2024-08-20T22:21:03.9599066Z ++ set -ex 2024-08-20T22:21:03.9599641Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:21:03.9600153Z ++ BUILD_TEST_LIBTORCH=0 2024-08-20T22:21:03.9601276Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]] 2024-08-20T22:21:03.9604493Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-20T22:21:03.9625292Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-20T22:21:03.9625732Z + trap_add cleanup_workspace EXIT 2024-08-20T22:21:03.9626155Z + trap_add_cmd=cleanup_workspace 2024-08-20T22:21:03.9626526Z + shift 2024-08-20T22:21:03.9629783Z + for trap_add_name in "$@" 2024-08-20T22:21:03.9633296Z +++ trap -p EXIT 2024-08-20T22:21:03.9637083Z ++ eval 'extract_trap_cmd ' 2024-08-20T22:21:03.9637472Z +++ extract_trap_cmd 2024-08-20T22:21:03.9637849Z +++ printf '%s\n' '' 2024-08-20T22:21:03.9638260Z ++ printf '%s\n' cleanup_workspace 2024-08-20T22:21:03.9640611Z + trap -- ' 2024-08-20T22:21:03.9641060Z cleanup_workspace' EXIT 2024-08-20T22:21:03.9641547Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-20T22:21:04.6797159Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-20T22:21:04.6819571Z + echo 'Environment variables:' 2024-08-20T22:21:04.6819985Z Environment variables: 2024-08-20T22:21:04.6820321Z + env 2024-08-20T22:21:04.6832056Z INSTALLED_DB=yes 2024-08-20T22:21:04.6833287Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-20T22:21:04.6833764Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:21:04.6834327Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-20T22:21:04.6835192Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:21:04.6835973Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:21:04.6836665Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-20T22:21:04.6837211Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:21:04.6837785Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:21:04.6838367Z HOSTNAME=ff7d2001e8e4 2024-08-20T22:21:04.6839499Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_ef369113-cdd6-42e8-a6fc-a682fe5183ce 2024-08-20T22:21:04.6840348Z GITHUB_ACTION=__self 2024-08-20T22:21:04.6840743Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:21:04.6844962Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-20T22:21:04.6849553Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-20T22:21:04.6850115Z NV_NVTX_VERSION=12.1.105-1 2024-08-20T22:21:04.6850462Z GITHUB_RUN_NUMBER=88082 2024-08-20T22:21:04.6850798Z TEST_CONFIG=inductor_timm 2024-08-20T22:21:04.6851160Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:21:04.6851635Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:21:04.6852089Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-20T22:21:04.6852530Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-20T22:21:04.6853280Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:21:04.6854064Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-20T22:21:04.6854823Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:21:04.6855311Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:21:04.6855767Z GITHUB_REF_TYPE=tag 2024-08-20T22:21:04.6856093Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:21:04.6856460Z NCCL_VERSION=2.17.1-1 2024-08-20T22:21:04.6856842Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.6857282Z XLA_CUDA= 2024-08-20T22:21:04.6857785Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:21:04.6860031Z *** 2024-08-20T22:21:04.6860324Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:21:04.6860719Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:21:04.6861074Z GITHUB_ACTIONS=true 2024-08-20T22:21:04.6861400Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:21:04.6861913Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-20T22:21:04.6862446Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-20T22:21:04.6862913Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.6863427Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:21:04.6863912Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.6864697Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:21:04.6865401Z UCC_HOME=/usr 2024-08-20T22:21:04.6865734Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-20T22:21:04.6866134Z VERBOSE_TEST_LOGS=False 2024-08-20T22:21:04.6866471Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:21:04.6866927Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-20T22:21:04.6867414Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:21:04.6867880Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-20T22:21:04.6868250Z SHARD_NUMBER=2 2024-08-20T22:21:04.6868550Z GITHUB_REF_PROTECTED=false 2024-08-20T22:21:04.6868895Z HOME=/var/lib/jenkins 2024-08-20T22:21:04.6869253Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:21:04.6869688Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:21:04.6870213Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:21:04.6870694Z SCCACHE_S3_KEY_PREFIX=inductor 2024-08-20T22:21:04.6871060Z CUDA_VERSION=12.1.1 2024-08-20T22:21:04.6871482Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-20T22:21:04.6871907Z NUM_TEST_SHARDS=2 2024-08-20T22:21:04.6872209Z UCX_HOME=/usr 2024-08-20T22:21:04.6872721Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-20T22:21:04.6873789Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_ef369113-cdd6-42e8-a6fc-a682fe5183ce 2024-08-20T22:21:04.6875016Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:21:04.6876239Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_ef369113-cdd6-42e8-a6fc-a682fe5183ce 2024-08-20T22:21:04.6877359Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:21:04.6878010Z GITHUB_EVENT_NAME=push 2024-08-20T22:21:04.6878334Z DASHBOARD_TAG= 2024-08-20T22:21:04.6878644Z GITHUB_RUN_ID=10479310725 2024-08-20T22:21:04.6879105Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-20T22:21:04.6879639Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-20T22:21:04.6880688Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_ef369113-cdd6-42e8-a6fc-a682fe5183ce 2024-08-20T22:21:04.6881588Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:21:04.6881997Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-20T22:21:04.6882362Z PR_NUMBER= 2024-08-20T22:21:04.6882645Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:21:04.6882979Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:21:04.6883414Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:21:04.6883857Z TERM=vt100 2024-08-20T22:21:04.6884194Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-20T22:21:04.6884601Z INSTALLED_VISION=yes 2024-08-20T22:21:04.6884906Z BRANCH= 2024-08-20T22:21:04.6885180Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:21:04.6886715Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:21:04.6887223Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:21:04.6887972Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:21:04.6888709Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:21:04.6889189Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:21:04.6889645Z REENABLED_ISSUES= 2024-08-20T22:21:04.6889960Z SHLVL=1 2024-08-20T22:21:04.6890247Z MAX_JOBS=14 2024-08-20T22:21:04.6890557Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-20T22:21:04.6890903Z NVARCH=x86_64 2024-08-20T22:21:04.6891191Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:21:04.6891642Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.6892174Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:21:04.6892659Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-20T22:21:04.6893316Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:21:04.6893876Z GITHUB_JOB=test 2024-08-20T22:21:04.6894390Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-20T22:21:04.6894955Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:21:04.6895460Z NO_TEST_TIMEOUT=False 2024-08-20T22:21:04.6895785Z TD_DISTRIBUTED=False 2024-08-20T22:21:04.6896166Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-20T22:21:04.6896593Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:21:04.6897016Z NV_NVPROF_VERSION=12.1.105-1 2024-08-20T22:21:04.6897377Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:21:04.6897724Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:21:04.6898062Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:21:04.6899088Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:21:04.6900159Z GITHUB_BASE_REF= 2024-08-20T22:21:04.6900494Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:21:04.6900861Z CI=true 2024-08-20T22:21:04.6901177Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:21:04.6901594Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:21:04.6901957Z JOB_ID=29026528125 2024-08-20T22:21:04.6902268Z INSTALLED_PROTOBUF=yes 2024-08-20T22:21:04.6902595Z GITHUB_HEAD_REF= 2024-08-20T22:21:04.6902899Z GITHUB_ACTION_REF= 2024-08-20T22:21:04.6903332Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:21:04.6903789Z TEST_SHOWLOCALS=False 2024-08-20T22:21:04.6904119Z GITHUB_WORKFLOW=inductor 2024-08-20T22:21:04.6904466Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:21:04.6905365Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_ef369113-cdd6-42e8-a6fc-a682fe5183ce 2024-08-20T22:21:04.6906174Z NO_TD=False 2024-08-20T22:21:04.6906472Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:21:04.6906825Z _=/usr/bin/env 2024-08-20T22:21:04.6907310Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-20T22:21:04.7066950Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-20T22:21:04.7068159Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-20T22:21:04.7069290Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-20T22:21:04.7070471Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-20T22:21:04.7071100Z + BUILD_DIR=build 2024-08-20T22:21:04.7071436Z + BUILD_RENAMED_DIR=build_renamed 2024-08-20T22:21:04.7071832Z + BUILD_BIN_DIR=build/bin 2024-08-20T22:21:04.7072181Z + SHARD_NUMBER=2 2024-08-20T22:21:04.7072492Z + NUM_TEST_SHARDS=2 2024-08-20T22:21:04.7072817Z + export VALGRIND=ON 2024-08-20T22:21:04.7073136Z + VALGRIND=ON 2024-08-20T22:21:04.7073626Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]] 2024-08-20T22:21:04.7074272Z + [[ 0 == \1 ]] 2024-08-20T22:21:04.7074661Z + [[ False == \1 ]] 2024-08-20T22:21:04.7075236Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]] 2024-08-20T22:21:04.7075774Z ++ realpath build/custom_test_artifacts 2024-08-20T22:21:04.7086439Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-20T22:21:04.7087414Z + [[ -n '' ]] 2024-08-20T22:21:04.7087781Z + echo 'Environment variables' 2024-08-20T22:21:04.7088170Z Environment variables 2024-08-20T22:21:04.7088494Z + env 2024-08-20T22:21:04.7094802Z INSTALLED_DB=yes 2024-08-20T22:21:04.7095301Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-20T22:21:04.7095802Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:21:04.7096334Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-20T22:21:04.7097158Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:21:04.7097943Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:21:04.7098628Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-20T22:21:04.7099273Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:21:04.7100119Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:21:04.7100764Z HOSTNAME=ff7d2001e8e4 2024-08-20T22:21:04.7101729Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_ef369113-cdd6-42e8-a6fc-a682fe5183ce 2024-08-20T22:21:04.7102698Z GITHUB_ACTION=__self 2024-08-20T22:21:04.7103100Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:21:04.7107289Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-20T22:21:04.7111804Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-20T22:21:04.7112329Z NV_NVTX_VERSION=12.1.105-1 2024-08-20T22:21:04.7112690Z GITHUB_RUN_NUMBER=88082 2024-08-20T22:21:04.7113041Z TEST_CONFIG=inductor_timm 2024-08-20T22:21:04.7113418Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:21:04.7113895Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:21:04.7114365Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-20T22:21:04.7114812Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-20T22:21:04.7115222Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:21:04.7115625Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-20T22:21:04.7116081Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:21:04.7116585Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:21:04.7117048Z GITHUB_REF_TYPE=tag 2024-08-20T22:21:04.7117383Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:21:04.7117766Z NCCL_VERSION=2.17.1-1 2024-08-20T22:21:04.7118169Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.7118611Z XLA_CUDA= 2024-08-20T22:21:04.7119079Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:21:04.7119479Z *** 2024-08-20T22:21:04.7119765Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:21:04.7120212Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:21:04.7120585Z GITHUB_ACTIONS=true 2024-08-20T22:21:04.7120932Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:21:04.7121439Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-20T22:21:04.7121981Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-20T22:21:04.7122462Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.7122992Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:21:04.7123480Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.7124267Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:21:04.7124976Z UCC_HOME=/usr 2024-08-20T22:21:04.7125323Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-20T22:21:04.7125731Z VERBOSE_TEST_LOGS=False 2024-08-20T22:21:04.7126086Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:21:04.7126786Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-20T22:21:04.7127288Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:21:04.7127757Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-20T22:21:04.7128148Z SHARD_NUMBER=2 2024-08-20T22:21:04.7128464Z GITHUB_REF_PROTECTED=false 2024-08-20T22:21:04.7128828Z HOME=/var/lib/jenkins 2024-08-20T22:21:04.7129194Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:21:04.7129638Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:21:04.7130120Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:21:04.7130602Z SCCACHE_S3_KEY_PREFIX=inductor 2024-08-20T22:21:04.7131020Z CUDA_VERSION=12.1.1 2024-08-20T22:21:04.7131451Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-20T22:21:04.7131894Z NUM_TEST_SHARDS=2 2024-08-20T22:21:04.7132202Z UCX_HOME=/usr 2024-08-20T22:21:04.7132721Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-20T22:21:04.7133820Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_ef369113-cdd6-42e8-a6fc-a682fe5183ce 2024-08-20T22:21:04.7135244Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:21:04.7136476Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_ef369113-cdd6-42e8-a6fc-a682fe5183ce 2024-08-20T22:21:04.7137584Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:21:04.7138230Z GITHUB_EVENT_NAME=push 2024-08-20T22:21:04.7138566Z DASHBOARD_TAG= 2024-08-20T22:21:04.7138880Z GITHUB_RUN_ID=10479310725 2024-08-20T22:21:04.7139342Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-20T22:21:04.7139893Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-20T22:21:04.7141114Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_ef369113-cdd6-42e8-a6fc-a682fe5183ce 2024-08-20T22:21:04.7142182Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:21:04.7142606Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-20T22:21:04.7142996Z PR_NUMBER= 2024-08-20T22:21:04.7143286Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:21:04.7143610Z VALGRIND=ON 2024-08-20T22:21:04.7143908Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:21:04.7144354Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:21:04.7144820Z TERM=vt100 2024-08-20T22:21:04.7145169Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-20T22:21:04.7145587Z INSTALLED_VISION=yes 2024-08-20T22:21:04.7145908Z BRANCH= 2024-08-20T22:21:04.7146198Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:21:04.7146598Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:21:04.7147030Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:21:04.7147793Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:21:04.7148550Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:21:04.7149036Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:21:04.7149519Z REENABLED_ISSUES= 2024-08-20T22:21:04.7149852Z SHLVL=1 2024-08-20T22:21:04.7150155Z MAX_JOBS=14 2024-08-20T22:21:04.7150472Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-20T22:21:04.7150830Z NVARCH=x86_64 2024-08-20T22:21:04.7151132Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:21:04.7151587Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.7152126Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:21:04.7152630Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-20T22:21:04.7153604Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:21:04.7154172Z GITHUB_JOB=test 2024-08-20T22:21:04.7154584Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-20T22:21:04.7155141Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:21:04.7155659Z NO_TEST_TIMEOUT=False 2024-08-20T22:21:04.7155987Z TD_DISTRIBUTED=False 2024-08-20T22:21:04.7156374Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-20T22:21:04.7156817Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:21:04.7157419Z NV_NVPROF_VERSION=12.1.105-1 2024-08-20T22:21:04.7157905Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:21:04.7158262Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:21:04.7158617Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:21:04.7159641Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:21:04.7160695Z GITHUB_BASE_REF= 2024-08-20T22:21:04.7161025Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:21:04.7161398Z CI=true 2024-08-20T22:21:04.7161721Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:21:04.7162137Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:21:04.7162518Z JOB_ID=29026528125 2024-08-20T22:21:04.7162837Z INSTALLED_PROTOBUF=yes 2024-08-20T22:21:04.7163170Z GITHUB_HEAD_REF= 2024-08-20T22:21:04.7163481Z GITHUB_ACTION_REF= 2024-08-20T22:21:04.7163925Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:21:04.7164390Z TEST_SHOWLOCALS=False 2024-08-20T22:21:04.7164738Z GITHUB_WORKFLOW=inductor 2024-08-20T22:21:04.7165106Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:21:04.7166019Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_ef369113-cdd6-42e8-a6fc-a682fe5183ce 2024-08-20T22:21:04.7166825Z NO_TD=False 2024-08-20T22:21:04.7167138Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:21:04.7167504Z _=/usr/bin/env 2024-08-20T22:21:04.7167857Z + echo 'Testing pytorch' 2024-08-20T22:21:04.7168199Z Testing pytorch 2024-08-20T22:21:04.7168533Z + export LANG=C.UTF-8 2024-08-20T22:21:04.7168879Z + LANG=C.UTF-8 2024-08-20T22:21:04.7169170Z + PR_NUMBER= 2024-08-20T22:21:04.7169497Z + [[ inductor_timm == \d\e\f\a\u\l\t ]] 2024-08-20T22:21:04.7169962Z + [[ inductor_timm == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-20T22:21:04.7170419Z + [[ inductor_timm == \s\l\o\w ]] 2024-08-20T22:21:04.7171022Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-08-20T22:21:04.7171726Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]] 2024-08-20T22:21:04.7172282Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:21:04.7172765Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:21:04.7173194Z + [[ inductor_timm == *crossref* ]] 2024-08-20T22:21:04.7173756Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:21:04.7174480Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]] 2024-08-20T22:21:04.7175134Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:21:04.7175692Z + pip_install --user ninja==1.10.2 2024-08-20T22:21:04.7176231Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-20T22:21:05.9910132Z Collecting ninja==1.10.2 2024-08-20T22:21:06.0095935Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-20T22:21:06.0428932Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-20T22:21:06.6685942Z Installing collected packages: ninja 2024-08-20T22:21:06.6765611Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-20T22:21:06.6767126Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-20T22:21:06.7068169Z Successfully installed ninja-1.10.2 2024-08-20T22:21:06.7855507Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:21:06.7857818Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:21:06.7859510Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:21:06.7860027Z + install_tlparse 2024-08-20T22:21:06.7860506Z + pip_install --user tlparse==0.3.25 2024-08-20T22:21:06.7861198Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-20T22:21:07.1987132Z Collecting tlparse==0.3.25 2024-08-20T22:21:07.2160410Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-20T22:21:07.2507900Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-20T22:21:07.8647339Z Installing collected packages: tlparse 2024-08-20T22:21:07.9244536Z Successfully installed tlparse-0.3.25 2024-08-20T22:21:08.0020217Z ++ python -m site --user-base 2024-08-20T22:21:08.0271360Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:21:08.0273084Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]] 2024-08-20T22:21:08.0273764Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]] 2024-08-20T22:21:08.0274441Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:21:08.0275419Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-08-20T22:21:08.0276581Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass 2024-08-20T22:21:08.0277277Z + cd test 2024-08-20T22:21:08.0277819Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-20T22:21:09.6559422Z + [[ inductor_timm == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-20T22:21:09.6560010Z + [[ inductor_timm == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-20T22:21:09.6563145Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-20T22:21:09.6563574Z + [[ inductor_timm == *pr_time_benchmarks* ]] 2024-08-20T22:21:09.6564058Z + [[ inductor_timm == *dynamo_eager* ]] 2024-08-20T22:21:09.6564504Z + [[ inductor_timm == *aot_eager* ]] 2024-08-20T22:21:09.6564944Z + [[ inductor_timm == *aot_inductor* ]] 2024-08-20T22:21:09.6565384Z + [[ inductor_timm == *inductor* ]] 2024-08-20T22:21:09.6565804Z + [[ inductor_timm != *perf* ]] 2024-08-20T22:21:09.6566493Z + DYNAMO_BENCHMARK_FLAGS+=(--inductor) 2024-08-20T22:21:09.6566947Z + [[ inductor_timm == *dynamic* ]] 2024-08-20T22:21:09.6567351Z + [[ inductor_timm == *cpu* ]] 2024-08-20T22:21:09.6567814Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-20T22:21:09.6601679Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:21:09.6602550Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-08-20T22:21:09.6605152Z + cd test 2024-08-20T22:21:09.6605720Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-20T22:21:11.1306482Z PyTorch built with: 2024-08-20T22:21:11.1307233Z - GCC 9.4 2024-08-20T22:21:11.1307637Z - C++ Version: 201703 2024-08-20T22:21:11.1308532Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:21:11.1309630Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:21:11.1310306Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:21:11.1310877Z - LAPACK is enabled (usually provided by MKL) 2024-08-20T22:21:11.1311399Z - NNPACK is enabled 2024-08-20T22:21:11.1311823Z - CPU capability usage: AVX2 2024-08-20T22:21:11.1312305Z - CUDA Runtime 12.1 2024-08-20T22:21:11.1312852Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-08-20T22:21:11.1313482Z - CuDNN 90.1 (built against CUDA 12.4) 2024-08-20T22:21:11.1313965Z - Magma 2.6.1 2024-08-20T22:21:11.1321126Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-20T22:21:11.1327567Z 2024-08-20T22:21:11.4520786Z + cd test 2024-08-20T22:21:11.4521551Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-20T22:21:12.8011857Z ATen/Parallel: 2024-08-20T22:21:12.8012443Z at::get_num_threads() : 8 2024-08-20T22:21:12.8012987Z at::get_num_interop_threads() : 16 2024-08-20T22:21:12.8013438Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:21:12.8013852Z omp_get_max_threads() : 8 2024-08-20T22:21:12.8015016Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:21:12.8015815Z mkl_get_max_threads() : 8 2024-08-20T22:21:12.8016446Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:21:12.8017097Z std::thread::hardware_concurrency() : 16 2024-08-20T22:21:12.8017543Z Environment variables: 2024-08-20T22:21:12.8017905Z OMP_NUM_THREADS : [not set] 2024-08-20T22:21:12.8018281Z MKL_NUM_THREADS : [not set] 2024-08-20T22:21:12.8018667Z ATen parallel backend: OpenMP 2024-08-20T22:21:12.8018926Z 2024-08-20T22:21:13.0797420Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:21:13.0798194Z + [[ inductor_timm == *backward* ]] 2024-08-20T22:21:13.0798656Z + [[ inductor_timm == *xla* ]] 2024-08-20T22:21:13.0799100Z + [[ inductor_timm == *executorch* ]] 2024-08-20T22:21:13.0799576Z + [[ inductor_timm == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-20T22:21:13.0800219Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:21:13.0800772Z + [[ inductor_timm == distributed ]] 2024-08-20T22:21:13.0801322Z + [[ inductor_timm == *inductor_distributed* ]] 2024-08-20T22:21:13.0801890Z + [[ inductor_timm == *inductor-halide* ]] 2024-08-20T22:21:13.0802605Z + [[ inductor_timm == *inductor-micro-benchmark* ]] 2024-08-20T22:21:13.0803263Z + [[ inductor_timm == *huggingface* ]] 2024-08-20T22:21:13.0803725Z + [[ inductor_timm == *timm* ]] 2024-08-20T22:21:13.0804105Z + install_torchvision 2024-08-20T22:21:13.0804441Z + local orig_preload 2024-08-20T22:21:13.0804764Z + local commit 2024-08-20T22:21:13.0805083Z ++ get_pinned_commit vision 2024-08-20T22:21:13.0805470Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-20T22:21:13.0820537Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:13.0821155Z + orig_preload= 2024-08-20T22:21:13.0821641Z + '[' -n '' ']' 2024-08-20T22:21:13.0822546Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:13.0823978Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:13.4165764Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:13.4170131Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-jkre0mmx 2024-08-20T22:21:13.4201926Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-jkre0mmx 2024-08-20T22:21:14.9447112Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-20T22:21:14.9476466Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:16.2844559Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:16.5943388Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:19.0860897Z Preparing metadata (setup.py) ... [?25l- \ done 2024-08-20T22:21:19.0895502Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.22.4) 2024-08-20T22:21:19.0897590Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git40ec5f6) 2024-08-20T22:21:19.0902669Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-20T22:21:19.0973279Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-20T22:21:19.0977839Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-20T22:21:19.0981096Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-20T22:21:19.0984247Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-20T22:21:19.0987566Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-20T22:21:19.0992564Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.1) 2024-08-20T22:21:19.1006532Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-20T22:21:19.1477541Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-20T22:21:19.1540478Z Building wheels for collected packages: torchvision 2024-08-20T22:22:36.1021412Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2024-08-20T22:22:36.1053994Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2115924 sha256=559d23119a6b3cc589d8dcd42b11648f2fc33851115b72faa50334d4ed41f9ae 2024-08-20T22:22:36.1057893Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-08-20T22:22:36.1092423Z Successfully built torchvision 2024-08-20T22:22:36.5833923Z Installing collected packages: torchvision 2024-08-20T22:22:36.9952404Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-20T22:22:37.1352037Z + '[' -n '' ']' 2024-08-20T22:22:37.1352388Z + id=1 2024-08-20T22:22:37.1357143Z + test_dynamo_benchmark timm_models 1 2024-08-20T22:22:37.1357711Z ++ pwd 2024-08-20T22:22:37.1359076Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:22:37.1359797Z + local suite=timm_models 2024-08-20T22:22:37.1360208Z + shift 2024-08-20T22:22:37.1360582Z + local shard_id=1 2024-08-20T22:22:37.1360999Z + shift 2024-08-20T22:22:37.1361407Z + [[ inductor_timm == *perf_compare* ]] 2024-08-20T22:22:37.1361930Z + [[ inductor_timm == *perf* ]] 2024-08-20T22:22:37.1363902Z + [[ inductor_timm == *cpu* ]] 2024-08-20T22:22:37.1364528Z + [[ inductor_timm == *aot_inductor* ]] 2024-08-20T22:22:37.1365499Z + test_single_dynamo_benchmark inference timm_models 1 --inference --bfloat16 2024-08-20T22:22:37.1366574Z ++ pwd 2024-08-20T22:22:37.1367476Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:22:37.1368219Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:22:37.1393409Z + local name=inference 2024-08-20T22:22:37.1393883Z + shift 2024-08-20T22:22:37.1394273Z + local suite=timm_models 2024-08-20T22:22:37.1394731Z + shift 2024-08-20T22:22:37.1395017Z + local shard_id=1 2024-08-20T22:22:37.1395328Z + shift 2024-08-20T22:22:37.1397536Z + partition_flags=() 2024-08-20T22:22:37.1397978Z + local partition_flags 2024-08-20T22:22:37.1398530Z + [[ -n 2 ]] 2024-08-20T22:22:37.1398934Z + [[ -n 1 ]] 2024-08-20T22:22:37.1399551Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-08-20T22:22:37.1400210Z + [[ inductor_timm == *perf_compare* ]] 2024-08-20T22:22:37.1400642Z + [[ inductor_timm == *perf* ]] 2024-08-20T22:22:37.1401053Z + [[ inductor_timm == *aot_inductor* ]] 2024-08-20T22:22:37.1401485Z + [[ inductor_timm == *_avx2* ]] 2024-08-20T22:22:37.1401910Z + [[ inductor_timm == *_avx512* ]] 2024-08-20T22:22:37.1403453Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --inductor --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 1 --output /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv 2024-08-20T22:22:43.5762008Z 2024-08-20T22:22:43.7217081Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:22:43.7217436Z 2024-08-20T22:22:43.8037627Z model.safetensors: 0% 0.00/29.6M [00:00 will be ignored 2024-08-20T22:46:13.7177304Z pass 2024-08-20T22:46:13.7223936Z TIMING: entire_frame_compile:68.85652 code_gen:30.38141 inductor_compile:48.15215 backend_compile:54.11503 total_wall_time:79.7301 2024-08-20T22:46:13.7227704Z STATS: call_* op count: 700 | FakeTensorMode.__torch_dispatch__:44300 | FakeTensor.__torch_dispatch__:16141 | ProxyTorchDispatchMode.__torch_dispatch__:17697 2024-08-20T22:46:13.7228876Z Dynamo produced 4 graphs covering 700 ops with 7 graph breaks (5 unique) 2024-08-20T22:46:21.1119241Z 2024-08-20T22:46:21.9791634Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:46:21.9792220Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:46:21.9792682Z cuda train mobilenetv2_100 2024-08-20T22:47:07.0900884Z W0820 22:47:07.089000 16043 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:47:47.0791723Z pass 2024-08-20T22:47:47.0839064Z TIMING: entire_frame_compile:66.84927 code_gen:29.25655 inductor_compile:47.94195 backend_compile:52.23477 total_wall_time:79.41382 2024-08-20T22:47:47.0841570Z STATS: call_* op count: 699 | FakeTensorMode.__torch_dispatch__:45004 | FakeTensor.__torch_dispatch__:17124 | ProxyTorchDispatchMode.__torch_dispatch__:17713 2024-08-20T22:47:47.0842750Z Dynamo produced 3 graphs covering 699 ops with 7 graph breaks (5 unique) 2024-08-20T22:47:54.4868190Z 2024-08-20T22:47:55.3929454Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:47:55.3930358Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:47:55.3931261Z cuda train mobilenetv3_large_100 2024-08-20T22:48:44.8869023Z W0820 22:48:44.885000 16829 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:49:30.8165344Z pass 2024-08-20T22:49:30.8235932Z TIMING: entire_frame_compile:74.51428 code_gen:34.92915 inductor_compile:55.01852 backend_compile:58.54364 total_wall_time:89.30699 2024-08-20T22:49:30.8238418Z STATS: call_* op count: 775 | FakeTensorMode.__torch_dispatch__:48411 | FakeTensor.__torch_dispatch__:18376 | ProxyTorchDispatchMode.__torch_dispatch__:19569 2024-08-20T22:49:30.8239606Z Dynamo produced 3 graphs covering 775 ops with 7 graph breaks (5 unique) 2024-08-20T22:49:38.7856950Z 2024-08-20T22:49:39.6996222Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:49:39.6999969Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:49:39.7001061Z cuda train mobilevit_s 2024-08-20T22:50:55.2500108Z pass 2024-08-20T22:50:55.2583179Z TIMING: entire_frame_compile:37.70952 code_gen:35.47267 inductor_compile:51.44114 backend_compile:29.4501 total_wall_time:69.52704 2024-08-20T22:50:55.2585766Z STATS: call_* op count: 363 | FakeTensorMode.__torch_dispatch__:39430 | FakeTensor.__torch_dispatch__:10260 | ProxyTorchDispatchMode.__torch_dispatch__:16472 2024-08-20T22:50:55.2586926Z Dynamo produced 3 graphs covering 363 ops with 6 graph breaks (5 unique) 2024-08-20T22:51:01.4274249Z 2024-08-20T22:51:02.6671622Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:51:02.6672099Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:51:02.6676637Z cuda train nfnet_l0 2024-08-20T22:51:49.5277255Z W0820 22:51:49.526000 19084 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:52:50.7053467Z pass 2024-08-20T22:52:50.7290464Z TIMING: entire_frame_compile:85.10074 code_gen:33.93562 inductor_compile:55.28783 backend_compile:63.31656 total_wall_time:100.61059 2024-08-20T22:52:50.7291877Z STATS: call_* op count: 1205 | FakeTensorMode.__torch_dispatch__:60165 | FakeTensor.__torch_dispatch__:20670 | ProxyTorchDispatchMode.__torch_dispatch__:25953 2024-08-20T22:52:50.7293017Z Dynamo produced 3 graphs covering 1205 ops with 7 graph breaks (5 unique) 2024-08-20T22:52:58.7084402Z 2024-08-20T22:52:59.8781018Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:52:59.8781623Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:52:59.8783493Z cuda train pit_b_224 2024-08-20T22:53:35.0071040Z pass 2024-08-20T22:53:35.0564881Z TIMING: entire_frame_compile:17.99272 code_gen:11.35095 inductor_compile:18.75665 backend_compile:13.58386 total_wall_time:29.28679 2024-08-20T22:53:35.0566448Z STATS: call_* op count: 310 | FakeTensorMode.__torch_dispatch__:28801 | FakeTensor.__torch_dispatch__:6181 | ProxyTorchDispatchMode.__torch_dispatch__:12561 2024-08-20T22:53:35.0569280Z Dynamo produced 2 graphs covering 310 ops with 6 graph breaks (5 unique) 2024-08-20T22:53:40.2456786Z 2024-08-20T22:53:41.9977219Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:53:41.9977866Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:53:41.9978461Z cuda train pnasnet5large 2024-08-20T22:58:58.7655807Z pass 2024-08-20T22:58:58.9093653Z TIMING: entire_frame_compile:148.30989 code_gen:182.26938 inductor_compile:224.59214 backend_compile:112.86174 total_wall_time:296.1033 2024-08-20T22:58:58.9095142Z STATS: call_* op count: 1161 | FakeTensorMode.__torch_dispatch__:129488 | FakeTensor.__torch_dispatch__:34521 | ProxyTorchDispatchMode.__torch_dispatch__:47606 2024-08-20T22:58:58.9096267Z Dynamo produced 2 graphs covering 1161 ops with 5 graph breaks (4 unique) 2024-08-20T22:59:12.8602084Z 2024-08-20T22:59:14.3459672Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:59:14.3460388Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:59:14.3461091Z cuda train poolformer_m36 2024-08-20T23:02:48.6188467Z pass 2024-08-20T23:02:48.6593738Z TIMING: entire_frame_compile:61.33454 code_gen:159.52934 inductor_compile:180.44776 backend_compile:50.75475 total_wall_time:205.44373 2024-08-20T23:02:48.6596974Z STATS: call_* op count: 554 | FakeTensorMode.__torch_dispatch__:50540 | ProxyTorchDispatchMode.__torch_dispatch__:22827 | FakeTensor.__torch_dispatch__:15814 2024-08-20T23:02:48.6598247Z Dynamo produced 2 graphs covering 554 ops with 6 graph breaks (5 unique) 2024-08-20T23:02:57.2667929Z 2024-08-20T23:02:58.1450064Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:02:58.1450542Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:02:58.1450998Z cuda train regnety_002 2024-08-20T23:03:41.4891121Z pass 2024-08-20T23:03:41.4928507Z TIMING: entire_frame_compile:28.73595 code_gen:13.77342 inductor_compile:23.33917 backend_compile:20.09651 total_wall_time:40.1749 2024-08-20T23:03:41.4930057Z STATS: call_* op count: 271 | FakeTensorMode.__torch_dispatch__:29920 | FakeTensor.__torch_dispatch__:8391 | ProxyTorchDispatchMode.__torch_dispatch__:12209 2024-08-20T23:03:41.4931443Z Dynamo produced 2 graphs covering 271 ops with 6 graph breaks (5 unique) 2024-08-20T23:03:47.1637868Z 2024-08-20T23:03:48.3169349Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:03:48.3169957Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:03:48.3170455Z cuda train repvgg_a2 2024-08-20T23:04:35.5061160Z W0820 23:04:35.505000 24186 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:05:18.7173751Z pass 2024-08-20T23:05:18.7331960Z TIMING: entire_frame_compile:72.3931 code_gen:30.72198 inductor_compile:49.71628 backend_compile:56.23622 total_wall_time:83.18925 2024-08-20T23:05:18.7333786Z STATS: call_* op count: 756 | FakeTensorMode.__torch_dispatch__:46838 | FakeTensor.__torch_dispatch__:17199 | ProxyTorchDispatchMode.__torch_dispatch__:18975 2024-08-20T23:05:18.7335232Z Dynamo produced 3 graphs covering 756 ops with 7 graph breaks (5 unique) 2024-08-20T23:05:26.3133676Z 2024-08-20T23:05:27.8703786Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:05:27.8704312Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:05:27.8704758Z cuda train res2net101_26w_4s 2024-08-20T23:08:05.5813426Z pass 2024-08-20T23:08:05.6152514Z TIMING: entire_frame_compile:101.28755 code_gen:61.60621 inductor_compile:96.0425 backend_compile:79.01572 total_wall_time:145.87937 2024-08-20T23:08:05.6154417Z STATS: call_* op count: 977 | FakeTensorMode.__torch_dispatch__:100624 | FakeTensor.__torch_dispatch__:30426 | ProxyTorchDispatchMode.__torch_dispatch__:37834 2024-08-20T23:08:05.6155572Z Dynamo produced 2 graphs covering 977 ops with 6 graph breaks (5 unique) 2024-08-20T23:08:16.9451925Z 2024-08-20T23:08:18.2180895Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:08:18.2181588Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:08:18.2182245Z cuda train res2net50_14w_8s 2024-08-20T23:10:44.8440830Z pass 2024-08-20T23:10:44.8645855Z TIMING: entire_frame_compile:93.0717 code_gen:60.90332 inductor_compile:91.22641 backend_compile:72.23116 total_wall_time:137.39037 2024-08-20T23:10:44.8647474Z STATS: call_* op count: 852 | FakeTensorMode.__torch_dispatch__:90422 | FakeTensor.__torch_dispatch__:26327 | ProxyTorchDispatchMode.__torch_dispatch__:32715 2024-08-20T23:10:44.8651300Z Dynamo produced 2 graphs covering 852 ops with 6 graph breaks (5 unique) 2024-08-20T23:10:55.0837266Z 2024-08-20T23:10:56.1963892Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:10:56.1964632Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:10:56.1965322Z cuda train res2next50 2024-08-20T23:12:12.2424624Z pass 2024-08-20T23:12:12.2601480Z TIMING: entire_frame_compile:46.3358 code_gen:28.30229 inductor_compile:44.8675 backend_compile:34.50363 total_wall_time:70.4442 2024-08-20T23:12:12.2606744Z STATS: call_* op count: 484 | FakeTensorMode.__torch_dispatch__:49960 | FakeTensor.__torch_dispatch__:15105 | ProxyTorchDispatchMode.__torch_dispatch__:18879 2024-08-20T23:12:12.2608471Z Dynamo produced 2 graphs covering 484 ops with 6 graph breaks (5 unique) 2024-08-20T23:12:19.1013011Z 2024-08-20T23:12:19.7893069Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:12:19.7893983Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:12:19.7894989Z cuda train resmlp_12_224 2024-08-20T23:12:42.1170702Z pass 2024-08-20T23:12:42.1298563Z TIMING: entire_frame_compile:12.51139 code_gen:6.60457 inductor_compile:10.99698 backend_compile:9.12025 total_wall_time:19.43859 2024-08-20T23:12:42.1302087Z STATS: call_* op count: 179 | FakeTensorMode.__torch_dispatch__:18242 | ProxyTorchDispatchMode.__torch_dispatch__:8345 | FakeTensor.__torch_dispatch__:3939 2024-08-20T23:12:42.1303263Z Dynamo produced 2 graphs covering 179 ops with 6 graph breaks (5 unique) 2024-08-20T23:12:46.9041607Z 2024-08-20T23:12:48.5338181Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:12:48.5338661Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:12:48.5339110Z cuda train resnest101e 2024-08-20T23:15:05.6949959Z pass 2024-08-20T23:15:05.7362982Z TIMING: entire_frame_compile:82.30603 code_gen:46.89595 inductor_compile:78.82018 backend_compile:61.2448 total_wall_time:125.52666 2024-08-20T23:15:05.7364392Z STATS: call_* op count: 1028 | FakeTensorMode.__torch_dispatch__:94317 | FakeTensor.__torch_dispatch__:26591 | ProxyTorchDispatchMode.__torch_dispatch__:38715 2024-08-20T23:15:05.7365527Z Dynamo produced 2 graphs covering 1028 ops with 6 graph breaks (5 unique) 2024-08-20T23:15:15.3727353Z 2024-08-20T23:15:16.5474064Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:15:16.5474688Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:15:16.5475168Z cuda train rexnet_100 2024-08-20T23:16:27.7244769Z W0820 23:16:27.723000 29090 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:17:27.7590899Z pass 2024-08-20T23:17:27.7676107Z TIMING: entire_frame_compile:100.72555 code_gen:50.0338 inductor_compile:77.57989 backend_compile:78.60566 total_wall_time:123.23752 2024-08-20T23:17:27.7677500Z STATS: call_* op count: 1034 | FakeTensorMode.__torch_dispatch__:64940 | FakeTensor.__torch_dispatch__:24077 | ProxyTorchDispatchMode.__torch_dispatch__:26051 2024-08-20T23:17:27.7679910Z Dynamo produced 3 graphs covering 1034 ops with 7 graph breaks (5 unique) 2024-08-20T23:17:37.1349524Z 2024-08-20T23:17:38.2374337Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:17:38.2375635Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:17:38.2376671Z cuda train sebotnet33ts_256 2024-08-20T23:18:44.8180784Z pass 2024-08-20T23:18:44.8307516Z TIMING: entire_frame_compile:39.62665 code_gen:28.30805 inductor_compile:41.25088 backend_compile:32.13402 total_wall_time:60.36547 2024-08-20T23:18:44.8311009Z STATS: call_* op count: 372 | FakeTensorMode.__torch_dispatch__:34242 | FakeTensor.__torch_dispatch__:9073 | ProxyTorchDispatchMode.__torch_dispatch__:13430 2024-08-20T23:18:44.8312157Z Dynamo produced 3 graphs covering 372 ops with 6 graph breaks (5 unique) 2024-08-20T23:18:50.9690882Z 2024-08-20T23:18:52.1777863Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:18:52.1782428Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:18:52.1782905Z cuda train selecsls42b 2024-08-20T23:19:26.9284539Z pass 2024-08-20T23:19:26.9556718Z TIMING: entire_frame_compile:22.09662 code_gen:11.10233 inductor_compile:19.68041 backend_compile:16.66702 total_wall_time:31.22504 2024-08-20T23:19:26.9560500Z STATS: call_* op count: 178 | FakeTensorMode.__torch_dispatch__:22496 | FakeTensor.__torch_dispatch__:6628 | ProxyTorchDispatchMode.__torch_dispatch__:8917 2024-08-20T23:19:26.9561673Z Dynamo produced 2 graphs covering 178 ops with 6 graph breaks (5 unique) 2024-08-20T23:19:32.2263187Z 2024-08-20T23:19:33.1324396Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:19:33.1324885Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:19:33.1325339Z cuda train spnasnet_100 2024-08-20T23:20:24.7104853Z W0820 23:20:24.709000 33497 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:21:14.1339029Z pass 2024-08-20T23:21:14.1379322Z TIMING: entire_frame_compile:81.43485 code_gen:34.62053 inductor_compile:56.11515 backend_compile:63.71007 total_wall_time:94.48559 2024-08-20T23:21:14.1383418Z STATS: call_* op count: 855 | FakeTensorMode.__torch_dispatch__:54124 | FakeTensor.__torch_dispatch__:19775 | ProxyTorchDispatchMode.__torch_dispatch__:21710 2024-08-20T23:21:14.1384571Z Dynamo produced 3 graphs covering 855 ops with 7 graph breaks (5 unique) 2024-08-20T23:21:22.2580060Z 2024-08-20T23:21:24.1871137Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:21:24.1871662Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:21:24.1874683Z cuda train swin_base_patch4_window7_224 2024-08-20T23:23:03.8746210Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-20T23:23:03.8747391Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 404, in torch_dynamo_resume_in_forward_and_backward_pass_at_402 2024-08-20T23:23:03.8751634Z pred = mod(*cloned_inputs) 2024-08-20T23:23:03.8753100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 838, in forward 2024-08-20T23:23:03.8753945Z x = self.forward_features(x) 2024-08-20T23:23:03.8754903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 830, in forward_features 2024-08-20T23:23:03.8755765Z x = self.layers(x) 2024-08-20T23:23:03.8757741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 559, in forward 2024-08-20T23:23:03.8759289Z x = self.blocks(x) 2024-08-20T23:23:03.8760276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 406, in forward 2024-08-20T23:23:03.8761240Z x = x + self.drop_path1(self._attn(self.norm1(x))) 2024-08-20T23:23:03.8762375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 390, in _attn 2024-08-20T23:23:03.8763550Z attn_windows = self.attn(x_windows, mask=attn_mask) # nW*B, window_size*window_size, C 2024-08-20T23:23:03.8764809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 194, in forward 2024-08-20T23:23:03.8765737Z attn = attn + self._get_rel_pos_bias() 2024-08-20T23:23:03.8766831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 165, in _get_rel_pos_bias 2024-08-20T23:23:03.8767909Z relative_position_bias = self.relative_position_bias_table[ 2024-08-20T23:23:03.8768391Z 2024-08-20T23:23:04.4015170Z W0820 23:23:04.400000 33825 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:24:22.8054686Z pass 2024-08-20T23:24:22.8471289Z TIMING: entire_frame_compile:132.05722 code_gen:66.2024 inductor_compile:101.82513 backend_compile:107.02323 total_wall_time:166.57832 2024-08-20T23:24:22.8473871Z STATS: call_* op count: 2446 | FakeTensorMode.__torch_dispatch__:115795 | FakeTensor.__torch_dispatch__:29780 | ProxyTorchDispatchMode.__torch_dispatch__:50142 2024-08-20T23:24:22.8475032Z Dynamo produced 3 graphs covering 2446 ops with 7 graph breaks (5 unique) 2024-08-20T23:24:33.2559767Z 2024-08-20T23:24:36.4817579Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:24:36.4818236Z loading model: 0it [00:03, ?it/s] 2024-08-20T23:24:36.4818705Z cuda train swsl_resnext101_32x16d 2024-08-20T23:25:58.6299415Z pass 2024-08-20T23:25:58.7667008Z TIMING: entire_frame_compile:49.58112 code_gen:20.74792 inductor_compile:37.91981 backend_compile:35.5459 total_wall_time:67.44996 2024-08-20T23:25:58.7669578Z STATS: call_* op count: 453 | FakeTensorMode.__torch_dispatch__:54554 | FakeTensor.__torch_dispatch__:16128 | ProxyTorchDispatchMode.__torch_dispatch__:22001 2024-08-20T23:25:58.7670738Z Dynamo produced 2 graphs covering 453 ops with 6 graph breaks (5 unique) 2024-08-20T23:26:05.6889400Z 2024-08-20T23:26:06.6247079Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:26:06.6247550Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:26:06.6248007Z cuda train tf_efficientnet_b0 2024-08-20T23:27:05.8981991Z pass 2024-08-20T23:27:05.9065355Z TIMING: entire_frame_compile:36.94417 code_gen:22.8776 inductor_compile:36.21204 backend_compile:27.92597 total_wall_time:55.05474 2024-08-20T23:27:05.9066751Z STATS: call_* op count: 297 | FakeTensorMode.__torch_dispatch__:36675 | FakeTensor.__torch_dispatch__:11256 | ProxyTorchDispatchMode.__torch_dispatch__:14649 2024-08-20T23:27:05.9069964Z Dynamo produced 2 graphs covering 297 ops with 6 graph breaks (5 unique) 2024-08-20T23:27:12.4436601Z 2024-08-20T23:27:13.4934005Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:27:13.4934485Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:27:13.4935009Z cuda train tf_mixnet_l 2024-08-20T23:28:37.8981030Z pass 2024-08-20T23:28:37.9075198Z TIMING: entire_frame_compile:54.06111 code_gen:33.08535 inductor_compile:50.39495 backend_compile:39.47761 total_wall_time:78.69787 2024-08-20T23:28:37.9077540Z STATS: call_* op count: 598 | FakeTensorMode.__torch_dispatch__:53845 | FakeTensor.__torch_dispatch__:14221 | ProxyTorchDispatchMode.__torch_dispatch__:19823 2024-08-20T23:28:37.9078702Z Dynamo produced 2 graphs covering 598 ops with 6 graph breaks (5 unique) 2024-08-20T23:28:45.4067902Z 2024-08-20T23:28:46.4113004Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:28:46.4115420Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:28:46.4115894Z cuda train tinynet_a 2024-08-20T23:29:51.6742838Z pass 2024-08-20T23:29:51.6827589Z TIMING: entire_frame_compile:41.45822 code_gen:23.90128 inductor_compile:39.11247 backend_compile:31.08546 total_wall_time:60.80524 2024-08-20T23:29:51.6829040Z STATS: call_* op count: 346 | FakeTensorMode.__torch_dispatch__:43017 | FakeTensor.__torch_dispatch__:13254 | ProxyTorchDispatchMode.__torch_dispatch__:17283 2024-08-20T23:29:51.6831930Z Dynamo produced 2 graphs covering 346 ops with 6 graph breaks (5 unique) 2024-08-20T23:29:58.2837465Z 2024-08-20T23:29:59.3860745Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:29:59.3861485Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:29:59.3862466Z cuda train tnt_s_patch16_224 2024-08-20T23:31:12.6279587Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-20T23:31:12.6281120Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 404, in torch_dynamo_resume_in_forward_and_backward_pass_at_402 2024-08-20T23:31:12.6282353Z pred = mod(*cloned_inputs) 2024-08-20T23:31:12.6283382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/tnt.py", line 335, in forward 2024-08-20T23:31:12.6284152Z x = self.forward_features(x) 2024-08-20T23:31:12.6285035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/tnt.py", line 311, in forward_features 2024-08-20T23:31:12.6286015Z pixel_embed = self.pixel_embed(x, self.pixel_pos) 2024-08-20T23:31:12.6287144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/tnt.py", line 183, in forward 2024-08-20T23:31:12.6288217Z x = self.unfold(x) 2024-08-20T23:31:12.6288445Z 2024-08-20T23:31:12.9477511Z W0820 23:31:12.946000 37994 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:32:36.6260096Z pass 2024-08-20T23:32:36.6507884Z TIMING: entire_frame_compile:123.73433 code_gen:55.38003 inductor_compile:89.55387 backend_compile:101.93076 total_wall_time:148.03715 2024-08-20T23:32:36.6510349Z STATS: call_* op count: 1849 | FakeTensorMode.__torch_dispatch__:109444 | FakeTensor.__torch_dispatch__:30233 | ProxyTorchDispatchMode.__torch_dispatch__:49261 2024-08-20T23:32:36.6511646Z Dynamo produced 3 graphs covering 1849 ops with 7 graph breaks (5 unique) 2024-08-20T23:32:47.3350053Z 2024-08-20T23:32:48.7731806Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:32:48.7732767Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:32:48.7733673Z cuda train twins_pcpvt_base 2024-08-20T23:34:14.2896287Z W0820 23:34:14.288000 38627 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:36:23.6449057Z pass 2024-08-20T23:36:23.6754348Z TIMING: entire_frame_compile:166.5455 code_gen:74.5146 inductor_compile:121.45445 backend_compile:133.01344 total_wall_time:199.04087 2024-08-20T23:36:23.6756062Z STATS: call_* op count: 2432 | FakeTensorMode.__torch_dispatch__:137141 | FakeTensor.__torch_dispatch__:43685 | ProxyTorchDispatchMode.__torch_dispatch__:59071 2024-08-20T23:36:23.6757491Z Dynamo produced 3 graphs covering 2432 ops with 7 graph breaks (5 unique) 2024-08-20T23:36:36.4546345Z 2024-08-20T23:36:37.6867522Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:36:37.6868002Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:36:37.6868451Z cuda train visformer_small 2024-08-20T23:37:13.9566779Z W0820 23:37:13.955000 39575 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:37:47.0041497Z pass 2024-08-20T23:37:47.0294205Z TIMING: entire_frame_compile:55.55061 code_gen:20.26328 inductor_compile:33.77118 backend_compile:42.92487 total_wall_time:63.53973 2024-08-20T23:37:47.0298117Z STATS: call_* op count: 703 | FakeTensorMode.__torch_dispatch__:39606 | FakeTensor.__torch_dispatch__:13021 | ProxyTorchDispatchMode.__torch_dispatch__:15868 2024-08-20T23:37:47.0299599Z Dynamo produced 3 graphs covering 703 ops with 7 graph breaks (5 unique) 2024-08-20T23:37:53.4036326Z 2024-08-20T23:37:54.9914680Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:37:54.9915234Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:37:54.9915751Z cuda train vit_base_patch16_224 2024-08-20T23:38:23.5720682Z W0820 23:38:23.571000 40176 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:38:59.0012084Z pass 2024-08-20T23:38:59.0532854Z TIMING: entire_frame_compile:47.54265 code_gen:20.16483 inductor_compile:33.45917 backend_compile:38.18814 total_wall_time:55.65371 2024-08-20T23:38:59.0534307Z STATS: call_* op count: 740 | FakeTensorMode.__torch_dispatch__:39863 | FakeTensor.__torch_dispatch__:12925 | ProxyTorchDispatchMode.__torch_dispatch__:17807 2024-08-20T23:38:59.0539145Z Dynamo produced 3 graphs covering 740 ops with 7 graph breaks (5 unique) 2024-08-20T23:39:05.3611448Z 2024-08-20T23:39:06.5192129Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:39:06.5192659Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:39:06.5193096Z cuda train volo_d1_224 2024-08-20T23:39:40.7487343Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-20T23:39:40.7488513Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 404, in torch_dynamo_resume_in_forward_and_backward_pass_at_402 2024-08-20T23:39:40.7489468Z pred = mod(*cloned_inputs) 2024-08-20T23:39:40.7491055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 822, in forward 2024-08-20T23:39:40.7491824Z x = self.forward_features(x) 2024-08-20T23:39:40.7492725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 795, in forward_features 2024-08-20T23:39:40.7493542Z x = self.forward_tokens(x) 2024-08-20T23:39:40.7494458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 642, in forward_tokens 2024-08-20T23:39:40.7495274Z x = block(x) 2024-08-20T23:39:40.7496042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 135, in forward 2024-08-20T23:39:40.7496866Z x = x + self.drop_path1(self.attn(self.norm1(x))) 2024-08-20T23:39:40.7497763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 90, in forward 2024-08-20T23:39:40.7498822Z x = F.fold(x, output_size=(H, W), kernel_size=self.kernel_size, padding=self.padding, stride=self.stride) 2024-08-20T23:39:40.7499424Z 2024-08-20T23:40:06.7254118Z W0820 23:40:06.724000 40485 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:41:03.5351888Z pass 2024-08-20T23:41:03.5464219Z TIMING: entire_frame_compile:86.23013 code_gen:44.29873 inductor_compile:68.00911 backend_compile:70.23864 total_wall_time:110.04464 2024-08-20T23:41:03.5465971Z STATS: call_* op count: 1282 | FakeTensorMode.__torch_dispatch__:71999 | FakeTensor.__torch_dispatch__:21124 | ProxyTorchDispatchMode.__torch_dispatch__:32632 2024-08-20T23:41:03.5469917Z Dynamo produced 3 graphs covering 1282 ops with 7 graph breaks (5 unique) 2024-08-20T23:41:12.0412613Z 2024-08-20T23:41:15.5265107Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:41:15.5265717Z loading model: 0it [00:03, ?it/s] 2024-08-20T23:41:15.5266278Z cuda train xcit_large_24_p8_224 2024-08-20T23:41:19.8380545Z WARNING:common:fp64 golden ref were not generated for xcit_large_24_p8_224. Setting accuracy check to cosine 2024-08-20T23:43:18.7354840Z W0820 23:43:18.734000 41119 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:46:00.2870339Z pass_due_to_skip 2024-08-20T23:46:00.3299724Z TIMING: entire_frame_compile:219.10596 code_gen:95.2575 inductor_compile:155.6783 backend_compile:173.24407 total_wall_time:260.77772 2024-08-20T23:46:00.3301654Z STATS: call_* op count: 3042 | FakeTensorMode.__torch_dispatch__:173695 | FakeTensor.__torch_dispatch__:57343 | ProxyTorchDispatchMode.__torch_dispatch__:78438 2024-08-20T23:46:00.3303010Z Dynamo produced 3 graphs covering 3042 ops with 7 graph breaks (5 unique) 2024-08-20T23:46:12.7123053Z accuracy pass_rate=96.77% 2024-08-20T23:46:12.7126410Z calls_captured gmean=691.18x mean=887.871x 2024-08-20T23:46:12.7129652Z unique_graphs gmean=2.49x mean=2.548x 2024-08-20T23:46:12.7133184Z graph_breaks gmean=6.39x mean=6.419x 2024-08-20T23:46:12.7137141Z unique_graph_breaks gmean=4.96x mean=4.968x 2024-08-20T23:46:12.7141040Z autograd_captures gmean=0.00x mean=0.000x 2024-08-20T23:46:12.7144711Z autograd_compiles gmean=0.00x mean=0.000x 2024-08-20T23:46:12.7148336Z cudagraph_skips gmean=0.00x mean=0.097x 2024-08-20T23:46:15.7046475Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_timm_training.csv 2024-08-20T23:46:15.9670516Z mixnet_l PASS 2024-08-20T23:46:15.9674953Z mnasnet_100 PASS 2024-08-20T23:46:15.9680005Z mobilenetv2_100 PASS 2024-08-20T23:46:15.9684830Z mobilenetv3_large_100 PASS 2024-08-20T23:46:15.9689705Z mobilevit_s PASS 2024-08-20T23:46:15.9694547Z nfnet_l0 PASS 2024-08-20T23:46:15.9699805Z pit_b_224 PASS 2024-08-20T23:46:15.9704480Z pnasnet5large PASS 2024-08-20T23:46:15.9709153Z poolformer_m36 PASS 2024-08-20T23:46:15.9714008Z regnety_002 PASS 2024-08-20T23:46:15.9718708Z repvgg_a2 PASS 2024-08-20T23:46:15.9723441Z res2net101_26w_4s PASS 2024-08-20T23:46:15.9728082Z res2net50_14w_8s PASS 2024-08-20T23:46:15.9732789Z res2next50 PASS 2024-08-20T23:46:15.9737729Z resmlp_12_224 PASS 2024-08-20T23:46:15.9742603Z resnest101e PASS 2024-08-20T23:46:15.9747326Z rexnet_100 PASS 2024-08-20T23:46:15.9752110Z sebotnet33ts_256 PASS 2024-08-20T23:46:15.9758986Z selecsls42b PASS 2024-08-20T23:46:15.9763725Z spnasnet_100 PASS 2024-08-20T23:46:15.9768318Z swin_base_patch4_window7_224 PASS 2024-08-20T23:46:15.9773043Z swsl_resnext101_32x16d PASS 2024-08-20T23:46:15.9778058Z tf_efficientnet_b0 PASS 2024-08-20T23:46:15.9782780Z tf_mixnet_l PASS 2024-08-20T23:46:15.9787370Z tinynet_a PASS 2024-08-20T23:46:15.9792227Z tnt_s_patch16_224 PASS 2024-08-20T23:46:15.9796842Z twins_pcpvt_base PASS 2024-08-20T23:46:15.9801668Z visformer_small PASS 2024-08-20T23:46:15.9806301Z vit_base_patch16_224 PASS 2024-08-20T23:46:15.9811080Z volo_d1_224 PASS 2024-08-20T23:46:15.9815921Z xcit_large_24_p8_224 XFAIL 2024-08-20T23:46:16.0244401Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_timm_training.csv 2024-08-20T23:46:16.2863567Z mixnet_l PASS 2024-08-20T23:46:16.2867987Z mnasnet_100 PASS 2024-08-20T23:46:16.2872818Z mobilenetv2_100 PASS 2024-08-20T23:46:16.2878123Z mobilenetv3_large_100 PASS 2024-08-20T23:46:16.2883197Z mobilevit_s PASS 2024-08-20T23:46:16.2888247Z nfnet_l0 PASS 2024-08-20T23:46:16.2893645Z pit_b_224 PASS 2024-08-20T23:46:16.2898924Z pnasnet5large PASS 2024-08-20T23:46:16.2903467Z poolformer_m36 PASS 2024-08-20T23:46:16.2908666Z regnety_002 PASS 2024-08-20T23:46:16.2913296Z repvgg_a2 PASS 2024-08-20T23:46:16.2918241Z res2net101_26w_4s PASS 2024-08-20T23:46:16.2923118Z res2net50_14w_8s PASS 2024-08-20T23:46:16.2927963Z res2next50 PASS 2024-08-20T23:46:16.2932986Z resmlp_12_224 PASS 2024-08-20T23:46:16.2938124Z resnest101e PASS 2024-08-20T23:46:16.2943118Z rexnet_100 PASS 2024-08-20T23:46:16.2948096Z sebotnet33ts_256 PASS 2024-08-20T23:46:16.2953528Z selecsls42b PASS 2024-08-20T23:46:16.2958650Z spnasnet_100 PASS 2024-08-20T23:46:16.2963742Z swin_base_patch4_window7_224 PASS 2024-08-20T23:46:16.2968600Z swsl_resnext101_32x16d PASS 2024-08-20T23:46:16.2973743Z tf_efficientnet_b0 PASS 2024-08-20T23:46:16.2979090Z tf_mixnet_l PASS 2024-08-20T23:46:16.2983964Z tinynet_a PASS 2024-08-20T23:46:16.2988933Z tnt_s_patch16_224 PASS 2024-08-20T23:46:16.2993766Z twins_pcpvt_base PASS 2024-08-20T23:46:16.2998933Z visformer_small PASS 2024-08-20T23:46:16.3003793Z vit_base_patch16_224 PASS 2024-08-20T23:46:16.3008894Z volo_d1_224 PASS 2024-08-20T23:46:16.3013765Z xcit_large_24_p8_224 PASS 2024-08-20T23:46:16.3444335Z + cleanup_workspace 2024-08-20T23:46:16.3445264Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-20T23:46:16.3446348Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-20T23:46:16.3447277Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-20T23:46:16.3447924Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-20T23:46:16.3448748Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-20T23:46:16.3449637Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-20T23:46:16.3450342Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-20T23:46:17.0803310Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-20T23:46:17.0803827Z cat test/**/*_toprint.log || true 2024-08-20T23:46:17.0817106Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:46:17.0817597Z env: 2024-08-20T23:46:17.0817888Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:17.0818347Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:17.0819084Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:17.0819721Z ##[endgroup] 2024-08-20T23:46:17.0916053Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-20T23:46:17.0949016Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:46:17.0949510Z kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:46:17.0958498Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:46:17.0958989Z env: 2024-08-20T23:46:17.0959275Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:17.0959736Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:17.0960482Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:17.0961138Z MONITOR_SCRIPT_PID: 52510 2024-08-20T23:46:17.0961512Z ##[endgroup] 2024-08-20T23:46:17.1156656Z Prepare all required actions 2024-08-20T23:46:17.1157105Z Getting action download info 2024-08-20T23:46:17.3202299Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-20T23:46:17.4754005Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-20T23:46:17.4754480Z with: 2024-08-20T23:46:17.4755028Z file-suffix: test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026528125 2024-08-20T23:46:17.4755706Z s3-bucket: gha-artifacts 2024-08-20T23:46:17.4756242Z env: 2024-08-20T23:46:17.4756531Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:17.4756998Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:17.4757751Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:17.4758394Z ##[endgroup] 2024-08-20T23:46:17.4785139Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-20T23:46:17.4785801Z # Remove any previous test jsons if they exist 2024-08-20T23:46:17.4786327Z rm -f test-jsons-*.zip 2024-08-20T23:46:17.4786864Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-20T23:46:17.4795965Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:46:17.4796465Z env: 2024-08-20T23:46:17.4796753Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:17.4797216Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:17.4797965Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:17.4798878Z FILE_SUFFIX: test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026528125 2024-08-20T23:46:17.4799530Z ##[endgroup] 2024-08-20T23:46:17.5160723Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-20T23:46:17.5190000Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-20T23:46:17.5190700Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-20T23:46:17.5196900Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-20T23:46:17.5202586Z adding: test/slow_tests.json (deflated 82%) 2024-08-20T23:46:17.5235358Z ##[group]Run # Remove any previous test reports if they exist 2024-08-20T23:46:17.5236011Z # Remove any previous test reports if they exist 2024-08-20T23:46:17.5236552Z rm -f test-reports-*.zip 2024-08-20T23:46:17.5237155Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-20T23:46:17.5246067Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:46:17.5246590Z env: 2024-08-20T23:46:17.5246870Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:17.5247333Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:17.5248073Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:17.5248985Z FILE_SUFFIX: test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026528125 2024-08-20T23:46:17.5249636Z ##[endgroup] 2024-08-20T23:46:17.5517048Z adding: test/test-reports/inference_timm_models.csv (deflated 69%) 2024-08-20T23:46:17.5518540Z adding: test/test-reports/training_timm_models.csv (deflated 66%) 2024-08-20T23:46:17.5522919Z adding: test/test-reports/training_timm_model_graph_breaks.csv (deflated 97%) 2024-08-20T23:46:17.5524799Z adding: test/test-reports/training_timm_model_graph_break_deduped.csv (deflated 74%) 2024-08-20T23:46:17.5556762Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-20T23:46:17.5557418Z # Remove any previous usage logs if they exist 2024-08-20T23:46:17.5557923Z rm -f logs-*.zip 2024-08-20T23:46:17.5558601Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-20T23:46:17.5559396Z # so check to see if the file exists first 2024-08-20T23:46:17.5559915Z if [ -f 'usage_log.txt' ]; then 2024-08-20T23:46:17.5560658Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-20T23:46:17.5561161Z fi 2024-08-20T23:46:17.5561538Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-20T23:46:17.5562127Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-20T23:46:17.5562623Z fi 2024-08-20T23:46:17.5571466Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:46:17.5571969Z env: 2024-08-20T23:46:17.5572261Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:17.5572717Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:17.5573652Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:17.5574635Z FILE_SUFFIX: test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026528125 2024-08-20T23:46:17.5575297Z ##[endgroup] 2024-08-20T23:46:17.5666519Z adding: usage_log.txt (deflated 92%) 2024-08-20T23:46:17.5756241Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-20T23:46:17.5756988Z # Remove any previous debugging artifacts if they exist 2024-08-20T23:46:17.5757615Z rm -f debug-*.zip 2024-08-20T23:46:17.5758019Z if [ -d 'test/debug' ]; then 2024-08-20T23:46:17.5758543Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-20T23:46:17.5759033Z fi 2024-08-20T23:46:17.5767381Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:46:17.5767939Z env: 2024-08-20T23:46:17.5768229Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:17.5768708Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:17.5769486Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:17.5770402Z FILE_SUFFIX: test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026528125 2024-08-20T23:46:17.5771068Z ##[endgroup] 2024-08-20T23:46:17.5869660Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:46:17.5870146Z with: 2024-08-20T23:46:17.5870438Z s3-bucket: gha-artifacts 2024-08-20T23:46:17.5870931Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:46:17.5871448Z retention-days: 14 2024-08-20T23:46:17.5871812Z if-no-files-found: warn 2024-08-20T23:46:17.5872188Z path: test-jsons-*.zip 2024-08-20T23:46:17.5872558Z name: artifact 2024-08-20T23:46:17.5872880Z region: us-east-1 2024-08-20T23:46:17.5873208Z env: 2024-08-20T23:46:17.5873488Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:17.5873973Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:17.5874718Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:17.5875385Z ##[endgroup] 2024-08-20T23:46:17.9416007Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:46:17.9417041Z With the provided path, there will be 1 file uploaded 2024-08-20T23:46:17.9417736Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:46:18.0617311Z Starting upload of test-jsons-test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026528125.zip 2024-08-20T23:46:18.1909254Z Finished upload of test-jsons-test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026528125.zip 2024-08-20T23:46:18.2186376Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:46:18.2186824Z with: 2024-08-20T23:46:18.2187115Z s3-bucket: gha-artifacts 2024-08-20T23:46:18.2187557Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:46:18.2188033Z retention-days: 14 2024-08-20T23:46:18.2188375Z if-no-files-found: error 2024-08-20T23:46:18.2188776Z path: test-reports-*.zip 2024-08-20T23:46:18.2189134Z name: artifact 2024-08-20T23:46:18.2189440Z region: us-east-1 2024-08-20T23:46:18.2189752Z env: 2024-08-20T23:46:18.2190035Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:18.2190499Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:18.2191228Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:18.2191871Z ##[endgroup] 2024-08-20T23:46:18.5703748Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:46:18.5704501Z With the provided path, there will be 1 file uploaded 2024-08-20T23:46:18.5705138Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:46:18.5760749Z Starting upload of test-reports-test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026528125.zip 2024-08-20T23:46:18.6720749Z Finished upload of test-reports-test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026528125.zip 2024-08-20T23:46:18.6997719Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:46:18.6998360Z with: 2024-08-20T23:46:18.6998649Z s3-bucket: gha-artifacts 2024-08-20T23:46:18.6999105Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:46:18.6999592Z retention-days: 14 2024-08-20T23:46:18.6999952Z if-no-files-found: ignore 2024-08-20T23:46:18.7000317Z path: logs-*.zip 2024-08-20T23:46:18.7000640Z name: artifact 2024-08-20T23:46:18.7000960Z region: us-east-1 2024-08-20T23:46:18.7001294Z env: 2024-08-20T23:46:18.7001578Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:18.7002052Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:18.7002804Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:18.7003466Z ##[endgroup] 2024-08-20T23:46:19.0522073Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:46:19.0523407Z With the provided path, there will be 1 file uploaded 2024-08-20T23:46:19.0524698Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:46:19.0579532Z Starting upload of logs-test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026528125.zip 2024-08-20T23:46:19.1979112Z Finished upload of logs-test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026528125.zip 2024-08-20T23:46:19.2257203Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:46:19.2257655Z with: 2024-08-20T23:46:19.2257983Z s3-bucket: gha-artifacts 2024-08-20T23:46:19.2258464Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:46:19.2258941Z retention-days: 14 2024-08-20T23:46:19.2259287Z if-no-files-found: ignore 2024-08-20T23:46:19.2259648Z path: debug-*.zip 2024-08-20T23:46:19.2259969Z name: artifact 2024-08-20T23:46:19.2260284Z region: us-east-1 2024-08-20T23:46:19.2260581Z env: 2024-08-20T23:46:19.2260868Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:19.2261327Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:19.2262078Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:19.2262729Z ##[endgroup] 2024-08-20T23:46:19.5770375Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-20T23:46:19.6046020Z ##[group]Run # shellcheck disable=SC2156 2024-08-20T23:46:19.6046506Z # shellcheck disable=SC2156 2024-08-20T23:46:19.6047299Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-20T23:46:19.6057379Z shell: /usr/bin/bash -e {0} 2024-08-20T23:46:19.6057827Z env: 2024-08-20T23:46:19.6058187Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:19.6058775Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:19.6059579Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:19.6060225Z ##[endgroup] 2024-08-20T23:46:19.8675807Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-20T23:46:19.8676359Z with: 2024-08-20T23:46:19.8676642Z env: 2024-08-20T23:46:19.8676922Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:19.8677380Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:19.8678110Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:19.8678750Z ##[endgroup] 2024-08-20T23:46:19.8697639Z ##[group]Run set -eou pipefail 2024-08-20T23:46:19.8698079Z set -eou pipefail 2024-08-20T23:46:19.8698471Z  2024-08-20T23:46:19.8698991Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-20T23:46:19.8699631Z for _ in $(seq 1440); do 2024-08-20T23:46:19.8700122Z  # Break if no ssh session exists anymore 2024-08-20T23:46:19.8700643Z  if [ "$(who)" = "" ]; then 2024-08-20T23:46:19.8701068Z  break 2024-08-20T23:46:19.8701434Z  fi 2024-08-20T23:46:19.8701744Z  echo "." 2024-08-20T23:46:19.8702092Z  sleep 5 2024-08-20T23:46:19.8702607Z done 2024-08-20T23:46:19.8711631Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:46:19.8712130Z env: 2024-08-20T23:46:19.8712422Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:19.8712895Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:19.8713655Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:19.8714292Z ##[endgroup] 2024-08-20T23:46:19.8744212Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-20T23:46:19.8875643Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:46:19.8876405Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:46:19.8877017Z # shellcheck disable=SC2046 2024-08-20T23:46:19.8877508Z docker stop $(docker ps -q) || true 2024-08-20T23:46:19.8878004Z # Prune all of the docker images 2024-08-20T23:46:19.8878514Z docker system prune -af 2024-08-20T23:46:19.8886574Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:46:19.8887092Z env: 2024-08-20T23:46:19.8887384Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:46:19.8887843Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:46:19.8888590Z DOCKER_CONTAINER_ID: ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:19.8889244Z ##[endgroup] 2024-08-20T23:46:20.3665404Z ff7d2001e8e4 2024-08-20T23:46:21.8166049Z Deleted Containers: 2024-08-20T23:46:21.8166613Z ff7d2001e8e4ced436a4584ab63b44cdb98f9483118b8b8e364a6edbda08cafe 2024-08-20T23:46:21.8167059Z 2024-08-20T23:46:33.0148074Z Deleted Images: 2024-08-20T23:46:33.0151693Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T23:46:33.0154514Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-20T23:46:33.0156173Z deleted: sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb 2024-08-20T23:46:33.0157048Z deleted: sha256:c2ce10014dd98f6abbfbf2265f9780a8204c2a7a27463e61ee565573a23d141d 2024-08-20T23:46:33.0157915Z deleted: sha256:ae249c0e6e6013bd6ffea2ca0b0160c015c9ebf7c03070bc2f983a9ff2415bf5 2024-08-20T23:46:33.0158799Z deleted: sha256:8a73883c1067c0e29ff128f553518682b47ed9707d2f35e7d5138e2caf029d61 2024-08-20T23:46:33.0159633Z deleted: sha256:9c040db729fa084584aaf5573082408a2196bc64e5952270c79dc610ed4680b6 2024-08-20T23:46:33.0160459Z deleted: sha256:17311eeeb8b71b16699880c0f646e185107496c4d41da5d23370603768ec030c 2024-08-20T23:46:33.0161362Z deleted: sha256:07cc10bece8501b1c8035d4385ad9e9867dd6e92fa32218a4dcb3e42af7e0411 2024-08-20T23:46:33.0162589Z deleted: sha256:f3e32d05099d1a3115a4722bc2e94a48752c3e965446773673808252671bb967 2024-08-20T23:46:33.0163462Z deleted: sha256:be7c3f41475e3e72ffe3ac2f92ab2f4cf49d5496b2aae738942ac3bb687d742c 2024-08-20T23:46:33.0164453Z deleted: sha256:4fcac490be825129b468cf458089f1fcf8fd56afc44ef4ab27c2eff41ab1ac90 2024-08-20T23:46:33.0165383Z deleted: sha256:31c66d80a2a4792897f267d363c0cf84fb821c5d4a275ab1636f778fd0284c08 2024-08-20T23:46:33.0166202Z deleted: sha256:8a8bcb6085e7e438e5533e8a5613e2ae5888543853568d023af3e2623a1593a9 2024-08-20T23:46:33.0167029Z deleted: sha256:2f6cb60029c79d11567b738f5bdbfb9cb1838cc0f194c4a5e6ae685178d2ed34 2024-08-20T23:46:33.0167864Z deleted: sha256:e546391804d3313f558e1ac16e93cb29a417a92e4eccaf0d4bd12af3bbcf39e3 2024-08-20T23:46:33.0168698Z deleted: sha256:1623b2aa16f2bdc6b572f6799498d9a92fa8114d07c440599b17aea72e22f8b1 2024-08-20T23:46:33.0169535Z deleted: sha256:c0386ca58b7cb7afee015b7e58d2f70525eaf73d6da2cc63868b20e14b1f9497 2024-08-20T23:46:33.0170380Z deleted: sha256:dcbf534ff289186e246531925fed90f72516eb3d63663bad234d611cdc025d62 2024-08-20T23:46:33.0171207Z deleted: sha256:0bab537af579f56c9cd51c7073fd53635fffb424b363236e0de38a7dd0e60ae8 2024-08-20T23:46:33.0172249Z deleted: sha256:d85a852ae1ee027e2fc17240ed68b87957b0a6d210c9119752d8979a0dcd55cf 2024-08-20T23:46:33.0173070Z deleted: sha256:f3821d761b555362e44cf9a1447d8fd2d95b37d54aea67c7519cd9e5db0c4d7e 2024-08-20T23:46:33.0173907Z deleted: sha256:c05e84a545db5dfcb0847c73d2b0f4749fa67aabce05705632ff7f76c84d4688 2024-08-20T23:46:33.0174868Z deleted: sha256:27054eae7ca67b2bd5ca0fd3f4488bf5c5f78e95487d8c45616f3d753d48a8b5 2024-08-20T23:46:33.0175704Z deleted: sha256:9804139f6ae97796aaa993fe4b89ea0cf111eb4d30a79f4c87c339c2e1e04fa0 2024-08-20T23:46:33.0176537Z deleted: sha256:ad6ae2fc594aaa359ff4e287323836570454eb24dd162930655e414d1d77c6dd 2024-08-20T23:46:33.0177370Z deleted: sha256:9cfbb36e328af52a248853a46aab88e266c09e2b44a03e0a42194a031b9b9453 2024-08-20T23:46:33.0178267Z deleted: sha256:a8a842bd7f37b8d50b6d12d6f95f46bbba57953db466fcbde9ba522625f024ef 2024-08-20T23:46:33.0179110Z deleted: sha256:da9738654b3c86f89ebeed2560f1bae093f1219f1eff90b0394b7f211ecabd04 2024-08-20T23:46:33.0179951Z deleted: sha256:cc683bf0cb245394e0f568984b2dd4fce1b6a3a7fe7c44734bae1aa1caaa3251 2024-08-20T23:46:33.0180783Z deleted: sha256:965f220e71c7d1ec19aaf41b1529ab287beb9d86eb8482c4c14094485ab34475 2024-08-20T23:46:33.0181614Z deleted: sha256:da32ecec41a73ce7dd29464f90c7951510a18cd378753659a78cd7fabd10dc36 2024-08-20T23:46:33.0182450Z deleted: sha256:d828d187cee9ad48dbc3702fb61d359619dd8a1ed582878680b51686bf117e5a 2024-08-20T23:46:33.0183278Z deleted: sha256:712111f4b12ff8e2787597d5fdcb043bd0824e55ddd97f262ef550506ca195f0 2024-08-20T23:46:33.0184096Z deleted: sha256:a064eea405958ad14dc6c794e5df9fd1d4515424272801ec559d45a59317309a 2024-08-20T23:46:33.0184920Z deleted: sha256:e124382d136bd5e9a8b409ca79c1f1ef76458deaf444b1b2060d3fa99dbeb506 2024-08-20T23:46:33.0185747Z deleted: sha256:6e10fe757cc190a1b31b47010f4f7610e02cb1b4b438f190dcd301aa4fe2278e 2024-08-20T23:46:33.0186574Z deleted: sha256:4cea352e10f46ca4729f96ac6c4600fc53d4c36e7658c2e0f19f4f309cd4a2a7 2024-08-20T23:46:33.0187412Z deleted: sha256:518bba00fe1d0933053066d21cac0235c9fb485abbc41557ec184b293095c924 2024-08-20T23:46:33.0188243Z deleted: sha256:a64f9ca42e9f27ff2690c15a5c84312e6efaccad45668d9b496cb542165cfb5a 2024-08-20T23:46:33.0189140Z deleted: sha256:096ea0fa14a6c420658f5276b8a8f5d264c0de7e18161ceda2cfc2f4a92cd9bf 2024-08-20T23:46:33.0190019Z deleted: sha256:6f91c94ce6e3f8034c91c66ee25c66a3d85dba9d813b749bd6d2000942a99edc 2024-08-20T23:46:33.0190864Z deleted: sha256:8f8d255eeae546cead8f9a94d237fa57a99940c0f5d51bc29a9d41c7df9cffd9 2024-08-20T23:46:33.0191745Z deleted: sha256:fe05276f4a49f91eef56bfbc3da87cfff5eb116a827fa2d30efe42ce1c134b40 2024-08-20T23:46:33.0192601Z deleted: sha256:5f83cd75d0669c813d4f58978c92e14305cd8f6a07eb9a60665a583460fa7b59 2024-08-20T23:46:33.0193418Z deleted: sha256:8f7ab6366169d39d8953129f650f1927d2fb1328b3f11e2fd9076adb5f46b93a 2024-08-20T23:46:33.0194381Z deleted: sha256:4146cfd5dba0f5308954749f98596b14a1d8f6d23bd93f7ad86c6c3d72239328 2024-08-20T23:46:33.0195231Z deleted: sha256:c6d83cb9ccc606ff63bb3c4d5a5e1ff1450c619b5aae494303a94ae25971cca0 2024-08-20T23:46:33.0196073Z deleted: sha256:71e037e0a3e0963c4565534a7387c082afcb04475d3ac629acf7a0ab2f49fadc 2024-08-20T23:46:33.0196883Z deleted: sha256:4b31135c564686bdaf23262b901237880aea635655721cc7e8c95dc42b05e06d 2024-08-20T23:46:33.0197704Z deleted: sha256:d09bd5c2f35cedcb846543941ee64ce1f684b52e4285696c93b3fb2032da8579 2024-08-20T23:46:33.0198547Z deleted: sha256:4de7fb7d4fc22212aae5fc1ceb637526916d0cced240d7c93d5edd6b0ea28013 2024-08-20T23:46:33.0199379Z deleted: sha256:69890e3e82746e10aff2f1084f11afb4b95a4a6c26b30f64b7001bc0a5fe32bf 2024-08-20T23:46:33.0200188Z deleted: sha256:0156df53b3224e645336ea52b4bb816f1e8d8c9b591748cd645e5e5649fa507b 2024-08-20T23:46:33.0201005Z deleted: sha256:0664624cb9cc4a386f31bf49919a8d8a11d9fea428d05585d9e6228f754bc7cd 2024-08-20T23:46:33.0201843Z deleted: sha256:8c2ddd5fa535c768c058a23f5cc4b6c64cdd71bb7458491282aabc7f7bb94d69 2024-08-20T23:46:33.0202695Z deleted: sha256:8dd03dbe0e1c5f47dd333074dff4bd00415f2f63727a65bf17866774e2ebbd0d 2024-08-20T23:46:33.0203622Z deleted: sha256:3c56b63e094bd18fad9da7a541d537210bca3b730c590d3641213259f6908d6f 2024-08-20T23:46:33.0204457Z deleted: sha256:51484fef07bd1266d4d6ff3d767751172660fc4cc6a1ffe883523a63c985e31e 2024-08-20T23:46:33.0205290Z deleted: sha256:03e83c94023982ab7dd8dc65a6df2024e342f1b760c91ec7a680ee45824869c7 2024-08-20T23:46:33.0206125Z deleted: sha256:58440ab3e8175b5de94a76485c8689a6dc186015475ebf4b386ebba309c5da34 2024-08-20T23:46:33.0206952Z deleted: sha256:9a980f3383e8aa511f0ef68982cb6dfc53b38d1ac8c3d673cb3ba3d45158c91e 2024-08-20T23:46:33.0207802Z deleted: sha256:cb0269c9cf1c47b6c4188ca7be4db261b5d3fc4b4a5153629e222dba7b1f9f98 2024-08-20T23:46:33.0208659Z deleted: sha256:41f1ba870cdffc05e0bf8d87b640fb9d4c153d7f4f3b775f37f94e9f3d584aad 2024-08-20T23:46:33.0209514Z deleted: sha256:7b502ab352f48e8b6eb5e6824a29f308952a5ee7c7d987562e1df7e47639d7f1 2024-08-20T23:46:33.0210341Z deleted: sha256:6ba5217e18738660e6ca8b7a7ec88845e70b0fd781301520bf46c71d99df3144 2024-08-20T23:46:33.0211189Z deleted: sha256:dd5a1c233594780163726bdbbf5de00f7b6b8dca1dace8a001340714c54baedb 2024-08-20T23:46:33.0212032Z deleted: sha256:c27f0db27da73e01f67f6cb78d10f355e350d829479ba4351d82055bb29f6597 2024-08-20T23:46:33.0212877Z deleted: sha256:4e3ce3c854f35c08be3dd95e0d6e718512a885e4614291d7d8b7cd23f48b7dd8 2024-08-20T23:46:33.0213721Z deleted: sha256:c2d6fb088bbec16e60cf0ad74b016f7287c92e659ec2db01271bc309b4dc6d83 2024-08-20T23:46:33.0214664Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-08-20T23:46:33.0215501Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-08-20T23:46:33.0216311Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-08-20T23:46:33.0217133Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-08-20T23:46:33.0217968Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-08-20T23:46:33.0218795Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-08-20T23:46:33.0219639Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-08-20T23:46:33.0220485Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-08-20T23:46:33.0221325Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-08-20T23:46:33.0222171Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-08-20T23:46:33.0222996Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-08-20T23:46:33.0223491Z 2024-08-20T23:46:33.0223652Z Total reclaimed space: 44.87GB 2024-08-20T23:46:33.0290799Z Post job cleanup. 2024-08-20T23:46:33.0341297Z Post job cleanup. 2024-08-20T23:46:33.1207239Z [command]/usr/bin/git version 2024-08-20T23:46:33.1255526Z git version 2.40.1 2024-08-20T23:46:33.1294101Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/f5fd9b87-6322-447d-9ea3-823f8c6c0f42' before making global git config changes 2024-08-20T23:46:33.1295434Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T23:46:33.1299388Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T23:46:33.1342752Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T23:46:33.1382079Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T23:46:33.1738930Z Entering 'android/libs/fbjni' 2024-08-20T23:46:33.1808606Z Entering 'third_party/FP16' 2024-08-20T23:46:33.1879939Z Entering 'third_party/FXdiv' 2024-08-20T23:46:33.1949586Z Entering 'third_party/NNPACK' 2024-08-20T23:46:33.2020058Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:46:33.2090255Z Entering 'third_party/XNNPACK' 2024-08-20T23:46:33.2180023Z Entering 'third_party/benchmark' 2024-08-20T23:46:33.2248968Z Entering 'third_party/cpp-httplib' 2024-08-20T23:46:33.2317657Z Entering 'third_party/cpuinfo' 2024-08-20T23:46:33.2388446Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:46:33.2459320Z Entering 'third_party/cutlass' 2024-08-20T23:46:33.2535940Z Entering 'third_party/eigen' 2024-08-20T23:46:33.2605599Z Entering 'third_party/fbgemm' 2024-08-20T23:46:33.2673209Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:46:33.2737851Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:46:33.2805769Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:46:33.2878522Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:46:33.2943863Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:46:33.3012838Z Entering 'third_party/flatbuffers' 2024-08-20T23:46:33.3086447Z Entering 'third_party/fmt' 2024-08-20T23:46:33.3158767Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:46:33.3227693Z Entering 'third_party/gloo' 2024-08-20T23:46:33.3298500Z Entering 'third_party/googletest' 2024-08-20T23:46:33.3368344Z Entering 'third_party/ideep' 2024-08-20T23:46:33.3434708Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:46:33.3508693Z Entering 'third_party/ittapi' 2024-08-20T23:46:33.3579012Z Entering 'third_party/kineto' 2024-08-20T23:46:33.3646738Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:46:33.3714440Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:46:33.3783820Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:46:33.3851120Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:46:33.3918560Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:46:33.3985172Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:46:33.4055191Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:46:33.4127537Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:46:33.4195590Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:46:33.4264152Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:46:33.4333866Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:46:33.4398988Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:46:33.4469602Z Entering 'third_party/mimalloc' 2024-08-20T23:46:33.4542611Z Entering 'third_party/nccl/nccl' 2024-08-20T23:46:33.4611073Z Entering 'third_party/nlohmann' 2024-08-20T23:46:33.4679242Z Entering 'third_party/onnx' 2024-08-20T23:46:33.4761852Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:46:33.4829637Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:46:33.4903001Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:46:33.4971169Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:46:33.5046519Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:46:33.5112498Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:46:33.5178357Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:46:33.5248563Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:46:33.5317283Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:46:33.5384706Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:46:33.5448503Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:46:33.5517395Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:46:33.5586899Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:46:33.5674408Z Entering 'third_party/pocketfft' 2024-08-20T23:46:33.5741833Z Entering 'third_party/protobuf' 2024-08-20T23:46:33.5811630Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:46:33.5877717Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:46:33.5947388Z Entering 'third_party/psimd' 2024-08-20T23:46:33.6017706Z Entering 'third_party/pthreadpool' 2024-08-20T23:46:33.6086482Z Entering 'third_party/pybind11' 2024-08-20T23:46:33.6157337Z Entering 'third_party/python-peachpy' 2024-08-20T23:46:33.6226168Z Entering 'third_party/sleef' 2024-08-20T23:46:33.6296041Z Entering 'third_party/tensorpipe' 2024-08-20T23:46:33.6363499Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:46:33.6429684Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:46:33.6495987Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:46:33.6563308Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:46:33.6626733Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:46:33.6719916Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T23:46:33.6757172Z http.https://github.com/.extraheader 2024-08-20T23:46:33.6767290Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-20T23:46:33.6813281Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T23:46:33.7181668Z Entering 'android/libs/fbjni' 2024-08-20T23:46:33.7224774Z http.https://github.com/.extraheader 2024-08-20T23:46:33.7269540Z Entering 'third_party/FP16' 2024-08-20T23:46:33.7312673Z http.https://github.com/.extraheader 2024-08-20T23:46:33.7360239Z Entering 'third_party/FXdiv' 2024-08-20T23:46:33.7403130Z http.https://github.com/.extraheader 2024-08-20T23:46:33.7446746Z Entering 'third_party/NNPACK' 2024-08-20T23:46:33.7491654Z http.https://github.com/.extraheader 2024-08-20T23:46:33.7536043Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:46:33.7579781Z http.https://github.com/.extraheader 2024-08-20T23:46:33.7626260Z Entering 'third_party/XNNPACK' 2024-08-20T23:46:33.7670108Z http.https://github.com/.extraheader 2024-08-20T23:46:33.7730727Z Entering 'third_party/benchmark' 2024-08-20T23:46:33.7774070Z http.https://github.com/.extraheader 2024-08-20T23:46:33.7817298Z Entering 'third_party/cpp-httplib' 2024-08-20T23:46:33.7862510Z http.https://github.com/.extraheader 2024-08-20T23:46:33.7906032Z Entering 'third_party/cpuinfo' 2024-08-20T23:46:33.7949156Z http.https://github.com/.extraheader 2024-08-20T23:46:33.7993011Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:46:33.8035752Z http.https://github.com/.extraheader 2024-08-20T23:46:33.8080405Z Entering 'third_party/cutlass' 2024-08-20T23:46:33.8124001Z http.https://github.com/.extraheader 2024-08-20T23:46:33.8175402Z Entering 'third_party/eigen' 2024-08-20T23:46:33.8218027Z http.https://github.com/.extraheader 2024-08-20T23:46:33.8263994Z Entering 'third_party/fbgemm' 2024-08-20T23:46:33.8307406Z http.https://github.com/.extraheader 2024-08-20T23:46:33.8350375Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:46:33.8394287Z http.https://github.com/.extraheader 2024-08-20T23:46:33.8438058Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:46:33.8481306Z http.https://github.com/.extraheader 2024-08-20T23:46:33.8525597Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:46:33.8569865Z http.https://github.com/.extraheader 2024-08-20T23:46:33.8619159Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:46:33.8662956Z http.https://github.com/.extraheader 2024-08-20T23:46:33.8706520Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:46:33.8748742Z http.https://github.com/.extraheader 2024-08-20T23:46:33.8797486Z Entering 'third_party/flatbuffers' 2024-08-20T23:46:33.8840713Z http.https://github.com/.extraheader 2024-08-20T23:46:33.8892450Z Entering 'third_party/fmt' 2024-08-20T23:46:33.8936342Z http.https://github.com/.extraheader 2024-08-20T23:46:33.8980442Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:46:33.9024421Z http.https://github.com/.extraheader 2024-08-20T23:46:33.9068957Z Entering 'third_party/gloo' 2024-08-20T23:46:33.9113332Z http.https://github.com/.extraheader 2024-08-20T23:46:33.9157894Z Entering 'third_party/googletest' 2024-08-20T23:46:33.9201406Z http.https://github.com/.extraheader 2024-08-20T23:46:33.9249044Z Entering 'third_party/ideep' 2024-08-20T23:46:33.9293847Z http.https://github.com/.extraheader 2024-08-20T23:46:33.9335856Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:46:33.9378603Z http.https://github.com/.extraheader 2024-08-20T23:46:33.9429951Z Entering 'third_party/ittapi' 2024-08-20T23:46:33.9474519Z http.https://github.com/.extraheader 2024-08-20T23:46:33.9518246Z Entering 'third_party/kineto' 2024-08-20T23:46:33.9562957Z http.https://github.com/.extraheader 2024-08-20T23:46:33.9607339Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:46:33.9649607Z http.https://github.com/.extraheader 2024-08-20T23:46:33.9691879Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:46:33.9739667Z http.https://github.com/.extraheader 2024-08-20T23:46:33.9785682Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:46:33.9827737Z http.https://github.com/.extraheader 2024-08-20T23:46:33.9872172Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:46:33.9914413Z http.https://github.com/.extraheader 2024-08-20T23:46:33.9959719Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:46:34.0001579Z http.https://github.com/.extraheader 2024-08-20T23:46:34.0044824Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:46:34.0093019Z http.https://github.com/.extraheader 2024-08-20T23:46:34.0142919Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:46:34.0186721Z http.https://github.com/.extraheader 2024-08-20T23:46:34.0231401Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:46:34.0275115Z http.https://github.com/.extraheader 2024-08-20T23:46:34.0319884Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:46:34.0363219Z http.https://github.com/.extraheader 2024-08-20T23:46:34.0408853Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:46:34.0451985Z http.https://github.com/.extraheader 2024-08-20T23:46:34.0502896Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:46:34.0549869Z http.https://github.com/.extraheader 2024-08-20T23:46:34.0593583Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:46:34.0636155Z http.https://github.com/.extraheader 2024-08-20T23:46:34.0685065Z Entering 'third_party/mimalloc' 2024-08-20T23:46:34.0728276Z http.https://github.com/.extraheader 2024-08-20T23:46:34.0773276Z Entering 'third_party/nccl/nccl' 2024-08-20T23:46:34.0817004Z http.https://github.com/.extraheader 2024-08-20T23:46:34.0861854Z Entering 'third_party/nlohmann' 2024-08-20T23:46:34.0906253Z http.https://github.com/.extraheader 2024-08-20T23:46:34.0951355Z Entering 'third_party/onnx' 2024-08-20T23:46:34.0995259Z http.https://github.com/.extraheader 2024-08-20T23:46:34.1053106Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:46:34.1103794Z http.https://github.com/.extraheader 2024-08-20T23:46:34.1146858Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:46:34.1190038Z http.https://github.com/.extraheader 2024-08-20T23:46:34.1238170Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:46:34.1282292Z http.https://github.com/.extraheader 2024-08-20T23:46:34.1327188Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:46:34.1369874Z http.https://github.com/.extraheader 2024-08-20T23:46:34.1411920Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:46:34.1460595Z http.https://github.com/.extraheader 2024-08-20T23:46:34.1504490Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:46:34.1546978Z http.https://github.com/.extraheader 2024-08-20T23:46:34.1589547Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:46:34.1631590Z http.https://github.com/.extraheader 2024-08-20T23:46:34.1676219Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:46:34.1717672Z http.https://github.com/.extraheader 2024-08-20T23:46:34.1761219Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:46:34.1802610Z http.https://github.com/.extraheader 2024-08-20T23:46:34.1846698Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:46:34.1891286Z http.https://github.com/.extraheader 2024-08-20T23:46:34.1936235Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:46:34.1979868Z http.https://github.com/.extraheader 2024-08-20T23:46:34.2027144Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:46:34.2070462Z http.https://github.com/.extraheader 2024-08-20T23:46:34.2118036Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:46:34.2161668Z http.https://github.com/.extraheader 2024-08-20T23:46:34.2224897Z Entering 'third_party/pocketfft' 2024-08-20T23:46:34.2269106Z http.https://github.com/.extraheader 2024-08-20T23:46:34.2312591Z Entering 'third_party/protobuf' 2024-08-20T23:46:34.2357168Z http.https://github.com/.extraheader 2024-08-20T23:46:34.2404730Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:46:34.2448890Z http.https://github.com/.extraheader 2024-08-20T23:46:34.2492363Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:46:34.2539753Z http.https://github.com/.extraheader 2024-08-20T23:46:34.2590155Z Entering 'third_party/psimd' 2024-08-20T23:46:34.2633682Z http.https://github.com/.extraheader 2024-08-20T23:46:34.2677436Z Entering 'third_party/pthreadpool' 2024-08-20T23:46:34.2720502Z http.https://github.com/.extraheader 2024-08-20T23:46:34.2763898Z Entering 'third_party/pybind11' 2024-08-20T23:46:34.2807605Z http.https://github.com/.extraheader 2024-08-20T23:46:34.2851654Z Entering 'third_party/python-peachpy' 2024-08-20T23:46:34.2896150Z http.https://github.com/.extraheader 2024-08-20T23:46:34.2939545Z Entering 'third_party/sleef' 2024-08-20T23:46:34.2983666Z http.https://github.com/.extraheader 2024-08-20T23:46:34.3027511Z Entering 'third_party/tensorpipe' 2024-08-20T23:46:34.3070711Z http.https://github.com/.extraheader 2024-08-20T23:46:34.3118564Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:46:34.3160726Z http.https://github.com/.extraheader 2024-08-20T23:46:34.3203925Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:46:34.3246067Z http.https://github.com/.extraheader 2024-08-20T23:46:34.3291543Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:46:34.3342935Z http.https://github.com/.extraheader 2024-08-20T23:46:34.3387669Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:46:34.3429766Z http.https://github.com/.extraheader 2024-08-20T23:46:34.3470608Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:46:34.3514270Z http.https://github.com/.extraheader 2024-08-20T23:46:34.3679368Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-20T23:46:34.3699607Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-20T23:46:34.3707720Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:46:34.3708250Z ##[endgroup] 2024-08-20T23:46:41.2358716Z Cleaning up orphan processes