2024-08-20T22:08:36.9024005Z Current runner version: '2.319.1' 2024-08-20T22:08:36.9030213Z Runner name: 'i-0aa26b6dbaf56f3e9' 2024-08-20T22:08:36.9031037Z Runner group name: 'Default' 2024-08-20T22:08:36.9032028Z Machine name: 'ip-10-0-67-7' 2024-08-20T22:08:36.9049056Z Testing runner upgrade compatibility 2024-08-20T22:08:37.1083314Z ##[group]GITHUB_TOKEN Permissions 2024-08-20T22:08:37.1085286Z Actions: read 2024-08-20T22:08:37.1085697Z Attestations: read 2024-08-20T22:08:37.1086145Z Checks: read 2024-08-20T22:08:37.1086609Z Contents: read 2024-08-20T22:08:37.1087029Z Deployments: read 2024-08-20T22:08:37.1087440Z Discussions: read 2024-08-20T22:08:37.1087930Z Issues: read 2024-08-20T22:08:37.1088355Z Metadata: read 2024-08-20T22:08:37.1088770Z Packages: read 2024-08-20T22:08:37.1089200Z Pages: read 2024-08-20T22:08:37.1089617Z PullRequests: read 2024-08-20T22:08:37.1090063Z RepositoryProjects: read 2024-08-20T22:08:37.1090552Z SecurityEvents: read 2024-08-20T22:08:37.1091009Z Statuses: read 2024-08-20T22:08:37.1091722Z ##[endgroup] 2024-08-20T22:08:37.1094708Z Secret source: Actions 2024-08-20T22:08:37.1095375Z Prepare workflow directory 2024-08-20T22:08:37.2010796Z Prepare all required actions 2024-08-20T22:08:37.2177384Z Getting action download info 2024-08-20T22:08:37.3735318Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-20T22:08:37.8131849Z Download action repository 'pytorch/pytorch@main' (SHA:1ae5d5bb62141d7c9b1b0b66c66a462b4e10b1f2) 2024-08-20T22:08:41.8040446Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-20T22:08:42.0036550Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-20T22:08:42.3030216Z Getting action download info 2024-08-20T22:08:42.3960732Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-20T22:08:42.6365648Z Getting action download info 2024-08-20T22:08:42.7217676Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-20T22:08:42.8847664Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/133712 (40ec5f6ddd9787aca0449b24128343ff4c4a88b3) 2024-08-20T22:08:42.8850076Z ##[group] Inputs 2024-08-20T22:08:42.8850594Z build-environment: linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-08-20T22:08:42.8851953Z test-matrix: {"include": [{"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:08:42.8853969Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:42.8855285Z sync-tag: linux-focal-cuda12_4-py3_10-gcc9-inductor-test 2024-08-20T22:08:42.8856288Z timeout-minutes: 240 2024-08-20T22:08:42.8856633Z use-gha: 2024-08-20T22:08:42.8856927Z dashboard-tag: 2024-08-20T22:08:42.8857251Z s3-bucket: gha-artifacts 2024-08-20T22:08:42.8857606Z aws-role-to-assume: 2024-08-20T22:08:42.8857943Z ##[endgroup] 2024-08-20T22:08:42.8858887Z Complete job name: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:08:42.9601829Z A job started hook has been configured by the self-hosted runner administrator 2024-08-20T22:08:42.9749268Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-20T22:08:42.9761213Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:42.9761746Z ##[endgroup] 2024-08-20T22:08:44.1537781Z Runner Type: amz2023.linux.g5.4xlarge.nvidia.gpu 2024-08-20T22:08:44.1538384Z Instance Type: g5.4xlarge 2024-08-20T22:08:44.1539195Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-20T22:08:44.1539756Z AMI ID: ami-06c68f701d8090592 2024-08-20T22:08:49.9112741Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-20T22:08:49.9113297Z with: 2024-08-20T22:08:49.9113990Z github-secret: *** 2024-08-20T22:08:49.9114933Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:08:49.9115943Z activate-with-label: false 2024-08-20T22:08:49.9116301Z label: with-ssh 2024-08-20T22:08:49.9116626Z remove-existing-keys: true 2024-08-20T22:08:49.9116999Z fail-silently: true 2024-08-20T22:08:49.9117309Z env: 2024-08-20T22:08:49.9117587Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:49.9117935Z ##[endgroup] 2024-08-20T22:08:50.0003143Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-20T22:08:50.0005873Z ciflow reference detected, attempting to extract PR number 2024-08-20T22:08:50.3407301Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-08-20T22:08:50.4474916Z No SSH keys found for user pytorch-bot[bot] 2024-08-20T22:08:50.4475646Z Grabbing public ssh keys from https://github.com/XuehaiPan.keys 2024-08-20T22:08:50.5151065Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-20T22:08:50.5165595Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-20T22:08:50.5188637Z Login using: ssh ec2-user@ec2-44-210-99-254.compute-1.amazonaws.com 2024-08-20T22:08:50.5189564Z All testing is done inside the container, to start an interactive session run: 2024-08-20T22:08:50.5190393Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:08:50.5321248Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-20T22:08:50.5321823Z with: 2024-08-20T22:08:50.5322117Z submodules: recursive 2024-08-20T22:08:50.5322463Z fetch-depth: 0 2024-08-20T22:08:50.5322775Z env: 2024-08-20T22:08:50.5323063Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:50.5323437Z ##[endgroup] 2024-08-20T22:08:50.5522003Z ##[group]Run retry () { 2024-08-20T22:08:50.5522397Z retry () { 2024-08-20T22:08:50.5522927Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-20T22:08:50.5523492Z } 2024-08-20T22:08:50.5523811Z echo "${GITHUB_WORKSPACE}" 2024-08-20T22:08:50.5524242Z if [ -z "${NO_SUDO}" ]; then 2024-08-20T22:08:50.5524738Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:08:50.5525202Z else 2024-08-20T22:08:50.5525557Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:08:50.5525995Z fi 2024-08-20T22:08:50.5526341Z mkdir "${GITHUB_WORKSPACE}" 2024-08-20T22:08:50.5539281Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:50.5539807Z env: 2024-08-20T22:08:50.5540103Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:50.5540458Z NO_SUDO: 2024-08-20T22:08:50.5540751Z ##[endgroup] 2024-08-20T22:08:50.5573186Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:08:50.6929071Z ##[group]Run malfet/checkout@silent-checkout 2024-08-20T22:08:50.6929505Z with: 2024-08-20T22:08:50.6929834Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:08:50.6930283Z fetch-depth: 0 2024-08-20T22:08:50.6930597Z submodules: recursive 2024-08-20T22:08:50.6930938Z quiet-checkout: true 2024-08-20T22:08:50.6931290Z repository: pytorch/pytorch 2024-08-20T22:08:50.6931765Z token: *** 2024-08-20T22:08:50.6932056Z ssh-strict: true 2024-08-20T22:08:50.6932374Z persist-credentials: true 2024-08-20T22:08:50.6932737Z clean: true 2024-08-20T22:08:50.6933082Z sparse-checkout-cone-mode: true 2024-08-20T22:08:50.6933479Z lfs: false 2024-08-20T22:08:50.6933781Z set-safe-directory: true 2024-08-20T22:08:50.6934120Z env: 2024-08-20T22:08:50.6934388Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:50.6934725Z ##[endgroup] 2024-08-20T22:08:50.7885179Z Syncing repository: pytorch/pytorch 2024-08-20T22:08:50.7887194Z ##[group]Getting Git version info 2024-08-20T22:08:50.7888004Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:08:50.7888866Z [command]/usr/bin/git version 2024-08-20T22:08:50.7889229Z git version 2.40.1 2024-08-20T22:08:50.7890692Z ##[endgroup] 2024-08-20T22:08:50.7901838Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/d173bd1a-bd34-4a24-b7cf-59dad9546b81' before making global git config changes 2024-08-20T22:08:50.7903095Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T22:08:50.7906476Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:08:50.7951400Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:08:50.7955217Z ##[group]Initializing the repository 2024-08-20T22:08:50.7957968Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:08:50.8002379Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-20T22:08:50.8003507Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-20T22:08:50.8004363Z hint: of your new repositories, which will suppress this warning, call: 2024-08-20T22:08:50.8004944Z hint: 2024-08-20T22:08:50.8005725Z hint: git config --global init.defaultBranch 2024-08-20T22:08:50.8006217Z hint: 2024-08-20T22:08:50.8006774Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-20T22:08:50.8007656Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-20T22:08:50.8008529Z hint: 2024-08-20T22:08:50.8008880Z hint: git branch -m 2024-08-20T22:08:50.8009659Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-20T22:08:50.8016641Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-20T22:08:50.8059347Z ##[endgroup] 2024-08-20T22:08:50.8059965Z ##[group]Disabling automatic garbage collection 2024-08-20T22:08:50.8062094Z [command]/usr/bin/git config --local gc.auto 0 2024-08-20T22:08:50.8102458Z ##[endgroup] 2024-08-20T22:08:50.8102987Z ##[group]Setting up auth 2024-08-20T22:08:50.8108492Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T22:08:50.8149424Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T22:08:50.8514180Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T22:08:50.8554378Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T22:08:50.8906395Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:08:50.8962952Z ##[endgroup] 2024-08-20T22:08:50.8963514Z ##[group]Fetching the repository 2024-08-20T22:08:50.8968702Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-20T22:08:53.7520910Z remote: Enumerating objects: 1008518 2024-08-20T22:08:53.7521627Z remote: Enumerating objects: 1010208, done. 2024-08-20T22:08:53.7522501Z remote: Counting objects: 0% (1/1690) 2024-08-20T22:08:53.7523181Z remote: Counting objects: 1% (17/1690) 2024-08-20T22:08:53.7523888Z remote: Counting objects: 2% (34/1690) 2024-08-20T22:08:53.7524530Z remote: Counting objects: 3% (51/1690) 2024-08-20T22:08:53.7525206Z remote: Counting objects: 4% (68/1690) 2024-08-20T22:08:53.7525880Z remote: Counting objects: 5% (85/1690) 2024-08-20T22:08:53.7526561Z remote: Counting objects: 6% (102/1690) 2024-08-20T22:08:53.7527562Z remote: Counting objects: 7% (119/1690) 2024-08-20T22:08:53.7528171Z remote: Counting objects: 8% (136/1690) 2024-08-20T22:08:53.7528675Z remote: Counting objects: 9% (153/1690) 2024-08-20T22:08:53.7529293Z remote: Counting objects: 10% (169/1690) 2024-08-20T22:08:53.7529795Z remote: Counting objects: 11% (186/1690) 2024-08-20T22:08:53.7530355Z remote: Counting objects: 12% (203/1690) 2024-08-20T22:08:53.7530867Z remote: Counting objects: 13% (220/1690) 2024-08-20T22:08:53.7531377Z remote: Counting objects: 14% (237/1690) 2024-08-20T22:08:53.7532250Z remote: Counting objects: 15% (254/1690) 2024-08-20T22:08:53.7532886Z remote: Counting objects: 16% (271/1690) 2024-08-20T22:08:53.7533385Z remote: Counting objects: 17% (288/1690) 2024-08-20T22:08:53.7533878Z remote: Counting objects: 18% (305/1690) 2024-08-20T22:08:53.7534510Z remote: Counting objects: 19% (322/1690) 2024-08-20T22:08:53.7535187Z remote: Counting objects: 20% (338/1690) 2024-08-20T22:08:53.7535859Z remote: Counting objects: 21% (355/1690) 2024-08-20T22:08:53.7536379Z remote: Counting objects: 22% (372/1690) 2024-08-20T22:08:53.7536871Z remote: Counting objects: 23% (389/1690) 2024-08-20T22:08:53.7537485Z remote: Counting objects: 24% (406/1690) 2024-08-20T22:08:53.7538144Z remote: Counting objects: 25% (423/1690) 2024-08-20T22:08:53.7538823Z remote: Counting objects: 26% (440/1690) 2024-08-20T22:08:53.7539418Z remote: Counting objects: 27% (457/1690) 2024-08-20T22:08:53.7540368Z remote: Counting objects: 28% (474/1690) 2024-08-20T22:08:53.7541028Z remote: Counting objects: 29% (491/1690) 2024-08-20T22:08:53.7541701Z remote: Counting objects: 30% (507/1690) 2024-08-20T22:08:53.7542355Z remote: Counting objects: 31% (524/1690) 2024-08-20T22:08:53.7542986Z remote: Counting objects: 32% (541/1690) 2024-08-20T22:08:53.7543485Z remote: Counting objects: 33% (558/1690) 2024-08-20T22:08:53.7544141Z remote: Counting objects: 34% (575/1690) 2024-08-20T22:08:53.7544798Z remote: Counting objects: 35% (592/1690) 2024-08-20T22:08:53.7545355Z remote: Counting objects: 36% (609/1690) 2024-08-20T22:08:53.7546027Z remote: Counting objects: 37% (626/1690) 2024-08-20T22:08:53.7546918Z remote: Counting objects: 38% (643/1690) 2024-08-20T22:08:53.7547451Z remote: Counting objects: 39% (660/1690) 2024-08-20T22:08:53.7547951Z remote: Counting objects: 40% (676/1690) 2024-08-20T22:08:53.7548461Z remote: Counting objects: 41% (693/1690) 2024-08-20T22:08:53.7548960Z remote: Counting objects: 42% (710/1690) 2024-08-20T22:08:53.7549446Z remote: Counting objects: 43% (727/1690) 2024-08-20T22:08:53.7549954Z remote: Counting objects: 44% (744/1690) 2024-08-20T22:08:53.7550463Z remote: Counting objects: 45% (761/1690) 2024-08-20T22:08:53.7550968Z remote: Counting objects: 46% (778/1690) 2024-08-20T22:08:53.7551461Z remote: Counting objects: 47% (795/1690) 2024-08-20T22:08:53.7551963Z remote: Counting objects: 48% (812/1690) 2024-08-20T22:08:53.7552467Z remote: Counting objects: 49% (829/1690) 2024-08-20T22:08:53.7552967Z remote: Counting objects: 50% (845/1690) 2024-08-20T22:08:53.7553459Z remote: Counting objects: 51% (862/1690) 2024-08-20T22:08:53.7553963Z remote: Counting objects: 52% (879/1690) 2024-08-20T22:08:53.7554460Z remote: Counting objects: 53% (896/1690) 2024-08-20T22:08:53.7554966Z remote: Counting objects: 54% (913/1690) 2024-08-20T22:08:53.7555456Z remote: Counting objects: 55% (930/1690) 2024-08-20T22:08:53.7555958Z remote: Counting objects: 56% (947/1690) 2024-08-20T22:08:53.7556463Z remote: Counting objects: 57% (964/1690) 2024-08-20T22:08:53.7557134Z remote: Counting objects: 58% (981/1690) 2024-08-20T22:08:53.7557630Z remote: Counting objects: 59% (998/1690) 2024-08-20T22:08:53.7558154Z remote: Counting objects: 60% (1014/1690) 2024-08-20T22:08:53.7558687Z remote: Counting objects: 61% (1031/1690) 2024-08-20T22:08:53.7559218Z remote: Counting objects: 62% (1048/1690) 2024-08-20T22:08:53.7559784Z remote: Counting objects: 63% (1065/1690) 2024-08-20T22:08:53.7560316Z remote: Counting objects: 64% (1082/1690) 2024-08-20T22:08:53.7560829Z remote: Counting objects: 65% (1099/1690) 2024-08-20T22:08:53.7561359Z remote: Counting objects: 66% (1116/1690) 2024-08-20T22:08:53.7561884Z remote: Counting objects: 67% (1133/1690) 2024-08-20T22:08:53.7562406Z remote: Counting objects: 68% (1150/1690) 2024-08-20T22:08:53.7562925Z remote: Counting objects: 69% (1167/1690) 2024-08-20T22:08:53.7563444Z remote: Counting objects: 70% (1183/1690) 2024-08-20T22:08:53.7563973Z remote: Counting objects: 71% (1200/1690) 2024-08-20T22:08:53.7564483Z remote: Counting objects: 72% (1217/1690) 2024-08-20T22:08:53.7565012Z remote: Counting objects: 73% (1234/1690) 2024-08-20T22:08:53.7565533Z remote: Counting objects: 74% (1251/1690) 2024-08-20T22:08:53.7566047Z remote: Counting objects: 75% (1268/1690) 2024-08-20T22:08:53.7566557Z remote: Counting objects: 76% (1285/1690) 2024-08-20T22:08:53.7567099Z remote: Counting objects: 77% (1302/1690) 2024-08-20T22:08:53.7567642Z remote: Counting objects: 78% (1319/1690) 2024-08-20T22:08:53.7568288Z remote: Counting objects: 79% (1336/1690) 2024-08-20T22:08:53.7568794Z remote: Counting objects: 80% (1352/1690) 2024-08-20T22:08:53.7569310Z remote: Counting objects: 81% (1369/1690) 2024-08-20T22:08:53.7569832Z remote: Counting objects: 82% (1386/1690) 2024-08-20T22:08:53.7570352Z remote: Counting objects: 83% (1403/1690) 2024-08-20T22:08:53.7570874Z remote: Counting objects: 84% (1420/1690) 2024-08-20T22:08:53.7571388Z remote: Counting objects: 85% (1437/1690) 2024-08-20T22:08:53.7571906Z remote: Counting objects: 86% (1454/1690) 2024-08-20T22:08:53.7572424Z remote: Counting objects: 87% (1471/1690) 2024-08-20T22:08:53.7572935Z remote: Counting objects: 88% (1488/1690) 2024-08-20T22:08:53.7573455Z remote: Counting objects: 89% (1505/1690) 2024-08-20T22:08:53.7573980Z remote: Counting objects: 90% (1521/1690) 2024-08-20T22:08:53.7574500Z remote: Counting objects: 91% (1538/1690) 2024-08-20T22:08:53.7575020Z remote: Counting objects: 92% (1555/1690) 2024-08-20T22:08:53.7575538Z remote: Counting objects: 93% (1572/1690) 2024-08-20T22:08:53.7576058Z remote: Counting objects: 94% (1589/1690) 2024-08-20T22:08:53.7576575Z remote: Counting objects: 95% (1606/1690) 2024-08-20T22:08:53.7577090Z remote: Counting objects: 96% (1623/1690) 2024-08-20T22:08:53.7577635Z remote: Counting objects: 97% (1640/1690) 2024-08-20T22:08:53.7578186Z remote: Counting objects: 98% (1657/1690) 2024-08-20T22:08:53.7578773Z remote: Counting objects: 99% (1674/1690) 2024-08-20T22:08:53.7579298Z remote: Counting objects: 100% (1690/1690) 2024-08-20T22:08:53.7579866Z remote: Counting objects: 100% (1690/1690), done. 2024-08-20T22:08:53.7647159Z remote: Compressing objects: 0% (1/875) 2024-08-20T22:08:53.7961643Z remote: Compressing objects: 1% (9/875) 2024-08-20T22:08:53.8226157Z remote: Compressing objects: 2% (18/875) 2024-08-20T22:08:53.8698329Z remote: Compressing objects: 3% (27/875) 2024-08-20T22:08:53.9386495Z remote: Compressing objects: 4% (35/875) 2024-08-20T22:08:53.9943800Z remote: Compressing objects: 5% (44/875) 2024-08-20T22:08:54.0395847Z remote: Compressing objects: 6% (53/875) 2024-08-20T22:08:54.0828835Z remote: Compressing objects: 7% (62/875) 2024-08-20T22:08:54.1271912Z remote: Compressing objects: 8% (70/875) 2024-08-20T22:08:54.1579908Z remote: Compressing objects: 9% (79/875) 2024-08-20T22:08:54.1812965Z remote: Compressing objects: 10% (88/875) 2024-08-20T22:08:54.2119248Z remote: Compressing objects: 11% (97/875) 2024-08-20T22:08:54.2336218Z remote: Compressing objects: 12% (105/875) 2024-08-20T22:08:54.2545778Z remote: Compressing objects: 13% (114/875) 2024-08-20T22:08:54.2710030Z remote: Compressing objects: 14% (123/875) 2024-08-20T22:08:54.2835063Z remote: Compressing objects: 15% (132/875) 2024-08-20T22:08:54.2917753Z remote: Compressing objects: 16% (140/875) 2024-08-20T22:08:54.3013064Z remote: Compressing objects: 17% (149/875) 2024-08-20T22:08:54.3074978Z remote: Compressing objects: 18% (158/875) 2024-08-20T22:08:54.3125975Z remote: Compressing objects: 19% (167/875) 2024-08-20T22:08:54.3167989Z remote: Compressing objects: 20% (175/875) 2024-08-20T22:08:54.3195172Z remote: Compressing objects: 21% (184/875) 2024-08-20T22:08:54.3212601Z remote: Compressing objects: 22% (193/875) 2024-08-20T22:08:54.3215950Z remote: Compressing objects: 23% (202/875) 2024-08-20T22:08:54.3224497Z remote: Compressing objects: 24% (210/875) 2024-08-20T22:08:54.3231861Z remote: Compressing objects: 25% (219/875) 2024-08-20T22:08:54.3238543Z remote: Compressing objects: 26% (228/875) 2024-08-20T22:08:54.3246805Z remote: Compressing objects: 27% (237/875) 2024-08-20T22:08:54.3269538Z remote: Compressing objects: 28% (245/875) 2024-08-20T22:08:54.3283666Z remote: Compressing objects: 29% (254/875) 2024-08-20T22:08:54.3299051Z remote: Compressing objects: 30% (263/875) 2024-08-20T22:08:54.3302955Z remote: Compressing objects: 31% (272/875) 2024-08-20T22:08:54.3313545Z remote: Compressing objects: 32% (280/875) 2024-08-20T22:08:54.3320056Z remote: Compressing objects: 33% (289/875) 2024-08-20T22:08:54.3327155Z remote: Compressing objects: 34% (298/875) 2024-08-20T22:08:54.3332797Z remote: Compressing objects: 35% (307/875) 2024-08-20T22:08:54.3344795Z remote: Compressing objects: 36% (315/875) 2024-08-20T22:08:54.3350767Z remote: Compressing objects: 37% (324/875) 2024-08-20T22:08:54.3358094Z remote: Compressing objects: 38% (333/875) 2024-08-20T22:08:54.3363649Z remote: Compressing objects: 39% (342/875) 2024-08-20T22:08:54.3369647Z remote: Compressing objects: 40% (350/875) 2024-08-20T22:08:54.3376687Z remote: Compressing objects: 41% (359/875) 2024-08-20T22:08:54.3380961Z remote: Compressing objects: 42% (368/875) 2024-08-20T22:08:54.3386051Z remote: Compressing objects: 43% (377/875) 2024-08-20T22:08:54.3393418Z remote: Compressing objects: 44% (385/875) 2024-08-20T22:08:54.3399161Z remote: Compressing objects: 45% (394/875) 2024-08-20T22:08:54.3403401Z remote: Compressing objects: 46% (403/875) 2024-08-20T22:08:54.3406406Z remote: Compressing objects: 47% (412/875) 2024-08-20T22:08:54.3410444Z remote: Compressing objects: 48% (420/875) 2024-08-20T22:08:54.3413696Z remote: Compressing objects: 49% (429/875) 2024-08-20T22:08:54.3416024Z remote: Compressing objects: 50% (438/875) 2024-08-20T22:08:54.3420053Z remote: Compressing objects: 51% (447/875) 2024-08-20T22:08:54.3422927Z remote: Compressing objects: 52% (455/875) 2024-08-20T22:08:54.3426098Z remote: Compressing objects: 53% (464/875) 2024-08-20T22:08:54.3428278Z remote: Compressing objects: 54% (473/875) 2024-08-20T22:08:54.3429447Z remote: Compressing objects: 55% (482/875) 2024-08-20T22:08:54.3430713Z remote: Compressing objects: 56% (490/875) 2024-08-20T22:08:54.3431355Z remote: Compressing objects: 57% (499/875) 2024-08-20T22:08:54.3432061Z remote: Compressing objects: 58% (508/875) 2024-08-20T22:08:54.3432692Z remote: Compressing objects: 59% (517/875) 2024-08-20T22:08:54.3433586Z remote: Compressing objects: 60% (525/875) 2024-08-20T22:08:54.3434571Z remote: Compressing objects: 61% (534/875) 2024-08-20T22:08:54.3435164Z remote: Compressing objects: 62% (543/875) 2024-08-20T22:08:54.3435685Z remote: Compressing objects: 63% (552/875) 2024-08-20T22:08:54.3440810Z remote: Compressing objects: 64% (560/875) 2024-08-20T22:08:54.3446107Z remote: Compressing objects: 65% (569/875) 2024-08-20T22:08:54.3453415Z remote: Compressing objects: 66% (578/875) 2024-08-20T22:08:54.3455649Z remote: Compressing objects: 67% (587/875) 2024-08-20T22:08:54.3461499Z remote: Compressing objects: 68% (595/875) 2024-08-20T22:08:54.3468709Z remote: Compressing objects: 69% (604/875) 2024-08-20T22:08:54.3470401Z remote: Compressing objects: 70% (613/875) 2024-08-20T22:08:54.3473357Z remote: Compressing objects: 71% (622/875) 2024-08-20T22:08:54.3476771Z remote: Compressing objects: 72% (630/875) 2024-08-20T22:08:54.3481438Z remote: Compressing objects: 73% (639/875) 2024-08-20T22:08:54.3483883Z remote: Compressing objects: 74% (648/875) 2024-08-20T22:08:54.3486776Z remote: Compressing objects: 75% (657/875) 2024-08-20T22:08:54.3489453Z remote: Compressing objects: 76% (665/875) 2024-08-20T22:08:54.3492867Z remote: Compressing objects: 77% (674/875) 2024-08-20T22:08:54.3495125Z remote: Compressing objects: 78% (683/875) 2024-08-20T22:08:54.3498439Z remote: Compressing objects: 79% (692/875) 2024-08-20T22:08:54.3499508Z remote: Compressing objects: 80% (700/875) 2024-08-20T22:08:54.3502814Z remote: Compressing objects: 81% (709/875) 2024-08-20T22:08:54.3503693Z remote: Compressing objects: 82% (718/875) 2024-08-20T22:08:54.3505703Z remote: Compressing objects: 83% (727/875) 2024-08-20T22:08:54.3507465Z remote: Compressing objects: 84% (735/875) 2024-08-20T22:08:54.3510377Z remote: Compressing objects: 85% (744/875) 2024-08-20T22:08:54.3511608Z remote: Compressing objects: 86% (753/875) 2024-08-20T22:08:54.3512788Z remote: Compressing objects: 87% (762/875) 2024-08-20T22:08:54.3515790Z remote: Compressing objects: 88% (770/875) 2024-08-20T22:08:54.3518060Z remote: Compressing objects: 89% (779/875) 2024-08-20T22:08:54.3519209Z remote: Compressing objects: 90% (788/875) 2024-08-20T22:08:54.3520092Z remote: Compressing objects: 91% (797/875) 2024-08-20T22:08:54.3522034Z remote: Compressing objects: 92% (805/875) 2024-08-20T22:08:54.3522562Z remote: Compressing objects: 93% (814/875) 2024-08-20T22:08:54.3523211Z remote: Compressing objects: 94% (823/875) 2024-08-20T22:08:54.3524197Z remote: Compressing objects: 95% (832/875) 2024-08-20T22:08:54.3526858Z remote: Compressing objects: 96% (840/875) 2024-08-20T22:08:54.3528035Z remote: Compressing objects: 97% (849/875) 2024-08-20T22:08:54.3528557Z remote: Compressing objects: 98% (858/875) 2024-08-20T22:08:54.3529585Z remote: Compressing objects: 99% (867/875) 2024-08-20T22:08:54.3531086Z remote: Compressing objects: 100% (875/875) 2024-08-20T22:08:54.3531648Z remote: Compressing objects: 100% (875/875), done. 2024-08-20T22:09:17.2795543Z remote: Total 1010208 (delta 1086), reused 1298 (delta 812), pack-reused 1008518 (from 1) 2024-08-20T22:09:39.3253908Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-20T22:09:39.3289692Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:09:39.3294756Z ##[endgroup] 2024-08-20T22:09:39.3295478Z ##[group]Determining the checkout info 2024-08-20T22:09:39.3296182Z ##[endgroup] 2024-08-20T22:09:39.3296702Z ##[group]Checking out the ref 2024-08-20T22:09:39.3299261Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:09:41.0469077Z ##[endgroup] 2024-08-20T22:09:41.0469694Z ##[group]Setting up auth for fetching submodules 2024-08-20T22:09:41.0472514Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:09:41.0542463Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-20T22:09:41.0581319Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-20T22:09:41.0620825Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-20T22:09:41.0657669Z ##[endgroup] 2024-08-20T22:09:41.0660846Z ##[group]Fetching submodules 2024-08-20T22:09:41.0661489Z [command]/usr/bin/git submodule sync --recursive 2024-08-20T22:09:41.1035973Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-20T22:09:41.1399796Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-20T22:09:41.1401382Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-20T22:09:41.1404569Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-20T22:09:41.1407653Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-20T22:09:41.1412586Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-20T22:09:41.1416058Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-20T22:09:41.1419578Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-20T22:09:41.1423324Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-20T22:09:41.1426738Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-20T22:09:41.1430622Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-20T22:09:41.1434267Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-20T22:09:41.1438014Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-20T22:09:41.1441844Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-20T22:09:41.1445900Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-20T22:09:41.1450436Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-20T22:09:41.1454517Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:09:41.1462221Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-20T22:09:41.1466433Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-20T22:09:41.1470627Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-20T22:09:41.1475212Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-20T22:09:41.1479641Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-20T22:09:41.1484033Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-20T22:09:41.1488472Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-20T22:09:41.1493362Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-20T22:09:41.1498061Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-20T22:09:41.1503376Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-20T22:09:41.1507912Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-20T22:09:41.1512825Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-20T22:09:41.1517750Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-20T22:09:41.1522783Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-20T22:09:41.1527738Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-20T22:09:41.1533144Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-20T22:09:41.1538074Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-20T22:09:41.1546976Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-20T22:09:41.1581235Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-20T22:09:41.4650678Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-20T22:09:41.6510661Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-20T22:09:41.8139019Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-20T22:09:42.0539883Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-20T22:09:43.9334806Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-20T22:09:55.2019964Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-20T22:09:55.6010878Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-20T22:09:56.0030530Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-20T22:09:56.5867659Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-20T22:09:57.8457596Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-20T22:09:59.9381286Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-20T22:10:06.0290270Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-20T22:10:07.5367573Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-20T22:10:09.3499186Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-20T22:10:10.6923573Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-20T22:10:11.1324014Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-20T22:10:11.4778450Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-20T22:10:12.7087560Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-20T22:10:13.1296720Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-20T22:10:13.3718902Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-20T22:10:14.9898963Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-20T22:10:15.6879130Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-20T22:10:16.3584220Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-20T22:10:22.5643686Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-20T22:10:24.5881762Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-20T22:10:28.9217397Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-20T22:10:29.1964076Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-20T22:10:38.3639855Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-20T22:10:38.5183499Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-20T22:10:38.7156946Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-20T22:10:39.6182266Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-20T22:10:39.9298843Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-20T22:10:40.5612626Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-20T22:10:41.0102176Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-20T22:10:41.0247180Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-20T22:10:41.0365102Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-20T22:10:41.0675957Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-20T22:10:41.1132103Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-20T22:10:42.2914002Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-20T22:10:42.3242113Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-20T22:10:42.3795531Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-20T22:10:42.4909476Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-20T22:10:42.5314110Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-20T22:10:43.1368551Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-20T22:10:43.4199974Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-20T22:10:43.5142991Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-20T22:10:43.5165971Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:10:43.5169008Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:10:43.5172379Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:10:43.5175915Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:10:43.5179674Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:10:43.5211161Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-20T22:10:44.5489883Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-20T22:10:45.1532491Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-20T22:10:47.1900569Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-20T22:10:48.2810179Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-20T22:10:48.6183147Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-20T22:10:48.7297057Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-20T22:10:49.2418043Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-20T22:10:49.3124652Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-20T22:10:49.3275260Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-20T22:10:49.4766036Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-20T22:10:49.5232008Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-20T22:10:49.5688824Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-20T22:10:49.6017686Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-20T22:10:49.6550469Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:10:49.6706632Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-20T22:10:49.6726593Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-20T22:10:49.6757333Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-20T22:11:02.9809800Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-20T22:11:03.0027319Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-20T22:11:03.1018346Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-20T22:11:03.1040468Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:11:03.1043009Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:11:03.1047077Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:11:03.1079567Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-20T22:11:03.6756278Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-20T22:11:04.8631358Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-20T22:11:05.9973175Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-20T22:11:05.9993283Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:11:05.9996643Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:11:06.0000131Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:11:06.0003805Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:11:06.0007535Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:11:06.0011465Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:11:06.0015255Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:11:06.0019122Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:11:06.0052455Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-20T22:11:06.8391420Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-20T22:11:07.2309837Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-20T22:11:08.4313980Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-20T22:11:08.7937155Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-20T22:11:09.3273936Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-20T22:11:10.3697463Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-20T22:11:17.7688009Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-20T22:11:18.1828771Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-20T22:11:18.2055342Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-20T22:11:18.2492626Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-20T22:11:18.2658882Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-20T22:11:18.2678596Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:11:18.2712953Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-20T22:11:18.5566044Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-20T22:11:18.5788485Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-20T22:11:18.6260441Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-20T22:11:18.7466346Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-20T22:11:18.7662533Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-20T22:11:18.8107406Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-20T22:11:18.8755679Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-20T22:11:18.9206266Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-20T22:11:18.9506545Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-20T22:11:19.0730249Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-20T22:11:19.5633890Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-20T22:11:19.5671468Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-20T22:11:19.5674250Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-20T22:11:19.5706386Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-20T22:11:20.0397853Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-20T22:11:22.0545330Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-20T22:11:22.0964027Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-20T22:11:22.1867110Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-20T22:11:22.1892318Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:11:22.1896204Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:11:22.1897882Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:11:22.1900931Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:11:22.1904721Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:11:22.1908289Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:11:22.1911993Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:11:22.1915659Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:11:22.1947964Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-20T22:11:22.6368802Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-20T22:11:23.7089728Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-20T22:11:24.0226527Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-20T22:11:30.0621192Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-20T22:11:30.3298167Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-20T22:11:30.5413412Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-20T22:11:30.8524778Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-20T22:11:37.7309893Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-20T22:11:37.7783745Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-20T22:11:37.7971344Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-20T22:11:37.9194057Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-20T22:11:37.9357000Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-20T22:11:37.9541227Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-20T22:11:37.9743874Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-20T22:11:37.9763662Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:11:37.9766825Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:11:37.9797616Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-20T22:11:39.8485173Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-20T22:11:41.1980573Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-20T22:11:41.2502935Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:11:41.9185404Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-20T22:11:41.9328180Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-20T22:11:42.2478386Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-20T22:11:42.2499869Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:11:42.2503326Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-20T22:11:42.2536717Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-20T22:11:42.6842549Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-20T22:11:43.7463360Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-20T22:11:43.8276335Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-20T22:11:43.8396279Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-20T22:11:43.8556780Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-20T22:11:43.9001989Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-20T22:11:43.9345540Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-20T22:11:43.9835418Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-20T22:11:44.0177497Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-20T22:11:44.0198412Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:11:44.0201640Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:11:44.0205355Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:11:44.0208741Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:11:44.0240219Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-20T22:11:45.0401872Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-20T22:11:45.2659560Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-20T22:11:46.5172526Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-20T22:11:47.4649294Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-20T22:11:47.4845835Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-20T22:11:47.5636730Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-20T22:11:47.5984950Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-20T22:11:47.6004870Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:11:47.6035557Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-20T22:11:47.8253518Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-20T22:11:47.8306258Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-20T22:11:47.8669852Z Entering 'android/libs/fbjni' 2024-08-20T22:11:47.8723378Z Entering 'third_party/FP16' 2024-08-20T22:11:47.8776806Z Entering 'third_party/FXdiv' 2024-08-20T22:11:47.8829049Z Entering 'third_party/NNPACK' 2024-08-20T22:11:47.8882224Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:11:47.8934744Z Entering 'third_party/XNNPACK' 2024-08-20T22:11:47.9005509Z Entering 'third_party/benchmark' 2024-08-20T22:11:47.9056849Z Entering 'third_party/cpp-httplib' 2024-08-20T22:11:47.9109402Z Entering 'third_party/cpuinfo' 2024-08-20T22:11:47.9162551Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:11:47.9215537Z Entering 'third_party/cutlass' 2024-08-20T22:11:47.9277731Z Entering 'third_party/eigen' 2024-08-20T22:11:47.9330918Z Entering 'third_party/fbgemm' 2024-08-20T22:11:47.9382588Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:11:47.9433795Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:11:47.9486756Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:11:47.9544253Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:11:47.9594497Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:11:47.9648353Z Entering 'third_party/flatbuffers' 2024-08-20T22:11:47.9703686Z Entering 'third_party/fmt' 2024-08-20T22:11:47.9756674Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:11:47.9809229Z Entering 'third_party/gloo' 2024-08-20T22:11:47.9861758Z Entering 'third_party/googletest' 2024-08-20T22:11:47.9913937Z Entering 'third_party/ideep' 2024-08-20T22:11:47.9964880Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:11:48.0023364Z Entering 'third_party/ittapi' 2024-08-20T22:11:48.0073251Z Entering 'third_party/kineto' 2024-08-20T22:11:48.0123487Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:11:48.0173360Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:11:48.0225560Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:11:48.0277257Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:11:48.0325460Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:11:48.0373757Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:11:48.0432923Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:11:48.0482825Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:11:48.0533030Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:11:48.0584889Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:11:48.0636906Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:11:48.0692698Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:11:48.0745464Z Entering 'third_party/mimalloc' 2024-08-20T22:11:48.0799032Z Entering 'third_party/nccl/nccl' 2024-08-20T22:11:48.0852839Z Entering 'third_party/nlohmann' 2024-08-20T22:11:48.0906719Z Entering 'third_party/onnx' 2024-08-20T22:11:48.0974436Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:11:48.1025914Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:11:48.1082566Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:11:48.1136205Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:11:48.1186035Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:11:48.1234583Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:11:48.1285668Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:11:48.1335538Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:11:48.1387365Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:11:48.1440175Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:11:48.1491987Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:11:48.1544566Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:11:48.1598520Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:11:48.1670913Z Entering 'third_party/pocketfft' 2024-08-20T22:11:48.1722928Z Entering 'third_party/protobuf' 2024-08-20T22:11:48.1779014Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:11:48.1831476Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:11:48.1885726Z Entering 'third_party/psimd' 2024-08-20T22:11:48.1938355Z Entering 'third_party/pthreadpool' 2024-08-20T22:11:48.1991080Z Entering 'third_party/pybind11' 2024-08-20T22:11:48.2043767Z Entering 'third_party/python-peachpy' 2024-08-20T22:11:48.2096888Z Entering 'third_party/sleef' 2024-08-20T22:11:48.2150177Z Entering 'third_party/tensorpipe' 2024-08-20T22:11:48.2202104Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:11:48.2254032Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:11:48.2304192Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:11:48.2358514Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:11:48.2406603Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:11:48.2483500Z ##[endgroup] 2024-08-20T22:11:48.2486418Z ##[group]Persisting credentials for submodules 2024-08-20T22:11:48.2489264Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-20T22:11:48.2855557Z Entering 'android/libs/fbjni' 2024-08-20T22:11:48.2923878Z Entering 'third_party/FP16' 2024-08-20T22:11:48.2996544Z Entering 'third_party/FXdiv' 2024-08-20T22:11:48.3068363Z Entering 'third_party/NNPACK' 2024-08-20T22:11:48.3136165Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:11:48.3208769Z Entering 'third_party/XNNPACK' 2024-08-20T22:11:48.3292904Z Entering 'third_party/benchmark' 2024-08-20T22:11:48.3361512Z Entering 'third_party/cpp-httplib' 2024-08-20T22:11:48.3428336Z Entering 'third_party/cpuinfo' 2024-08-20T22:11:48.3497210Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:11:48.3568999Z Entering 'third_party/cutlass' 2024-08-20T22:11:48.3643105Z Entering 'third_party/eigen' 2024-08-20T22:11:48.3714450Z Entering 'third_party/fbgemm' 2024-08-20T22:11:48.3781594Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:11:48.3850628Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:11:48.3916743Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:11:48.3989360Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:11:48.4055724Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:11:48.4125183Z Entering 'third_party/flatbuffers' 2024-08-20T22:11:48.4196605Z Entering 'third_party/fmt' 2024-08-20T22:11:48.4264195Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:11:48.4336639Z Entering 'third_party/gloo' 2024-08-20T22:11:48.4404464Z Entering 'third_party/googletest' 2024-08-20T22:11:48.4477167Z Entering 'third_party/ideep' 2024-08-20T22:11:48.4541513Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:11:48.4616138Z Entering 'third_party/ittapi' 2024-08-20T22:11:48.4687907Z Entering 'third_party/kineto' 2024-08-20T22:11:48.4759839Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:11:48.4825027Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:11:48.4894207Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:11:48.4960784Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:11:48.5027596Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:11:48.5094562Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:11:48.5165995Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:11:48.5233654Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:11:48.5301421Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:11:48.5370696Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:11:48.5441334Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:11:48.5512178Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:11:48.5582364Z Entering 'third_party/mimalloc' 2024-08-20T22:11:48.5651887Z Entering 'third_party/nccl/nccl' 2024-08-20T22:11:48.5719785Z Entering 'third_party/nlohmann' 2024-08-20T22:11:48.5788869Z Entering 'third_party/onnx' 2024-08-20T22:11:48.5868626Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:11:48.5937440Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:11:48.6012461Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:11:48.6083955Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:11:48.6151317Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:11:48.6217199Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:11:48.6285541Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:11:48.6354118Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:11:48.6420671Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:11:48.6487878Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:11:48.6552685Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:11:48.6620550Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:11:48.6695633Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:11:48.6783039Z Entering 'third_party/pocketfft' 2024-08-20T22:11:48.6850110Z Entering 'third_party/protobuf' 2024-08-20T22:11:48.6922009Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:11:48.6988595Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:11:48.7059715Z Entering 'third_party/psimd' 2024-08-20T22:11:48.7127699Z Entering 'third_party/pthreadpool' 2024-08-20T22:11:48.7195605Z Entering 'third_party/pybind11' 2024-08-20T22:11:48.7267850Z Entering 'third_party/python-peachpy' 2024-08-20T22:11:48.7334767Z Entering 'third_party/sleef' 2024-08-20T22:11:48.7407653Z Entering 'third_party/tensorpipe' 2024-08-20T22:11:48.7479052Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:11:48.7544936Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:11:48.7612351Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:11:48.7677929Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:11:48.7740810Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:11:48.7836368Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-20T22:11:48.8207987Z Entering 'android/libs/fbjni' 2024-08-20T22:11:48.8270520Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-20T22:11:48.8292174Z Entering 'third_party/FP16' 2024-08-20T22:11:48.8354369Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-20T22:11:48.8376008Z Entering 'third_party/FXdiv' 2024-08-20T22:11:48.8443001Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-20T22:11:48.8464887Z Entering 'third_party/NNPACK' 2024-08-20T22:11:48.8531506Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-20T22:11:48.8553710Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:11:48.8614961Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-20T22:11:48.8636522Z Entering 'third_party/XNNPACK' 2024-08-20T22:11:48.8698689Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-20T22:11:48.8735633Z Entering 'third_party/benchmark' 2024-08-20T22:11:48.8798013Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:11:48.8819450Z Entering 'third_party/cpp-httplib' 2024-08-20T22:11:48.8896428Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-20T22:11:48.8909941Z Entering 'third_party/cpuinfo' 2024-08-20T22:11:48.8971657Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:11:48.8994081Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:11:48.9060922Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-20T22:11:48.9082857Z Entering 'third_party/cutlass' 2024-08-20T22:11:48.9143636Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:11:48.9172917Z Entering 'third_party/eigen' 2024-08-20T22:11:48.9233960Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-20T22:11:48.9257743Z Entering 'third_party/fbgemm' 2024-08-20T22:11:48.9324088Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-20T22:11:48.9344451Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:11:48.9406661Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-20T22:11:48.9426092Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:11:48.9487042Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:11:48.9507159Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:11:48.9567044Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:11:48.9593936Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:11:48.9656556Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:11:48.9676295Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:11:48.9737417Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-20T22:11:48.9761873Z Entering 'third_party/flatbuffers' 2024-08-20T22:11:48.9823636Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-20T22:11:48.9847791Z Entering 'third_party/fmt' 2024-08-20T22:11:48.9912236Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:11:48.9933537Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:11:48.9995838Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-20T22:11:49.0017639Z Entering 'third_party/gloo' 2024-08-20T22:11:49.0079656Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-20T22:11:49.0101025Z Entering 'third_party/googletest' 2024-08-20T22:11:49.0162348Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:11:49.0184033Z Entering 'third_party/ideep' 2024-08-20T22:11:49.0245255Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-20T22:11:49.0266250Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:11:49.0327921Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-20T22:11:49.0356074Z Entering 'third_party/ittapi' 2024-08-20T22:11:49.0417598Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-20T22:11:49.0438563Z Entering 'third_party/kineto' 2024-08-20T22:11:49.0501896Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-20T22:11:49.0520826Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:11:49.0584330Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-20T22:11:49.0603517Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:11:49.0665074Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-20T22:11:49.0687167Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:11:49.0750929Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-20T22:11:49.0771891Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:11:49.0835280Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:11:49.0856939Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:11:49.0918247Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-20T22:11:49.0936401Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:11:49.1000710Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-20T22:11:49.1024221Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:11:49.1087570Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-20T22:11:49.1108363Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:11:49.1172641Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:11:49.1193626Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:11:49.1254608Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-20T22:11:49.1276260Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:11:49.1345787Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-20T22:11:49.1369156Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:11:49.1431889Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-20T22:11:49.1453092Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:11:49.1514014Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-20T22:11:49.1537092Z Entering 'third_party/mimalloc' 2024-08-20T22:11:49.1598765Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-20T22:11:49.1621461Z Entering 'third_party/nccl/nccl' 2024-08-20T22:11:49.1684320Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-20T22:11:49.1706796Z Entering 'third_party/nlohmann' 2024-08-20T22:11:49.1769129Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-20T22:11:49.1792626Z Entering 'third_party/onnx' 2024-08-20T22:11:49.1854273Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-20T22:11:49.1889547Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:11:49.1953338Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:11:49.1975483Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:11:49.2037495Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:11:49.2064835Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:11:49.2125993Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-20T22:11:49.2147895Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:11:49.2210414Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:11:49.2230543Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:11:49.2292136Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:11:49.2312348Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:11:49.2375654Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-20T22:11:49.2395701Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:11:49.2458336Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-20T22:11:49.2486212Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:11:49.2547644Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-20T22:11:49.2568521Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:11:49.2634251Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-20T22:11:49.2654521Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:11:49.2716797Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-20T22:11:49.2734734Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:11:49.2795852Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-20T22:11:49.2818745Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:11:49.2885186Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-20T22:11:49.2908509Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:11:49.2968703Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-20T22:11:49.3013176Z Entering 'third_party/pocketfft' 2024-08-20T22:11:49.3078481Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-20T22:11:49.3100802Z Entering 'third_party/protobuf' 2024-08-20T22:11:49.3162293Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-20T22:11:49.3185978Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:11:49.3252008Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:11:49.3272888Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:11:49.3335462Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:11:49.3360549Z Entering 'third_party/psimd' 2024-08-20T22:11:49.3422283Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-20T22:11:49.3443920Z Entering 'third_party/pthreadpool' 2024-08-20T22:11:49.3506059Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-20T22:11:49.3527786Z Entering 'third_party/pybind11' 2024-08-20T22:11:49.3593309Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:11:49.3615562Z Entering 'third_party/python-peachpy' 2024-08-20T22:11:49.3675770Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-20T22:11:49.3698032Z Entering 'third_party/sleef' 2024-08-20T22:11:49.3761470Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-20T22:11:49.3782723Z Entering 'third_party/tensorpipe' 2024-08-20T22:11:49.3844381Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-20T22:11:49.3867252Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:11:49.3927800Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:11:49.3949990Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:11:49.4011651Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-20T22:11:49.4032571Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:11:49.4092867Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-20T22:11:49.4114581Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:11:49.4175173Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:11:49.4195445Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:11:49.4258622Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-20T22:11:49.5719082Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-20T22:11:49.6087417Z Entering 'android/libs/fbjni' 2024-08-20T22:11:49.6139155Z Entering 'third_party/FP16' 2024-08-20T22:11:49.6190832Z Entering 'third_party/FXdiv' 2024-08-20T22:11:49.6242742Z Entering 'third_party/NNPACK' 2024-08-20T22:11:49.6297375Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:11:49.6350584Z Entering 'third_party/XNNPACK' 2024-08-20T22:11:49.6417527Z Entering 'third_party/benchmark' 2024-08-20T22:11:49.6472218Z Entering 'third_party/cpp-httplib' 2024-08-20T22:11:49.6526527Z Entering 'third_party/cpuinfo' 2024-08-20T22:11:49.6581101Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:11:49.6633667Z Entering 'third_party/cutlass' 2024-08-20T22:11:49.6695621Z Entering 'third_party/eigen' 2024-08-20T22:11:49.6752986Z Entering 'third_party/fbgemm' 2024-08-20T22:11:49.6803148Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:11:49.6853414Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:11:49.6905161Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:11:49.6963434Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:11:49.7013637Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:11:49.7065632Z Entering 'third_party/flatbuffers' 2024-08-20T22:11:49.7121220Z Entering 'third_party/fmt' 2024-08-20T22:11:49.7173506Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:11:49.7227523Z Entering 'third_party/gloo' 2024-08-20T22:11:49.7280269Z Entering 'third_party/googletest' 2024-08-20T22:11:49.7331849Z Entering 'third_party/ideep' 2024-08-20T22:11:49.7382355Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:11:49.7440283Z Entering 'third_party/ittapi' 2024-08-20T22:11:49.7492073Z Entering 'third_party/kineto' 2024-08-20T22:11:49.7544212Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:11:49.7595817Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:11:49.7649776Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:11:49.7701973Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:11:49.7753741Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:11:49.7803797Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:11:49.7859648Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:11:49.7911801Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:11:49.7963883Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:11:49.8019201Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:11:49.8072850Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:11:49.8123100Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:11:49.8175654Z Entering 'third_party/mimalloc' 2024-08-20T22:11:49.8226898Z Entering 'third_party/nccl/nccl' 2024-08-20T22:11:49.8277385Z Entering 'third_party/nlohmann' 2024-08-20T22:11:49.8329993Z Entering 'third_party/onnx' 2024-08-20T22:11:49.8398387Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:11:49.8451423Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:11:49.8507076Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:11:49.8562153Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:11:49.8610428Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:11:49.8659537Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:11:49.8707955Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:11:49.8759447Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:11:49.8808586Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:11:49.8857422Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:11:49.8905445Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:11:49.8960880Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:11:49.9014922Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:11:49.9084658Z Entering 'third_party/pocketfft' 2024-08-20T22:11:49.9136959Z Entering 'third_party/protobuf' 2024-08-20T22:11:49.9193016Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:11:49.9243865Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:11:49.9298490Z Entering 'third_party/psimd' 2024-08-20T22:11:49.9351935Z Entering 'third_party/pthreadpool' 2024-08-20T22:11:49.9404481Z Entering 'third_party/pybind11' 2024-08-20T22:11:49.9458674Z Entering 'third_party/python-peachpy' 2024-08-20T22:11:49.9509990Z Entering 'third_party/sleef' 2024-08-20T22:11:49.9564014Z Entering 'third_party/tensorpipe' 2024-08-20T22:11:49.9614825Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:11:49.9666934Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:11:49.9716170Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:11:49.9767563Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:11:49.9816087Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:11:49.9891033Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-20T22:11:50.0261890Z Entering 'android/libs/fbjni' 2024-08-20T22:11:50.0312600Z Entering 'third_party/FP16' 2024-08-20T22:11:50.0366066Z Entering 'third_party/FXdiv' 2024-08-20T22:11:50.0418265Z Entering 'third_party/NNPACK' 2024-08-20T22:11:50.0471926Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:11:50.0524038Z Entering 'third_party/XNNPACK' 2024-08-20T22:11:50.0592499Z Entering 'third_party/benchmark' 2024-08-20T22:11:50.0644935Z Entering 'third_party/cpp-httplib' 2024-08-20T22:11:50.0697572Z Entering 'third_party/cpuinfo' 2024-08-20T22:11:50.0754647Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:11:50.0807343Z Entering 'third_party/cutlass' 2024-08-20T22:11:50.0867386Z Entering 'third_party/eigen' 2024-08-20T22:11:50.0921633Z Entering 'third_party/fbgemm' 2024-08-20T22:11:50.0974104Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:11:50.1026808Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:11:50.1077526Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:11:50.1135503Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:11:50.1187036Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:11:50.1239741Z Entering 'third_party/flatbuffers' 2024-08-20T22:11:50.1295322Z Entering 'third_party/fmt' 2024-08-20T22:11:50.1349264Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:11:50.1402623Z Entering 'third_party/gloo' 2024-08-20T22:11:50.1456074Z Entering 'third_party/googletest' 2024-08-20T22:11:50.1508162Z Entering 'third_party/ideep' 2024-08-20T22:11:50.1558761Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:11:50.1617119Z Entering 'third_party/ittapi' 2024-08-20T22:11:50.1674073Z Entering 'third_party/kineto' 2024-08-20T22:11:50.1725593Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:11:50.1778473Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:11:50.1830013Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:11:50.1882081Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:11:50.1931848Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:11:50.1980232Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:11:50.2035878Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:11:50.2087740Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:11:50.2140189Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:11:50.2193315Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:11:50.2246958Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:11:50.2300047Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:11:50.2354707Z Entering 'third_party/mimalloc' 2024-08-20T22:11:50.2407189Z Entering 'third_party/nccl/nccl' 2024-08-20T22:11:50.2460121Z Entering 'third_party/nlohmann' 2024-08-20T22:11:50.2513405Z Entering 'third_party/onnx' 2024-08-20T22:11:50.2581243Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:11:50.2632185Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:11:50.2689864Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:11:50.2745058Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:11:50.2796219Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:11:50.2847457Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:11:50.2901926Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:11:50.2953886Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:11:50.3004737Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:11:50.3059166Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:11:50.3108488Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:11:50.3161100Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:11:50.3215245Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:11:50.3288563Z Entering 'third_party/pocketfft' 2024-08-20T22:11:50.3340525Z Entering 'third_party/protobuf' 2024-08-20T22:11:50.3393735Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:11:50.3444888Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:11:50.3499707Z Entering 'third_party/psimd' 2024-08-20T22:11:50.3555584Z Entering 'third_party/pthreadpool' 2024-08-20T22:11:50.3607338Z Entering 'third_party/pybind11' 2024-08-20T22:11:50.3664723Z Entering 'third_party/python-peachpy' 2024-08-20T22:11:50.3717194Z Entering 'third_party/sleef' 2024-08-20T22:11:50.3770037Z Entering 'third_party/tensorpipe' 2024-08-20T22:11:50.3821606Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:11:50.3874100Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:11:50.3924015Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:11:50.3975061Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:11:50.4024315Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:11:50.4097501Z ##[endgroup] 2024-08-20T22:11:50.4145723Z [command]/usr/bin/git log -1 --format='%H' 2024-08-20T22:11:50.4182188Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-20T22:11:50.4692918Z Prepare all required actions 2024-08-20T22:11:50.4693611Z Getting action download info 2024-08-20T22:11:50.6074635Z ##[group]Run ./.github/actions/setup-linux 2024-08-20T22:11:50.6075076Z env: 2024-08-20T22:11:50.6075365Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:50.6075712Z ##[endgroup] 2024-08-20T22:11:50.6135533Z ##[group]Run set -euo pipefail 2024-08-20T22:11:50.6135962Z set -euo pipefail 2024-08-20T22:11:50.6136359Z function get_ec2_metadata() { 2024-08-20T22:11:50.6136900Z  # Pulled from instance metadata endpoint for EC2 2024-08-20T22:11:50.6137794Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-20T22:11:50.6138755Z  category=$1 2024-08-20T22:11:50.6139379Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-20T22:11:50.6140025Z  runner_name_str=i-0aa26b6dbaf56f3e9 2024-08-20T22:11:50.6140527Z  if [[ -f /.inarc ]]; then 2024-08-20T22:11:50.6141034Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-20T22:11:50.6141607Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-20T22:11:50.6142302Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-20T22:11:50.6142909Z  else 2024-08-20T22:11:50.6143412Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-20T22:11:50.6143995Z  fi 2024-08-20T22:11:50.6144284Z } 2024-08-20T22:11:50.6144648Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-20T22:11:50.6145252Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-20T22:11:50.6145916Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-20T22:11:50.6146739Z echo "system info $(uname -a)" 2024-08-20T22:11:50.6157488Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:50.6158004Z env: 2024-08-20T22:11:50.6158301Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:50.6158664Z ##[endgroup] 2024-08-20T22:11:50.6272143Z ami-id: ami-06c68f701d8090592 2024-08-20T22:11:50.6330356Z instance-id: i-0aa26b6dbaf56f3e9 2024-08-20T22:11:50.6392150Z instance-type: g5.4xlarge 2024-08-20T22:11:50.6407321Z system info Linux ip-10-0-67-7.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-20T22:11:50.6431612Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:11:50.6432530Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:11:50.6441705Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:50.6442210Z env: 2024-08-20T22:11:50.6442502Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:50.6442862Z ##[endgroup] 2024-08-20T22:11:50.6523757Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-20T22:11:50.6524385Z if systemctl is-active --quiet docker; then 2024-08-20T22:11:50.6524933Z  echo "Docker daemon is running..."; 2024-08-20T22:11:50.6525393Z else 2024-08-20T22:11:50.6525891Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-20T22:11:50.6526482Z fi 2024-08-20T22:11:50.6534664Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:50.6535174Z env: 2024-08-20T22:11:50.6535462Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:50.6535820Z ##[endgroup] 2024-08-20T22:11:50.6625778Z Docker daemon is running... 2024-08-20T22:11:50.6674157Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:11:50.6674747Z with: 2024-08-20T22:11:50.6675022Z shell: bash 2024-08-20T22:11:50.6675322Z timeout_minutes: 5 2024-08-20T22:11:50.6675659Z max_attempts: 3 2024-08-20T22:11:50.6675988Z retry_wait_seconds: 30 2024-08-20T22:11:50.6679167Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-20T22:11:50.6682157Z polling_interval_seconds: 1 2024-08-20T22:11:50.6682676Z warning_on_retry: true 2024-08-20T22:11:50.6683033Z continue_on_error: false 2024-08-20T22:11:50.6683378Z env: 2024-08-20T22:11:50.6683657Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:50.6684025Z AWS_RETRY_MODE: standard 2024-08-20T22:11:50.6684378Z AWS_MAX_ATTEMPTS: 5 2024-08-20T22:11:50.6684734Z AWS_DEFAULT_REGION: us-east-1 2024-08-20T22:11:50.6685120Z ##[endgroup] 2024-08-20T22:11:51.8190928Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:11:51.8191735Z Configure a credential helper to remove this warning. See 2024-08-20T22:11:51.8192592Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:11:51.8193112Z 2024-08-20T22:11:51.8193233Z Login Succeeded 2024-08-20T22:11:52.7238033Z Command completed after 1 attempt(s). 2024-08-20T22:11:52.7302695Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:11:52.7303413Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:11:52.7304087Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:11:52.7314784Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:52.7315291Z env: 2024-08-20T22:11:52.7315579Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:52.7315936Z ##[endgroup] 2024-08-20T22:11:52.7412715Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:11:52.7413485Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:11:52.7414077Z # shellcheck disable=SC2046 2024-08-20T22:11:52.7414542Z docker stop $(docker ps -q) || true 2024-08-20T22:11:52.7415015Z # Prune all of the docker images 2024-08-20T22:11:52.7415465Z docker system prune -af 2024-08-20T22:11:52.7424130Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:52.7424643Z env: 2024-08-20T22:11:52.7424927Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:52.7425295Z ##[endgroup] 2024-08-20T22:11:52.7744479Z "docker stop" requires at least 1 argument. 2024-08-20T22:11:52.7745399Z See 'docker stop --help'. 2024-08-20T22:11:52.7745733Z 2024-08-20T22:11:52.7746035Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-20T22:11:52.7746764Z 2024-08-20T22:11:52.7746967Z Stop one or more running containers 2024-08-20T22:11:52.7959095Z Total reclaimed space: 0B 2024-08-20T22:11:52.8002302Z ##[group]Run set +e 2024-08-20T22:11:52.8002669Z set +e 2024-08-20T22:11:52.8002978Z set -x 2024-08-20T22:11:52.8003276Z  2024-08-20T22:11:52.8003607Z PT_DOMAIN=download.pytorch.org 2024-08-20T22:11:52.8004398Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-20T22:11:52.8005481Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-20T22:11:52.8006236Z # one is returned at random 2024-08-20T22:11:52.8006797Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-20T22:11:52.8007309Z  2024-08-20T22:11:52.8007640Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:11:52.8008268Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-20T22:11:52.8009227Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-20T22:11:52.8009783Z  2024-08-20T22:11:52.8010116Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:11:52.8010675Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-20T22:11:52.8011198Z  exit 1 2024-08-20T22:11:52.8011513Z  fi 2024-08-20T22:11:52.8011806Z fi 2024-08-20T22:11:52.8012089Z  2024-08-20T22:11:52.8012461Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-20T22:11:52.8012983Z  # Clean up any old records first 2024-08-20T22:11:52.8013499Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-20T22:11:52.8014128Z fi 2024-08-20T22:11:52.8014409Z  2024-08-20T22:11:52.8014852Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-20T22:11:52.8015403Z cat /etc/hosts 2024-08-20T22:11:52.8024506Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:52.8025021Z env: 2024-08-20T22:11:52.8025319Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:52.8025682Z ##[endgroup] 2024-08-20T22:11:52.8053840Z + PT_DOMAIN=download.pytorch.org 2024-08-20T22:11:52.8060111Z ++ dig -4 +short download.pytorch.org 2024-08-20T22:11:52.8060590Z ++ tail -n1 2024-08-20T22:11:52.8562052Z + RESOLVED_IP=18.160.10.76 2024-08-20T22:11:52.8562647Z + '[' -z 18.160.10.76 ']' 2024-08-20T22:11:52.8563207Z + grep -r download.pytorch.org /etc/hosts 2024-08-20T22:11:52.8580300Z + echo '18.160.10.76 download.pytorch.org' 2024-08-20T22:11:52.8580864Z + sudo tee -a /etc/hosts 2024-08-20T22:11:53.0845086Z 18.160.10.76 download.pytorch.org 2024-08-20T22:11:53.0868496Z + cat /etc/hosts 2024-08-20T22:11:53.0879101Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-20T22:11:53.0891452Z ::1 localhost6 localhost6.localdomain6 2024-08-20T22:11:53.0891979Z 18.160.10.76 download.pytorch.org 2024-08-20T22:11:53.1047325Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-20T22:11:53.1047956Z with: 2024-08-20T22:11:53.1049081Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.1050328Z docker-build-dir: .ci/docker 2024-08-20T22:11:53.1050730Z working-directory: . 2024-08-20T22:11:53.1051216Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:53.1051756Z force-push: false 2024-08-20T22:11:53.1052079Z env: 2024-08-20T22:11:53.1052374Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:53.1052728Z ##[endgroup] 2024-08-20T22:11:53.1072133Z ##[group]Run set -ex 2024-08-20T22:11:53.1072503Z set -ex 2024-08-20T22:11:53.1072804Z  2024-08-20T22:11:53.1073379Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-20T22:11:53.1074371Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-20T22:11:53.1075181Z # job could then download the pre-built image as usual 2024-08-20T22:11:53.1075937Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-20T22:11:53.1076621Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:53.1077271Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:53.1077843Z  2024-08-20T22:11:53.1078364Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-20T22:11:53.1079002Z  exit 0 2024-08-20T22:11:53.1079316Z else 2024-08-20T22:11:53.1079697Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:53.1080160Z fi 2024-08-20T22:11:53.1080454Z  2024-08-20T22:11:53.1080941Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-20T22:11:53.1081797Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-20T22:11:53.1082593Z  # use it as it is, but first let's extract the tag 2024-08-20T22:11:53.1083309Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-20T22:11:53.1084047Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:53.1084756Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:53.1085319Z else 2024-08-20T22:11:53.1085770Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-20T22:11:53.1086454Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:53.1087523Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:53.1088270Z fi 2024-08-20T22:11:53.1100858Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:53.1101364Z env: 2024-08-20T22:11:53.1101656Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:53.1102025Z REPO_NAME: pytorch 2024-08-20T22:11:53.1103172Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.1104374Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:11:53.1104878Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:53.1105396Z ##[endgroup] 2024-08-20T22:11:53.1138131Z + [[ ! -d .ci/docker ]] 2024-08-20T22:11:53.1138595Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-20T22:11:53.1138980Z + echo skip=false 2024-08-20T22:11:53.1141099Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-20T22:11:53.1147005Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.1148396Z ++ awk -F '[:,]' '{print $2}' 2024-08-20T22:11:53.1175603Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.1177306Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.1179124Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.1214051Z ##[group]Run set +e 2024-08-20T22:11:53.1214510Z set +e 2024-08-20T22:11:53.1214943Z set -x 2024-08-20T22:11:53.1215388Z  2024-08-20T22:11:53.1215792Z login() { 2024-08-20T22:11:53.1216529Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:11:53.1217398Z } 2024-08-20T22:11:53.1217792Z  2024-08-20T22:11:53.1218139Z retry () { 2024-08-20T22:11:53.1218680Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:11:53.1219353Z } 2024-08-20T22:11:53.1219818Z  2024-08-20T22:11:53.1220202Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:11:53.1220742Z  2024-08-20T22:11:53.1221395Z # Check if image already exists, if it does then skip building it 2024-08-20T22:11:53.1222225Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-20T22:11:53.1222783Z  exit 0 2024-08-20T22:11:53.1223238Z fi 2024-08-20T22:11:53.1223632Z  2024-08-20T22:11:53.1224244Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-20T22:11:53.1225202Z # be empty. The default action would be to continue rebuild the image 2024-08-20T22:11:53.1226057Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-20T22:11:53.1226819Z  # if we're on the base branch then use the parent commit 2024-08-20T22:11:53.1227559Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-20T22:11:53.1228126Z else 2024-08-20T22:11:53.1228697Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-20T22:11:53.1229561Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-20T22:11:53.1230151Z fi 2024-08-20T22:11:53.1230482Z  2024-08-20T22:11:53.1230964Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-20T22:11:53.1231558Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:53.1232102Z  2024-08-20T22:11:53.1233071Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-20T22:11:53.1233920Z  exit 0 2024-08-20T22:11:53.1234333Z fi 2024-08-20T22:11:53.1234746Z  2024-08-20T22:11:53.1235321Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-20T22:11:53.1236406Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-20T22:11:53.1237344Z  exit 1 2024-08-20T22:11:53.1237742Z fi 2024-08-20T22:11:53.1238097Z  2024-08-20T22:11:53.1238725Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-20T22:11:53.1239792Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-20T22:11:53.1240808Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-20T22:11:53.1241846Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-20T22:11:53.1243071Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-20T22:11:53.1244068Z fi 2024-08-20T22:11:53.1244538Z  2024-08-20T22:11:53.1244985Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:53.1254562Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:53.1255156Z env: 2024-08-20T22:11:53.1255605Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:53.1256050Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:11:53.1256614Z BASE_REVISION: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:11:53.1258088Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.1259531Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.1260235Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:53.1260957Z ##[endgroup] 2024-08-20T22:11:53.1291342Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:53.1292269Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:53.1293482Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:11:53.1295364Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:53.6586130Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:11:53.6587168Z Configure a credential helper to remove this warning. See 2024-08-20T22:11:53.6588472Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:11:53.6589064Z 2024-08-20T22:11:53.6589218Z Login Succeeded 2024-08-20T22:11:53.6610337Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.8678054Z { 2024-08-20T22:11:53.8678603Z "schemaVersion": 2, 2024-08-20T22:11:53.8679377Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-20T22:11:53.8681125Z "config": { 2024-08-20T22:11:53.8681859Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-20T22:11:53.8682749Z "size": 49627, 2024-08-20T22:11:53.8685976Z "digest": "sha256:cee8d41a96d3e7eccb178c1529633c2dd768035bec694588cf53029b783ac292" 2024-08-20T22:11:53.8687691Z }, 2024-08-20T22:11:53.8688238Z "layers": [ 2024-08-20T22:11:53.8689404Z { 2024-08-20T22:11:53.8690574Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8691575Z "size": 28584317, 2024-08-20T22:11:53.8692554Z "digest": "sha256:63e9bbe323274e77e58d77c6ab6802d247458f784222fbb07a2556d6ec74ee05" 2024-08-20T22:11:53.8693460Z }, 2024-08-20T22:11:53.8693932Z { 2024-08-20T22:11:53.8694915Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8695591Z "size": 7944771, 2024-08-20T22:11:53.8696341Z "digest": "sha256:cfb3d849840ee60cee7b02bad68c1fc3c15928ebcd88f327754766b670578ed6" 2024-08-20T22:11:53.8697160Z }, 2024-08-20T22:11:53.8697524Z { 2024-08-20T22:11:53.8698015Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8698733Z "size": 57593718, 2024-08-20T22:11:53.8699538Z "digest": "sha256:968831e596a6288f0fed9b8612ee4ee8e75511037c4305058805492c5162e481" 2024-08-20T22:11:53.8700286Z }, 2024-08-20T22:11:53.8700593Z { 2024-08-20T22:11:53.8701269Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8701956Z "size": 187, 2024-08-20T22:11:53.8702609Z "digest": "sha256:ea310eb267ca1cab61b6b16f566cd28bfd59a741395a011f5e76716e15ba57c6" 2024-08-20T22:11:53.8704097Z }, 2024-08-20T22:11:53.8704490Z { 2024-08-20T22:11:53.8705031Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8705644Z "size": 6885, 2024-08-20T22:11:53.8706336Z "digest": "sha256:3af11d09e9cd1eb9c379f0a4071231e5a5642eb728b4b33bcb76be291f3c9488" 2024-08-20T22:11:53.8707066Z }, 2024-08-20T22:11:53.8707588Z { 2024-08-20T22:11:53.8708169Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8708871Z "size": 1361380219, 2024-08-20T22:11:53.8709591Z "digest": "sha256:ebfec18059b91e56882881ac34754f917861edb5f732c395d2a1a851bbd6db46" 2024-08-20T22:11:53.8710314Z }, 2024-08-20T22:11:53.8710615Z { 2024-08-20T22:11:53.8711305Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8712025Z "size": 62686, 2024-08-20T22:11:53.8712670Z "digest": "sha256:533b4aebf16914c763b7b0de3ce657590c6f979045e9fdf1f816adaf68d8a4d3" 2024-08-20T22:11:53.8713394Z }, 2024-08-20T22:11:53.8713759Z { 2024-08-20T22:11:53.8714279Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8714956Z "size": 1685, 2024-08-20T22:11:53.8715578Z "digest": "sha256:9dd75d06a0910f28cb1e484b8808724e5a6ee570ecb8fc04631368f546b39ed9" 2024-08-20T22:11:53.8716274Z }, 2024-08-20T22:11:53.8716834Z + exit 0 2024-08-20T22:11:53.8717156Z { 2024-08-20T22:11:53.8717823Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8718469Z "size": 1523, 2024-08-20T22:11:53.8719100Z "digest": "sha256:30bfca4dd3492d60ed8035b0eeb1229897041140db117f5663465a551e25851d" 2024-08-20T22:11:53.8719837Z }, 2024-08-20T22:11:53.8720185Z { 2024-08-20T22:11:53.8720706Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8721422Z "size": 2608186849, 2024-08-20T22:11:53.8722051Z "digest": "sha256:1b57ce94cad9a5097d99b8d6f7bcd51df5a162fc3c5f5686b689b76993724bc8" 2024-08-20T22:11:53.8722762Z }, 2024-08-20T22:11:53.8723173Z { 2024-08-20T22:11:53.8723659Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8724401Z "size": 86631, 2024-08-20T22:11:53.8725084Z "digest": "sha256:9ee6bdb31195dd42fd98147a75d540efe0c5708e0ecda866a0bca060084fddab" 2024-08-20T22:11:53.8725779Z }, 2024-08-20T22:11:53.8726098Z { 2024-08-20T22:11:53.8726669Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8727308Z "size": 1823, 2024-08-20T22:11:53.8728014Z "digest": "sha256:722a6699b4ebdf1fff415451da7fd768fe9423762ae79022b3ef197dd336aba1" 2024-08-20T22:11:53.8728732Z }, 2024-08-20T22:11:53.8729142Z { 2024-08-20T22:11:53.8729737Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8730390Z "size": 245783893, 2024-08-20T22:11:53.8731074Z "digest": "sha256:164218df90f4c2138aa8523cb457e541c046b1165679e431cd9de868b53f2f64" 2024-08-20T22:11:53.8731843Z }, 2024-08-20T22:11:53.8732241Z { 2024-08-20T22:11:53.8732754Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8733506Z "size": 545, 2024-08-20T22:11:53.8734216Z "digest": "sha256:f1d74e82c83914108a9f58ec156ba74104629fffbc1d62bfd5f64a5d6de00338" 2024-08-20T22:11:53.8734924Z }, 2024-08-20T22:11:53.8735235Z { 2024-08-20T22:11:53.8735834Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8736477Z "size": 1259, 2024-08-20T22:11:53.8737099Z "digest": "sha256:13b68a95b298f6875179a04030c62eb4a361b69cac013519115ae86fa102a98a" 2024-08-20T22:11:53.8737820Z }, 2024-08-20T22:11:53.8738241Z { 2024-08-20T22:11:53.8738832Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8739581Z "size": 484, 2024-08-20T22:11:53.8740219Z "digest": "sha256:c95fa6ffe9908fff217023af77b2359f99f9fb5c01519080d04e280191c354ab" 2024-08-20T22:11:53.8740946Z }, 2024-08-20T22:11:53.8741333Z { 2024-08-20T22:11:53.8741819Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8742488Z "size": 91712634, 2024-08-20T22:11:53.8743208Z "digest": "sha256:831d1d1ad5de706debd5baab20657e9694be7412d6d7f775460ad00c5c5bd203" 2024-08-20T22:11:53.8743915Z }, 2024-08-20T22:11:53.8744235Z { 2024-08-20T22:11:53.8744879Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8745647Z "size": 3392, 2024-08-20T22:11:53.8746667Z "digest": "sha256:3116a975d40f2c45dfc323466942aca5d68ba4d81049fb90fb4f9ca02475aca1" 2024-08-20T22:11:53.8747521Z }, 2024-08-20T22:11:53.8747898Z { 2024-08-20T22:11:53.8748571Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8749243Z "size": 1910, 2024-08-20T22:11:53.8749902Z "digest": "sha256:aed1c44ec0f677f20a8336422625eec3f66558ecbc61b9e49856085188b117e9" 2024-08-20T22:11:53.8750608Z }, 2024-08-20T22:11:53.8751028Z { 2024-08-20T22:11:53.8751525Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8752174Z "size": 700, 2024-08-20T22:11:53.8752959Z "digest": "sha256:657a0885dc72f558a3fdaffb85a7d85c2ab25985a395ec9fdb422bd2048a9de4" 2024-08-20T22:11:53.8753678Z }, 2024-08-20T22:11:53.8753990Z { 2024-08-20T22:11:53.8754586Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8755238Z "size": 2880110522, 2024-08-20T22:11:53.8755859Z "digest": "sha256:e834cd84960314101228fad9e84dc2ac3661d07b32945d943eff70e57fa73e67" 2024-08-20T22:11:53.8756631Z }, 2024-08-20T22:11:53.8756972Z { 2024-08-20T22:11:53.8757572Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8758177Z "size": 380, 2024-08-20T22:11:53.8758809Z "digest": "sha256:6d388069c2d7b7900bee880ba0a9bb683165f3978613fec36ed7e3edf04b180b" 2024-08-20T22:11:53.8759650Z }, 2024-08-20T22:11:53.8759989Z { 2024-08-20T22:11:53.8760481Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8761190Z "size": 12906, 2024-08-20T22:11:53.8761819Z "digest": "sha256:51d2d37470818e64147c1b99897175234adecf55386b5ea8da0e800d80bd24b8" 2024-08-20T22:11:53.8762541Z }, 2024-08-20T22:11:53.8762894Z { 2024-08-20T22:11:53.8763422Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8764083Z "size": 805, 2024-08-20T22:11:53.8764728Z "digest": "sha256:84d7c1aff0165ae94c936361c872a5d98a96f81286d4901bfa4b8f8a6e0ed869" 2024-08-20T22:11:53.8765431Z }, 2024-08-20T22:11:53.8765795Z { 2024-08-20T22:11:53.8766434Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8767043Z "size": 106, 2024-08-20T22:11:53.8767701Z "digest": "sha256:ce74fbfc871d05c302bc99afff486a44c401c4d7fc1f2856d593e4d7c8e61a5c" 2024-08-20T22:11:53.8768461Z }, 2024-08-20T22:11:53.8768821Z { 2024-08-20T22:11:53.8769311Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8770005Z "size": 504, 2024-08-20T22:11:53.8770644Z "digest": "sha256:0972844ea4ec494726981a800e4a687d65d178eda00ef010e5f4031042e7b052" 2024-08-20T22:11:53.8771500Z }, 2024-08-20T22:11:53.8771867Z { 2024-08-20T22:11:53.8772410Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8773125Z "size": 121477355, 2024-08-20T22:11:53.8773803Z "digest": "sha256:20952c6c7b153921a34a3deca88e1bd7373bbefca60bea51531ae313dd6ad858" 2024-08-20T22:11:53.8774518Z }, 2024-08-20T22:11:53.8774849Z { 2024-08-20T22:11:53.8775444Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8776044Z "size": 109, 2024-08-20T22:11:53.8776654Z "digest": "sha256:98f539161af00f8a9e6085712bde8384f11669fb341a1387672b3814c3ebe1ab" 2024-08-20T22:11:53.8777415Z }, 2024-08-20T22:11:53.8777747Z { 2024-08-20T22:11:53.8778225Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8778982Z "size": 489, 2024-08-20T22:11:53.8779781Z "digest": "sha256:0183570ab6f1b76f21e0bd1aebd83ac04068546eaf762334a3398f1726b954c2" 2024-08-20T22:11:53.8780444Z }, 2024-08-20T22:11:53.8780870Z { 2024-08-20T22:11:53.8781380Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8782085Z "size": 297, 2024-08-20T22:11:53.8782683Z "digest": "sha256:58e41754815ca62fced600e17f7a2dd2ac4e1c5f759d64c3b72e9d390afe94db" 2024-08-20T22:11:53.8783531Z }, 2024-08-20T22:11:53.8783944Z { 2024-08-20T22:11:53.8784457Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8785077Z "size": 103, 2024-08-20T22:11:53.8785742Z "digest": "sha256:f16e739cc132112eb4aee330a7096c38d3063a202f55ddc660b172da4c94c278" 2024-08-20T22:11:53.8786506Z }, 2024-08-20T22:11:53.8786828Z { 2024-08-20T22:11:53.8787388Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8788023Z "size": 1473, 2024-08-20T22:11:53.8788725Z "digest": "sha256:ddc70883fbfec3c44ac5b825395f40dc6a3f2201314f4242f197735abcfae4f5" 2024-08-20T22:11:53.8789448Z }, 2024-08-20T22:11:53.8789795Z { 2024-08-20T22:11:53.8790330Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8791026Z "size": 594731067, 2024-08-20T22:11:53.8791655Z "digest": "sha256:3c5150459092a973024c73ab1af7d259d27bc39e1f0f6d3b321d5b1421c58ef8" 2024-08-20T22:11:53.8792339Z }, 2024-08-20T22:11:53.8792737Z { 2024-08-20T22:11:53.8793343Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8793942Z "size": 164, 2024-08-20T22:11:53.8794612Z "digest": "sha256:697db697809d30a05a9d159e1ca0c4ff57743783a5dd677bd048c3b72c5c2e74" 2024-08-20T22:11:53.8795329Z }, 2024-08-20T22:11:53.8795628Z { 2024-08-20T22:11:53.8796200Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8796862Z "size": 563, 2024-08-20T22:11:53.8797477Z "digest": "sha256:01240c436d2fed24c59304ba81b8c20d47365178244a3eafd102c03ed2a8d640" 2024-08-20T22:11:53.8798236Z }, 2024-08-20T22:11:53.8798572Z { 2024-08-20T22:11:53.8799089Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8799870Z "size": 35865715, 2024-08-20T22:11:53.8800477Z "digest": "sha256:526c21e3e021736f8e11a7c72706caed8b29a6ef45b8acd1a8ce101bda1d35cb" 2024-08-20T22:11:53.8801274Z }, 2024-08-20T22:11:53.8801669Z { 2024-08-20T22:11:53.8802164Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8802835Z "size": 104, 2024-08-20T22:11:53.8803505Z "digest": "sha256:8a1507f1f6002a8b82b720956e1fe6a67455c651a95a90db071880112ea83dd8" 2024-08-20T22:11:53.8804199Z }, 2024-08-20T22:11:53.8804536Z { 2024-08-20T22:11:53.8805103Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8805751Z "size": 425, 2024-08-20T22:11:53.8806471Z "digest": "sha256:efb435cd2eaa405daeb98ce6116c948b9d3290850c23691461be7310bec69bdd" 2024-08-20T22:11:53.8807256Z }, 2024-08-20T22:11:53.8807616Z { 2024-08-20T22:11:53.8808190Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8808933Z "size": 20262177, 2024-08-20T22:11:53.8809616Z "digest": "sha256:6b78a8535b2ce76b1dd44cada752d9cb93b90df8b51ee855854febe7ab2cb7b9" 2024-08-20T22:11:53.8810387Z }, 2024-08-20T22:11:53.8810712Z { 2024-08-20T22:11:53.8811234Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8811930Z "size": 440, 2024-08-20T22:11:53.8812539Z "digest": "sha256:1a5959b2d611f7a5d21dbb0c5f25c9315be475b25545e77317037b366a55aeba" 2024-08-20T22:11:53.8813239Z }, 2024-08-20T22:11:53.8813699Z { 2024-08-20T22:11:53.8814243Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8814846Z "size": 700, 2024-08-20T22:11:53.8815530Z "digest": "sha256:657a0885dc72f558a3fdaffb85a7d85c2ab25985a395ec9fdb422bd2048a9de4" 2024-08-20T22:11:53.8816259Z }, 2024-08-20T22:11:53.8816603Z { 2024-08-20T22:11:53.8817181Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8817858Z "size": 142, 2024-08-20T22:11:53.8818536Z "digest": "sha256:ec22bfa269b65f52c12ea974dbbbe850632692df1636da93618fb1cd5ba2759d" 2024-08-20T22:11:53.8819418Z }, 2024-08-20T22:11:53.8819729Z { 2024-08-20T22:11:53.8820337Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8821166Z "size": 135, 2024-08-20T22:11:53.8821770Z "digest": "sha256:143fcaf620b2ae761c579c3687fa0aace809e7f758b89f9538fc40777e9be91c" 2024-08-20T22:11:53.8822503Z }, 2024-08-20T22:11:53.8822896Z { 2024-08-20T22:11:53.8823431Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8824060Z "size": 4838067845, 2024-08-20T22:11:53.8824751Z "digest": "sha256:d9ebe2872e744f11fc55059a7be35140cad3699967c198ec877ad046d4e873ef" 2024-08-20T22:11:53.8825441Z }, 2024-08-20T22:11:53.8825842Z { 2024-08-20T22:11:53.8826325Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8827051Z "size": 195, 2024-08-20T22:11:53.8827739Z "digest": "sha256:2dfbafd7cb9fa64cc49904c931bfcefd64fff2a49ef4912b00540f96f2d6d11e" 2024-08-20T22:11:53.8828424Z }, 2024-08-20T22:11:53.8828781Z { 2024-08-20T22:11:53.8829344Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8829982Z "size": 566, 2024-08-20T22:11:53.8830591Z "digest": "sha256:5010474278e0c7ee4080071b6a3619c6db59e42e10ec9c7528ef3cd983e4a537" 2024-08-20T22:11:53.8831327Z }, 2024-08-20T22:11:53.8831658Z { 2024-08-20T22:11:53.8832191Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8832841Z "size": 43163542, 2024-08-20T22:11:53.8833498Z "digest": "sha256:62e2be9188ac5963bd23a889747b73501ea9712d1d9e09584cecefc6b30f9c1a" 2024-08-20T22:11:53.8834271Z }, 2024-08-20T22:11:53.8834633Z { 2024-08-20T22:11:53.8835189Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8835832Z "size": 106, 2024-08-20T22:11:53.8836515Z "digest": "sha256:8738005d6660c2efca91d8b34cb3f269d342d6a9c37b2d9c004c61aad6993e3f" 2024-08-20T22:11:53.8837210Z }, 2024-08-20T22:11:53.8837558Z { 2024-08-20T22:11:53.8838134Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8838785Z "size": 1380, 2024-08-20T22:11:53.8839366Z "digest": "sha256:7397e0c0192133e7084b2cdb03325a59350b480527751a8384b95ee1b73ff40b" 2024-08-20T22:11:53.8840120Z }, 2024-08-20T22:11:53.8840516Z { 2024-08-20T22:11:53.8841045Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8841733Z "size": 700, 2024-08-20T22:11:53.8842372Z "digest": "sha256:657a0885dc72f558a3fdaffb85a7d85c2ab25985a395ec9fdb422bd2048a9de4" 2024-08-20T22:11:53.8843086Z }, 2024-08-20T22:11:53.8843466Z { 2024-08-20T22:11:53.8843990Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8844631Z "size": 140, 2024-08-20T22:11:53.8845335Z "digest": "sha256:d31b6d35a64ac472ba535e111b62433faaa9f4f19e2f2bbcf221088e1dcb1dc7" 2024-08-20T22:11:53.8846093Z }, 2024-08-20T22:11:53.8846776Z { 2024-08-20T22:11:53.8847383Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8848080Z "size": 120, 2024-08-20T22:11:53.8848700Z "digest": "sha256:6e06b9b118be0d2c3ab2c4ab2070763bd63142fe0cc29cfe4534493d3a2aa9d5" 2024-08-20T22:11:53.8849455Z }, 2024-08-20T22:11:53.8849819Z { 2024-08-20T22:11:53.8850299Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8850977Z "size": 4324622506, 2024-08-20T22:11:53.8851663Z "digest": "sha256:1ba96cf48ba3c58c7a414e250aecfab00a65acb0c0e4d57d26c0cbbafcc7a826" 2024-08-20T22:11:53.8852430Z }, 2024-08-20T22:11:53.8852734Z { 2024-08-20T22:11:53.8853279Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8853962Z "size": 176, 2024-08-20T22:11:53.8854638Z "digest": "sha256:8269d46e5fda811dab8d04ad8629add72ba91233a8d3394b46c27b9569a00e7d" 2024-08-20T22:11:53.8855322Z }, 2024-08-20T22:11:53.8855709Z { 2024-08-20T22:11:53.8856259Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8856904Z "size": 907, 2024-08-20T22:11:53.8857553Z "digest": "sha256:877a9eaa525ffcd309df81c8ad41a0ce7788becba7998484e9835823532c4103" 2024-08-20T22:11:53.8858432Z }, 2024-08-20T22:11:53.8858782Z { 2024-08-20T22:11:53.8859402Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8860063Z "size": 700, 2024-08-20T22:11:53.8860695Z "digest": "sha256:657a0885dc72f558a3fdaffb85a7d85c2ab25985a395ec9fdb422bd2048a9de4" 2024-08-20T22:11:53.8861549Z }, 2024-08-20T22:11:53.8861850Z { 2024-08-20T22:11:53.8862374Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8863090Z "size": 134, 2024-08-20T22:11:53.8863716Z "digest": "sha256:b4a44fd6dcb4500b86d306c24275b2066f74fc58a6d754701aeafd4ab6503237" 2024-08-20T22:11:53.8864379Z }, 2024-08-20T22:11:53.8864793Z { 2024-08-20T22:11:53.8865306Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8865913Z "size": 32, 2024-08-20T22:11:53.8866616Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:11:53.8867331Z }, 2024-08-20T22:11:53.8867706Z { 2024-08-20T22:11:53.8868336Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8868982Z "size": 157, 2024-08-20T22:11:53.8869643Z "digest": "sha256:3923dee95bc8123433f79dec72afdab61db4bb0b6cee2cf18697e0f670302da7" 2024-08-20T22:11:53.8870405Z }, 2024-08-20T22:11:53.8870709Z { 2024-08-20T22:11:53.8871248Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8871925Z "size": 1843, 2024-08-20T22:11:53.8872585Z "digest": "sha256:3385be080d47f0bf4f800da707b391ef2aff82d4bedb454fbf033f5ffc0ea1f0" 2024-08-20T22:11:53.8873262Z }, 2024-08-20T22:11:53.8873641Z { 2024-08-20T22:11:53.8874173Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8874779Z "size": 7529774, 2024-08-20T22:11:53.8875510Z "digest": "sha256:fb3b968bdf0d9633f908cd7e2980ca6eb34998708148d9d50bed730d3d00a311" 2024-08-20T22:11:53.8876229Z }, 2024-08-20T22:11:53.8876614Z { 2024-08-20T22:11:53.8877104Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8877755Z "size": 165, 2024-08-20T22:11:53.8878445Z "digest": "sha256:dca079b1da9ded43032fb32760349d22466d7f35d328acb504c3a0915a7d399c" 2024-08-20T22:11:53.8879169Z }, 2024-08-20T22:11:53.8879467Z { 2024-08-20T22:11:53.8880023Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8880677Z "size": 7943, 2024-08-20T22:11:53.8881254Z "digest": "sha256:5809b0e302890af8a867695be09f99403b949cc81b11c292b048b7cd063b4622" 2024-08-20T22:11:53.8882039Z }, 2024-08-20T22:11:53.8882398Z { 2024-08-20T22:11:53.8882908Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8883550Z "size": 8070, 2024-08-20T22:11:53.8884348Z "digest": "sha256:3226b5c3a909ead1adffdfc2b46c9b07b660fd28700f5aae1f7d32dcbbb4af72" 2024-08-20T22:11:53.8885066Z }, 2024-08-20T22:11:53.8885487Z { 2024-08-20T22:11:53.8885983Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8886636Z "size": 304, 2024-08-20T22:11:53.8887326Z "digest": "sha256:16d2c60c3be566081400461f3a4fb263a7efb7038c767ec32afa61ea3a43606f" 2024-08-20T22:11:53.8888026Z }, 2024-08-20T22:11:53.8888386Z { 2024-08-20T22:11:53.8889028Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8889673Z "size": 7630410, 2024-08-20T22:11:53.8890282Z "digest": "sha256:a3645dbfb35189d14514844cc6ede8a6c7b144ffa36a918eeb7da013432d75b9" 2024-08-20T22:11:53.8891050Z }, 2024-08-20T22:11:53.8891399Z { 2024-08-20T22:11:53.8891942Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8892608Z "size": 108, 2024-08-20T22:11:53.8893247Z "digest": "sha256:450f041fd9b29b129617d3242c9ab41f8f9d81b2ce961dab6cd264110e5579f7" 2024-08-20T22:11:53.8893973Z }, 2024-08-20T22:11:53.8894368Z { 2024-08-20T22:11:53.8894858Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8895676Z "size": 54145659, 2024-08-20T22:11:53.8896376Z "digest": "sha256:fa90062cf4e64751f618b4107a8c0157b0b2d8d37da6b413579a9d03f54c5ce6" 2024-08-20T22:11:53.8897092Z }, 2024-08-20T22:11:53.8897397Z { 2024-08-20T22:11:53.8897972Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8898667Z "size": 474, 2024-08-20T22:11:53.8899350Z "digest": "sha256:c3392f1762d82dc09ebc3f2846d9d7efcea9dc0dd40d433c3f5eee69362d7705" 2024-08-20T22:11:53.8900113Z }, 2024-08-20T22:11:53.8900469Z { 2024-08-20T22:11:53.8901047Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8901675Z "size": 1374858412, 2024-08-20T22:11:53.8902370Z "digest": "sha256:e52dd7f40be6e1973f87fba1296f2bba0461107e2cc03c1bbb6468151ae99554" 2024-08-20T22:11:53.8903120Z }, 2024-08-20T22:11:53.8903475Z { 2024-08-20T22:11:53.8903953Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8904631Z "size": 106, 2024-08-20T22:11:53.8905282Z "digest": "sha256:87b663a94d66bfc4467688beab83925679fbd74e70bc85b2ef97dffd8b40e004" 2024-08-20T22:11:53.8905944Z }, 2024-08-20T22:11:53.8906327Z { 2024-08-20T22:11:53.8906862Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8907489Z "size": 559, 2024-08-20T22:11:53.8908190Z "digest": "sha256:1df0f571a9950af44d446e82235480d3626daf9c3aea3ed59a7ac23b36fdf449" 2024-08-20T22:11:53.8908931Z }, 2024-08-20T22:11:53.8909264Z { 2024-08-20T22:11:53.8909841Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8910439Z "size": 46248351, 2024-08-20T22:11:53.8911080Z "digest": "sha256:9fb55636587d5dfac735bccabbe62dc6802da082dba88c9859adb8402ff72fb0" 2024-08-20T22:11:53.8911857Z }, 2024-08-20T22:11:53.8912153Z { 2024-08-20T22:11:53.8912669Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8913368Z "size": 111, 2024-08-20T22:11:53.8914006Z "digest": "sha256:6f5f644dad086485e9dab47c9e119d092d3db8ea0d70ba65e5683fd8ae2c4813" 2024-08-20T22:11:53.8914696Z }, 2024-08-20T22:11:53.8915081Z { 2024-08-20T22:11:53.8915647Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8916310Z "size": 32, 2024-08-20T22:11:53.8916959Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:11:53.8917664Z }, 2024-08-20T22:11:53.8918019Z { 2024-08-20T22:11:53.8918595Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8919240Z "size": 32, 2024-08-20T22:11:53.8919881Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:11:53.8920642Z }, 2024-08-20T22:11:53.8921060Z { 2024-08-20T22:11:53.8921571Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8922327Z "size": 32, 2024-08-20T22:11:53.8922973Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:11:53.8923632Z }, 2024-08-20T22:11:53.8924034Z { 2024-08-20T22:11:53.8924549Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:53.8925229Z "size": 32, 2024-08-20T22:11:53.8925835Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:11:53.8926537Z } 2024-08-20T22:11:53.8926915Z ] 2024-08-20T22:11:53.8927238Z } 2024-08-20T22:11:53.9225185Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:11:53.9225689Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:11:53.9226242Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-20T22:11:53.9235424Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:53.9235931Z env: 2024-08-20T22:11:53.9236210Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:53.9237378Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.9238560Z ##[endgroup] 2024-08-20T22:11:53.9270474Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.9322231Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-20T22:11:53.9322810Z with: 2024-08-20T22:11:53.9323887Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.9325205Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:53.9325729Z env: 2024-08-20T22:11:53.9326019Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:53.9326389Z ##[endgroup] 2024-08-20T22:11:53.9343785Z ##[group]Run set -x 2024-08-20T22:11:53.9344140Z set -x 2024-08-20T22:11:53.9344449Z set +e 2024-08-20T22:11:53.9344750Z  2024-08-20T22:11:53.9345030Z login() { 2024-08-20T22:11:53.9345700Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:11:53.9346783Z } 2024-08-20T22:11:53.9347069Z  2024-08-20T22:11:53.9347376Z retry () { 2024-08-20T22:11:53.9347777Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:11:53.9348267Z } 2024-08-20T22:11:53.9348558Z  2024-08-20T22:11:53.9348882Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:11:53.9349299Z  2024-08-20T22:11:53.9349577Z set -e 2024-08-20T22:11:53.9350058Z # ignore output since only exit code is used for conditional 2024-08-20T22:11:53.9350769Z # only pull docker image if it's not available locally 2024-08-20T22:11:53.9351550Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-20T22:11:53.9352248Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-20T22:11:53.9352694Z fi 2024-08-20T22:11:53.9360846Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:53.9361351Z env: 2024-08-20T22:11:53.9361639Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:53.9362808Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.9364139Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:53.9364663Z ##[endgroup] 2024-08-20T22:11:53.9395923Z + set +e 2024-08-20T22:11:53.9396722Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:53.9397403Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:53.9400047Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:11:53.9400840Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:54.4670556Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:11:54.4671466Z Configure a credential helper to remove this warning. See 2024-08-20T22:11:54.4672427Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:11:54.4673036Z 2024-08-20T22:11:54.4673172Z Login Succeeded 2024-08-20T22:11:54.4693373Z + set -e 2024-08-20T22:11:54.4694701Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:54.4860894Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:54.4863083Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:54.7341505Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks 2024-08-20T22:11:54.7343298Z 63e9bbe32327: Pulling fs layer 2024-08-20T22:11:54.7343873Z cfb3d849840e: Pulling fs layer 2024-08-20T22:11:54.7344876Z 968831e596a6: Pulling fs layer 2024-08-20T22:11:54.7345396Z ea310eb267ca: Pulling fs layer 2024-08-20T22:11:54.7345920Z 3af11d09e9cd: Pulling fs layer 2024-08-20T22:11:54.7346720Z ebfec18059b9: Pulling fs layer 2024-08-20T22:11:54.7347128Z 533b4aebf169: Pulling fs layer 2024-08-20T22:11:54.7347512Z 9dd75d06a091: Pulling fs layer 2024-08-20T22:11:54.7348017Z 30bfca4dd349: Pulling fs layer 2024-08-20T22:11:54.7348410Z 1b57ce94cad9: Pulling fs layer 2024-08-20T22:11:54.7348787Z 9ee6bdb31195: Pulling fs layer 2024-08-20T22:11:54.7349174Z 722a6699b4eb: Pulling fs layer 2024-08-20T22:11:54.7349666Z 164218df90f4: Pulling fs layer 2024-08-20T22:11:54.7350173Z f1d74e82c839: Pulling fs layer 2024-08-20T22:11:54.7350676Z 13b68a95b298: Pulling fs layer 2024-08-20T22:11:54.7351157Z c95fa6ffe990: Pulling fs layer 2024-08-20T22:11:54.7351530Z 831d1d1ad5de: Pulling fs layer 2024-08-20T22:11:54.7352018Z 3116a975d40f: Pulling fs layer 2024-08-20T22:11:54.7352531Z aed1c44ec0f6: Pulling fs layer 2024-08-20T22:11:54.7352941Z 657a0885dc72: Pulling fs layer 2024-08-20T22:11:54.7353309Z e834cd849603: Pulling fs layer 2024-08-20T22:11:54.7353685Z 6d388069c2d7: Pulling fs layer 2024-08-20T22:11:54.7354058Z 51d2d3747081: Pulling fs layer 2024-08-20T22:11:54.7354431Z 84d7c1aff016: Pulling fs layer 2024-08-20T22:11:54.7354802Z ce74fbfc871d: Pulling fs layer 2024-08-20T22:11:54.7355180Z 0972844ea4ec: Pulling fs layer 2024-08-20T22:11:54.7355554Z 20952c6c7b15: Pulling fs layer 2024-08-20T22:11:54.7355939Z 98f539161af0: Pulling fs layer 2024-08-20T22:11:54.7356396Z 0183570ab6f1: Pulling fs layer 2024-08-20T22:11:54.7356896Z 58e41754815c: Pulling fs layer 2024-08-20T22:11:54.7357387Z f16e739cc132: Pulling fs layer 2024-08-20T22:11:54.7357884Z ddc70883fbfe: Pulling fs layer 2024-08-20T22:11:54.7358383Z 3c5150459092: Pulling fs layer 2024-08-20T22:11:54.7358874Z 697db697809d: Pulling fs layer 2024-08-20T22:11:54.7359277Z 01240c436d2f: Pulling fs layer 2024-08-20T22:11:54.7359632Z 533b4aebf169: Waiting 2024-08-20T22:11:54.7359978Z 526c21e3e021: Pulling fs layer 2024-08-20T22:11:54.7360353Z 8a1507f1f600: Pulling fs layer 2024-08-20T22:11:54.7360730Z efb435cd2eaa: Pulling fs layer 2024-08-20T22:11:54.7361085Z 9dd75d06a091: Waiting 2024-08-20T22:11:54.7361408Z ea310eb267ca: Waiting 2024-08-20T22:11:54.7361745Z 6b78a8535b2c: Pulling fs layer 2024-08-20T22:11:54.7362104Z 3af11d09e9cd: Waiting 2024-08-20T22:11:54.7362426Z 9ee6bdb31195: Waiting 2024-08-20T22:11:54.7362743Z 722a6699b4eb: Waiting 2024-08-20T22:11:54.7363073Z 1a5959b2d611: Pulling fs layer 2024-08-20T22:11:54.7363646Z ec22bfa269b6: Pulling fs layer 2024-08-20T22:11:54.7364010Z 831d1d1ad5de: Waiting 2024-08-20T22:11:54.7364346Z 143fcaf620b2: Pulling fs layer 2024-08-20T22:11:54.7364725Z d9ebe2872e74: Pulling fs layer 2024-08-20T22:11:54.7365106Z 2dfbafd7cb9f: Pulling fs layer 2024-08-20T22:11:54.7365523Z 5010474278e0: Pulling fs layer 2024-08-20T22:11:54.7365883Z ebfec18059b9: Waiting 2024-08-20T22:11:54.7366222Z 62e2be9188ac: Pulling fs layer 2024-08-20T22:11:54.7366595Z 8738005d6660: Pulling fs layer 2024-08-20T22:11:54.7366966Z 7397e0c01921: Pulling fs layer 2024-08-20T22:11:54.7367325Z 3116a975d40f: Waiting 2024-08-20T22:11:54.7367661Z d31b6d35a64a: Pulling fs layer 2024-08-20T22:11:54.7368064Z 6e06b9b118be: Pulling fs layer 2024-08-20T22:11:54.7368448Z e834cd849603: Waiting 2024-08-20T22:11:54.7368786Z 1ba96cf48ba3: Pulling fs layer 2024-08-20T22:11:54.7369151Z aed1c44ec0f6: Waiting 2024-08-20T22:11:54.7369467Z 6d388069c2d7: Waiting 2024-08-20T22:11:54.7369782Z 51d2d3747081: Waiting 2024-08-20T22:11:54.7370110Z ddc70883fbfe: Waiting 2024-08-20T22:11:54.7370420Z 0183570ab6f1: Waiting 2024-08-20T22:11:54.7370737Z 0972844ea4ec: Waiting 2024-08-20T22:11:54.7371050Z 3c5150459092: Waiting 2024-08-20T22:11:54.7371363Z 20952c6c7b15: Waiting 2024-08-20T22:11:54.7371676Z ce74fbfc871d: Waiting 2024-08-20T22:11:54.7371991Z 98f539161af0: Waiting 2024-08-20T22:11:54.7372309Z 84d7c1aff016: Waiting 2024-08-20T22:11:54.7372627Z 657a0885dc72: Waiting 2024-08-20T22:11:54.7372946Z 697db697809d: Waiting 2024-08-20T22:11:54.7373467Z 01240c436d2f: Waiting 2024-08-20T22:11:54.7373789Z 164218df90f4: Waiting 2024-08-20T22:11:54.7374103Z 526c21e3e021: Waiting 2024-08-20T22:11:54.7374418Z 6b78a8535b2c: Waiting 2024-08-20T22:11:54.7374737Z 13b68a95b298: Waiting 2024-08-20T22:11:54.7375059Z f1d74e82c839: Waiting 2024-08-20T22:11:54.7375390Z 8269d46e5fda: Pulling fs layer 2024-08-20T22:11:54.7375760Z d9ebe2872e74: Waiting 2024-08-20T22:11:54.7376101Z 877a9eaa525f: Pulling fs layer 2024-08-20T22:11:54.7376462Z 8a1507f1f600: Waiting 2024-08-20T22:11:54.7376806Z b4a44fd6dcb4: Pulling fs layer 2024-08-20T22:11:54.7377174Z 1a5959b2d611: Waiting 2024-08-20T22:11:54.7377493Z 143fcaf620b2: Waiting 2024-08-20T22:11:54.7377825Z 4f4fb700ef54: Pulling fs layer 2024-08-20T22:11:54.7378196Z ec22bfa269b6: Waiting 2024-08-20T22:11:54.7378516Z 58e41754815c: Waiting 2024-08-20T22:11:54.7378844Z 3923dee95bc8: Pulling fs layer 2024-08-20T22:11:54.7379304Z 62e2be9188ac: Waiting 2024-08-20T22:11:54.7379655Z 3385be080d47: Pulling fs layer 2024-08-20T22:11:54.7380022Z 8738005d6660: Waiting 2024-08-20T22:11:54.7380353Z fb3b968bdf0d: Pulling fs layer 2024-08-20T22:11:54.7380740Z dca079b1da9d: Pulling fs layer 2024-08-20T22:11:54.7381124Z 5809b0e30289: Pulling fs layer 2024-08-20T22:11:54.7381500Z 3226b5c3a909: Pulling fs layer 2024-08-20T22:11:54.7381873Z 16d2c60c3be5: Pulling fs layer 2024-08-20T22:11:54.7382240Z d31b6d35a64a: Waiting 2024-08-20T22:11:54.7382578Z a3645dbfb351: Pulling fs layer 2024-08-20T22:11:54.7382951Z 450f041fd9b2: Pulling fs layer 2024-08-20T22:11:54.7383336Z fa90062cf4e6: Pulling fs layer 2024-08-20T22:11:54.7383699Z 7397e0c01921: Waiting 2024-08-20T22:11:54.7384019Z 6e06b9b118be: Waiting 2024-08-20T22:11:54.7384335Z 4f4fb700ef54: Waiting 2024-08-20T22:11:54.7384671Z c3392f1762d8: Pulling fs layer 2024-08-20T22:11:54.7385052Z e52dd7f40be6: Pulling fs layer 2024-08-20T22:11:54.7385436Z 87b663a94d66: Pulling fs layer 2024-08-20T22:11:54.7385812Z 1df0f571a995: Pulling fs layer 2024-08-20T22:11:54.7386200Z 9fb55636587d: Pulling fs layer 2024-08-20T22:11:54.7386563Z 3923dee95bc8: Waiting 2024-08-20T22:11:54.7386880Z b4a44fd6dcb4: Waiting 2024-08-20T22:11:54.7387222Z 6f5f644dad08: Pulling fs layer 2024-08-20T22:11:54.7387591Z 3226b5c3a909: Waiting 2024-08-20T22:11:54.7387913Z fa90062cf4e6: Waiting 2024-08-20T22:11:54.7388231Z e52dd7f40be6: Waiting 2024-08-20T22:11:54.7388558Z 16d2c60c3be5: Waiting 2024-08-20T22:11:54.7388886Z f16e739cc132: Waiting 2024-08-20T22:11:54.7389209Z a3645dbfb351: Waiting 2024-08-20T22:11:54.7389525Z c3392f1762d8: Waiting 2024-08-20T22:11:54.7389933Z 450f041fd9b2: Waiting 2024-08-20T22:11:54.7390254Z 8269d46e5fda: Waiting 2024-08-20T22:11:54.7390567Z 87b663a94d66: Waiting 2024-08-20T22:11:54.7390883Z 9fb55636587d: Waiting 2024-08-20T22:11:54.7391200Z 5809b0e30289: Waiting 2024-08-20T22:11:54.7391519Z 6f5f644dad08: Waiting 2024-08-20T22:11:54.7391832Z 1df0f571a995: Waiting 2024-08-20T22:11:54.7392151Z 5010474278e0: Waiting 2024-08-20T22:11:54.7392473Z 1ba96cf48ba3: Waiting 2024-08-20T22:11:54.7392795Z fb3b968bdf0d: Waiting 2024-08-20T22:11:54.7393118Z 877a9eaa525f: Waiting 2024-08-20T22:11:54.7393444Z efb435cd2eaa: Waiting 2024-08-20T22:11:54.8581663Z cfb3d849840e: Verifying Checksum 2024-08-20T22:11:54.8582130Z cfb3d849840e: Download complete 2024-08-20T22:11:54.9454605Z ea310eb267ca: Verifying Checksum 2024-08-20T22:11:54.9455569Z ea310eb267ca: Download complete 2024-08-20T22:11:55.0109400Z 3af11d09e9cd: Verifying Checksum 2024-08-20T22:11:55.0110015Z 3af11d09e9cd: Download complete 2024-08-20T22:11:55.0850548Z 63e9bbe32327: Verifying Checksum 2024-08-20T22:11:55.0851074Z 63e9bbe32327: Download complete 2024-08-20T22:11:55.1645867Z 533b4aebf169: Verifying Checksum 2024-08-20T22:11:55.1646635Z 533b4aebf169: Download complete 2024-08-20T22:11:55.2414384Z 9dd75d06a091: Download complete 2024-08-20T22:11:55.3152469Z 30bfca4dd349: Verifying Checksum 2024-08-20T22:11:55.3152961Z 30bfca4dd349: Download complete 2024-08-20T22:11:55.3626268Z 968831e596a6: Verifying Checksum 2024-08-20T22:11:55.3627110Z 968831e596a6: Download complete 2024-08-20T22:11:55.4482323Z 9ee6bdb31195: Verifying Checksum 2024-08-20T22:11:55.4482850Z 9ee6bdb31195: Download complete 2024-08-20T22:11:55.5382683Z 722a6699b4eb: Download complete 2024-08-20T22:11:56.2549869Z 63e9bbe32327: Pull complete 2024-08-20T22:11:56.5769201Z cfb3d849840e: Pull complete 2024-08-20T22:11:57.5040443Z 968831e596a6: Pull complete 2024-08-20T22:11:57.5218331Z ea310eb267ca: Pull complete 2024-08-20T22:11:57.5386471Z 3af11d09e9cd: Pull complete 2024-08-20T22:12:01.3740360Z 164218df90f4: Verifying Checksum 2024-08-20T22:12:01.3740891Z 164218df90f4: Download complete 2024-08-20T22:12:01.5030261Z f1d74e82c839: Verifying Checksum 2024-08-20T22:12:01.5030714Z f1d74e82c839: Download complete 2024-08-20T22:12:01.6058079Z 13b68a95b298: Download complete 2024-08-20T22:12:01.6970347Z c95fa6ffe990: Verifying Checksum 2024-08-20T22:12:01.6970819Z c95fa6ffe990: Download complete 2024-08-20T22:12:04.8992490Z 831d1d1ad5de: Verifying Checksum 2024-08-20T22:12:04.8992994Z 831d1d1ad5de: Download complete 2024-08-20T22:12:05.0202969Z 3116a975d40f: Verifying Checksum 2024-08-20T22:12:05.1240280Z aed1c44ec0f6: Verifying Checksum 2024-08-20T22:12:05.1240889Z aed1c44ec0f6: Download complete 2024-08-20T22:12:05.2277676Z 657a0885dc72: Verifying Checksum 2024-08-20T22:12:05.2278217Z 657a0885dc72: Download complete 2024-08-20T22:12:15.5219150Z ebfec18059b9: Verifying Checksum 2024-08-20T22:12:15.5219730Z ebfec18059b9: Download complete 2024-08-20T22:12:15.6006723Z 6d388069c2d7: Verifying Checksum 2024-08-20T22:12:15.6007243Z 6d388069c2d7: Download complete 2024-08-20T22:12:15.7224676Z 51d2d3747081: Download complete 2024-08-20T22:12:15.8091954Z 84d7c1aff016: Download complete 2024-08-20T22:12:15.9179401Z ce74fbfc871d: Verifying Checksum 2024-08-20T22:12:15.9179966Z ce74fbfc871d: Download complete 2024-08-20T22:12:16.0097361Z 0972844ea4ec: Download complete 2024-08-20T22:12:19.7125478Z 20952c6c7b15: Download complete 2024-08-20T22:12:19.7907287Z 98f539161af0: Verifying Checksum 2024-08-20T22:12:19.7908031Z 98f539161af0: Download complete 2024-08-20T22:12:19.8612447Z 0183570ab6f1: Download complete 2024-08-20T22:12:19.9543526Z 58e41754815c: Download complete 2024-08-20T22:12:20.0307198Z f16e739cc132: Verifying Checksum 2024-08-20T22:12:20.1190327Z f16e739cc132: Download complete 2024-08-20T22:12:20.1190805Z ddc70883fbfe: Verifying Checksum 2024-08-20T22:12:20.1191224Z ddc70883fbfe: Download complete 2024-08-20T22:12:28.2887184Z ebfec18059b9: Pull complete 2024-08-20T22:12:28.5092095Z 533b4aebf169: Pull complete 2024-08-20T22:12:28.7318853Z 9dd75d06a091: Pull complete 2024-08-20T22:12:28.9502601Z 30bfca4dd349: Pull complete 2024-08-20T22:12:35.3232476Z 1b57ce94cad9: Verifying Checksum 2024-08-20T22:12:35.3233092Z 1b57ce94cad9: Download complete 2024-08-20T22:12:35.4127505Z 697db697809d: Verifying Checksum 2024-08-20T22:12:35.4128067Z 697db697809d: Download complete 2024-08-20T22:12:35.5137528Z 01240c436d2f: Download complete 2024-08-20T22:12:36.4490235Z 526c21e3e021: Verifying Checksum 2024-08-20T22:12:36.4490851Z 526c21e3e021: Download complete 2024-08-20T22:12:36.5544781Z 8a1507f1f600: Download complete 2024-08-20T22:12:36.6309507Z efb435cd2eaa: Verifying Checksum 2024-08-20T22:12:36.6310181Z efb435cd2eaa: Download complete 2024-08-20T22:12:37.2391669Z 3c5150459092: Verifying Checksum 2024-08-20T22:12:37.2392303Z 3c5150459092: Download complete 2024-08-20T22:12:37.3355216Z 1a5959b2d611: Verifying Checksum 2024-08-20T22:12:37.3355849Z 1a5959b2d611: Download complete 2024-08-20T22:12:37.4172689Z ec22bfa269b6: Download complete 2024-08-20T22:12:37.5166372Z 6b78a8535b2c: Verifying Checksum 2024-08-20T22:12:37.5166923Z 6b78a8535b2c: Download complete 2024-08-20T22:12:37.5241023Z 143fcaf620b2: Verifying Checksum 2024-08-20T22:12:37.5241540Z 143fcaf620b2: Download complete 2024-08-20T22:12:37.6296359Z 2dfbafd7cb9f: Verifying Checksum 2024-08-20T22:12:37.6296867Z 2dfbafd7cb9f: Download complete 2024-08-20T22:12:37.7186829Z 5010474278e0: Download complete 2024-08-20T22:12:39.1374224Z 62e2be9188ac: Verifying Checksum 2024-08-20T22:12:39.1374996Z 62e2be9188ac: Download complete 2024-08-20T22:12:39.2104648Z 8738005d6660: Download complete 2024-08-20T22:12:39.2911267Z 7397e0c01921: Verifying Checksum 2024-08-20T22:12:39.2911893Z 7397e0c01921: Download complete 2024-08-20T22:12:39.3914966Z d31b6d35a64a: Download complete 2024-08-20T22:12:39.4746885Z 6e06b9b118be: Verifying Checksum 2024-08-20T22:12:39.4747339Z 6e06b9b118be: Download complete 2024-08-20T22:13:06.6311034Z 1b57ce94cad9: Pull complete 2024-08-20T22:13:06.8604001Z 9ee6bdb31195: Pull complete 2024-08-20T22:13:07.0884470Z 722a6699b4eb: Pull complete 2024-08-20T22:13:15.5375868Z 164218df90f4: Pull complete 2024-08-20T22:13:15.7548681Z f1d74e82c839: Pull complete 2024-08-20T22:13:15.9639001Z 13b68a95b298: Pull complete 2024-08-20T22:13:16.1737256Z c95fa6ffe990: Pull complete 2024-08-20T22:13:18.8226132Z 831d1d1ad5de: Pull complete 2024-08-20T22:13:19.0521526Z 3116a975d40f: Pull complete 2024-08-20T22:13:19.2681199Z aed1c44ec0f6: Pull complete 2024-08-20T22:13:19.3582972Z 657a0885dc72: Pull complete 2024-08-20T22:13:22.2872240Z e834cd849603: Verifying Checksum 2024-08-20T22:13:22.2872713Z e834cd849603: Download complete 2024-08-20T22:13:22.3841665Z 8269d46e5fda: Verifying Checksum 2024-08-20T22:13:22.3842110Z 8269d46e5fda: Download complete 2024-08-20T22:13:22.4589636Z 877a9eaa525f: Verifying Checksum 2024-08-20T22:13:22.4590088Z 877a9eaa525f: Download complete 2024-08-20T22:13:22.5378351Z b4a44fd6dcb4: Download complete 2024-08-20T22:13:22.5469595Z 4f4fb700ef54: Verifying Checksum 2024-08-20T22:13:22.5470046Z 4f4fb700ef54: Download complete 2024-08-20T22:13:22.6578712Z 3923dee95bc8: Verifying Checksum 2024-08-20T22:13:22.6579234Z 3923dee95bc8: Download complete 2024-08-20T22:13:22.7183071Z 3385be080d47: Verifying Checksum 2024-08-20T22:13:22.7183671Z 3385be080d47: Download complete 2024-08-20T22:13:23.0585026Z fb3b968bdf0d: Verifying Checksum 2024-08-20T22:13:23.0585497Z fb3b968bdf0d: Download complete 2024-08-20T22:13:23.1284253Z dca079b1da9d: Verifying Checksum 2024-08-20T22:13:23.1284795Z dca079b1da9d: Download complete 2024-08-20T22:13:23.2087505Z 5809b0e30289: Verifying Checksum 2024-08-20T22:13:23.2088207Z 5809b0e30289: Download complete 2024-08-20T22:13:23.2866175Z 3226b5c3a909: Download complete 2024-08-20T22:13:23.3733550Z 16d2c60c3be5: Verifying Checksum 2024-08-20T22:13:23.3733988Z 16d2c60c3be5: Download complete 2024-08-20T22:13:23.6459262Z a3645dbfb351: Verifying Checksum 2024-08-20T22:13:23.6459797Z a3645dbfb351: Download complete 2024-08-20T22:13:23.7559874Z 450f041fd9b2: Download complete 2024-08-20T22:13:25.0859459Z fa90062cf4e6: Verifying Checksum 2024-08-20T22:13:25.0860084Z fa90062cf4e6: Download complete 2024-08-20T22:13:25.2018207Z c3392f1762d8: Download complete 2024-08-20T22:14:04.9966440Z e52dd7f40be6: Verifying Checksum 2024-08-20T22:14:04.9967093Z e52dd7f40be6: Download complete 2024-08-20T22:14:05.0822784Z 87b663a94d66: Verifying Checksum 2024-08-20T22:14:05.0823254Z 87b663a94d66: Download complete 2024-08-20T22:14:05.1559985Z 1df0f571a995: Verifying Checksum 2024-08-20T22:14:05.1560622Z 1df0f571a995: Download complete 2024-08-20T22:14:06.5950684Z 9fb55636587d: Verifying Checksum 2024-08-20T22:14:06.5951249Z 9fb55636587d: Download complete 2024-08-20T22:14:06.6873600Z 6f5f644dad08: Verifying Checksum 2024-08-20T22:14:06.6874192Z 6f5f644dad08: Download complete 2024-08-20T22:14:42.9893130Z 1ba96cf48ba3: Verifying Checksum 2024-08-20T22:14:42.9893573Z 1ba96cf48ba3: Download complete 2024-08-20T22:14:50.2075826Z d9ebe2872e74: Verifying Checksum 2024-08-20T22:14:50.2076366Z d9ebe2872e74: Download complete 2024-08-20T22:15:02.4182336Z e834cd849603: Pull complete 2024-08-20T22:15:02.5616100Z 6d388069c2d7: Pull complete 2024-08-20T22:15:02.7233313Z 51d2d3747081: Pull complete 2024-08-20T22:15:02.9063137Z 84d7c1aff016: Pull complete 2024-08-20T22:15:03.1255837Z ce74fbfc871d: Pull complete 2024-08-20T22:15:03.3175002Z 0972844ea4ec: Pull complete 2024-08-20T22:15:06.9370916Z 20952c6c7b15: Pull complete 2024-08-20T22:15:07.1557271Z 98f539161af0: Pull complete 2024-08-20T22:15:07.3896514Z 0183570ab6f1: Pull complete 2024-08-20T22:15:07.6184210Z 58e41754815c: Pull complete 2024-08-20T22:15:07.7570980Z f16e739cc132: Pull complete 2024-08-20T22:15:07.8412316Z ddc70883fbfe: Pull complete 2024-08-20T22:15:18.1066113Z 3c5150459092: Pull complete 2024-08-20T22:15:18.3181689Z 697db697809d: Pull complete 2024-08-20T22:15:18.5113245Z 01240c436d2f: Pull complete 2024-08-20T22:15:19.4854477Z 526c21e3e021: Pull complete 2024-08-20T22:15:19.6800547Z 8a1507f1f600: Pull complete 2024-08-20T22:15:19.8764076Z efb435cd2eaa: Pull complete 2024-08-20T22:15:20.3510175Z 6b78a8535b2c: Pull complete 2024-08-20T22:15:20.5342579Z 1a5959b2d611: Pull complete 2024-08-20T22:15:20.8808968Z ec22bfa269b6: Pull complete 2024-08-20T22:15:21.0661338Z 143fcaf620b2: Pull complete 2024-08-20T22:15:59.5580710Z d9ebe2872e74: Pull complete 2024-08-20T22:15:59.7834483Z 2dfbafd7cb9f: Pull complete 2024-08-20T22:16:00.0210246Z 5010474278e0: Pull complete 2024-08-20T22:16:02.5674476Z 62e2be9188ac: Pull complete 2024-08-20T22:16:02.5855187Z 8738005d6660: Pull complete 2024-08-20T22:16:02.6052719Z 7397e0c01921: Pull complete 2024-08-20T22:16:02.6400144Z d31b6d35a64a: Pull complete 2024-08-20T22:16:02.6584174Z 6e06b9b118be: Pull complete 2024-08-20T22:17:23.6213260Z 1ba96cf48ba3: Pull complete 2024-08-20T22:17:23.8366672Z 8269d46e5fda: Pull complete 2024-08-20T22:17:24.0491920Z 877a9eaa525f: Pull complete 2024-08-20T22:17:24.4608809Z b4a44fd6dcb4: Pull complete 2024-08-20T22:17:24.6759707Z 4f4fb700ef54: Pull complete 2024-08-20T22:17:24.9013776Z 3923dee95bc8: Pull complete 2024-08-20T22:17:25.1279048Z 3385be080d47: Pull complete 2024-08-20T22:17:25.5047930Z fb3b968bdf0d: Pull complete 2024-08-20T22:17:25.7412331Z dca079b1da9d: Pull complete 2024-08-20T22:17:25.9610921Z 5809b0e30289: Pull complete 2024-08-20T22:17:26.1941436Z 3226b5c3a909: Pull complete 2024-08-20T22:17:26.4116355Z 16d2c60c3be5: Pull complete 2024-08-20T22:17:27.8267434Z a3645dbfb351: Pull complete 2024-08-20T22:17:28.0480196Z 450f041fd9b2: Pull complete 2024-08-20T22:17:30.2633140Z fa90062cf4e6: Pull complete 2024-08-20T22:17:30.4797000Z c3392f1762d8: Pull complete 2024-08-20T22:17:43.8368731Z e52dd7f40be6: Pull complete 2024-08-20T22:17:44.0452358Z 87b663a94d66: Pull complete 2024-08-20T22:17:44.2705337Z 1df0f571a995: Pull complete 2024-08-20T22:17:45.1654241Z 9fb55636587d: Pull complete 2024-08-20T22:17:45.3813493Z 6f5f644dad08: Pull complete 2024-08-20T22:17:46.2807076Z Digest: sha256:4753b1ce6a27c0af5c669dd74e9e16618d0a1a015eb474ef14f6fd51409fc5f7 2024-08-20T22:17:46.3244815Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:17:46.3526002Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:17:46.3636530Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:17:46.3637475Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:17:46.3648298Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:17:46.3648809Z env: 2024-08-20T22:17:46.3649091Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:17:46.3649444Z ##[endgroup] 2024-08-20T22:17:46.3857474Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-20T22:17:46.3858038Z with: 2024-08-20T22:17:46.3858340Z driver-version: 550.54.15 2024-08-20T22:17:46.3858786Z env: 2024-08-20T22:17:46.3859072Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:17:46.3859422Z ##[endgroup] 2024-08-20T22:17:46.3938507Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:17:46.3939190Z with: 2024-08-20T22:17:46.3939464Z timeout_minutes: 10 2024-08-20T22:17:46.3939780Z max_attempts: 3 2024-08-20T22:17:46.3972835Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-20T22:17:46.4005969Z retry_wait_seconds: 10 2024-08-20T22:17:46.4006335Z polling_interval_seconds: 1 2024-08-20T22:17:46.4006703Z warning_on_retry: true 2024-08-20T22:17:46.4007058Z continue_on_error: false 2024-08-20T22:17:46.4007394Z env: 2024-08-20T22:17:46.4007662Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:17:46.4008008Z DRIVER_VERSION: 550.54.15 2024-08-20T22:17:46.4008355Z ##[endgroup] 2024-08-20T22:17:46.4813574Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-20T22:17:46.4814418Z + pre_install_nvidia_driver_amzn2 2024-08-20T22:17:46.4816612Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-20T22:17:46.9388033Z No match for argument: nvidia-driver-latest-dkms 2024-08-20T22:17:46.9388756Z No packages marked for removal. 2024-08-20T22:17:46.9452116Z Dependencies resolved. 2024-08-20T22:17:46.9462192Z Nothing to do. 2024-08-20T22:17:46.9462617Z Complete! 2024-08-20T22:17:47.0348636Z + install_nvidia_driver_common 2024-08-20T22:17:47.0351352Z + echo 'Before installing NVIDIA driver' 2024-08-20T22:17:47.0351960Z Before installing NVIDIA driver 2024-08-20T22:17:47.0352588Z + lspci 2024-08-20T22:17:47.1302780Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:17:47.1304054Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:17:47.1305335Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:17:47.1306384Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:17:47.1307492Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:17:47.1308451Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:17:47.1309193Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:17:47.1310020Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:17:47.1310618Z + lsmod 2024-08-20T22:17:47.1347167Z Module Size Used by 2024-08-20T22:17:47.1347792Z xt_conntrack 16384 1 2024-08-20T22:17:47.1348313Z nft_chain_nat 16384 3 2024-08-20T22:17:47.1348710Z xt_MASQUERADE 20480 1 2024-08-20T22:17:47.1349306Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-20T22:17:47.1349969Z nf_conntrack_netlink 57344 0 2024-08-20T22:17:47.1350672Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:17:47.1351320Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:17:47.1352489Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:17:47.1353065Z xfrm_user 57344 1 2024-08-20T22:17:47.1353540Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:17:47.1354120Z xt_addrtype 16384 2 2024-08-20T22:17:47.1354518Z nft_compat 20480 4 2024-08-20T22:17:47.1355136Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:17:47.1355968Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:17:47.1356862Z br_netfilter 36864 0 2024-08-20T22:17:47.1357457Z bridge 307200 1 br_netfilter 2024-08-20T22:17:47.1358052Z stp 16384 1 bridge 2024-08-20T22:17:47.1358605Z llc 16384 2 bridge,stp 2024-08-20T22:17:47.1359159Z overlay 167936 0 2024-08-20T22:17:47.1359644Z tls 114688 0 2024-08-20T22:17:47.1360134Z nls_ascii 16384 1 2024-08-20T22:17:47.1360638Z nls_cp437 20480 1 2024-08-20T22:17:47.1361004Z vfat 24576 1 2024-08-20T22:17:47.1361375Z fat 86016 1 vfat 2024-08-20T22:17:47.1361782Z sunrpc 692224 1 2024-08-20T22:17:47.1362165Z ghash_clmulni_intel 16384 0 2024-08-20T22:17:47.1362541Z ena 167936 0 2024-08-20T22:17:47.1362908Z ptp 36864 1 ena 2024-08-20T22:17:47.1363306Z pps_core 24576 1 ptp 2024-08-20T22:17:47.1363708Z aesni_intel 393216 0 2024-08-20T22:17:47.1364068Z i8042 45056 0 2024-08-20T22:17:47.1364440Z serio 28672 3 i8042 2024-08-20T22:17:47.1364840Z button 24576 0 2024-08-20T22:17:47.1365235Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:17:47.1365755Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:17:47.1366305Z sch_fq_codel 20480 17 2024-08-20T22:17:47.1366692Z dm_mod 188416 0 2024-08-20T22:17:47.1367191Z dax 45056 1 dm_mod 2024-08-20T22:17:47.1367724Z loop 36864 0 2024-08-20T22:17:47.1368201Z fuse 163840 1 2024-08-20T22:17:47.1368687Z configfs 57344 1 2024-08-20T22:17:47.1369140Z dmi_sysfs 20480 0 2024-08-20T22:17:47.1369520Z crc32_pclmul 16384 0 2024-08-20T22:17:47.1369889Z crc32c_intel 24576 0 2024-08-20T22:17:47.1370261Z efivarfs 24576 1 2024-08-20T22:17:47.1370621Z + modinfo nvidia 2024-08-20T22:17:47.1371349Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:17:47.1372249Z alias: char-major-195-* 2024-08-20T22:17:47.1372641Z version: 550.54.15 2024-08-20T22:17:47.1372989Z supported: external 2024-08-20T22:17:47.1373343Z license: NVIDIA 2024-08-20T22:17:47.1373730Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:17:47.1374223Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:17:47.1374687Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:17:47.1375165Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:17:47.1375675Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:17:47.1376223Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:17:47.1376719Z depends: i2c-core,drm 2024-08-20T22:17:47.1377090Z retpoline: Y 2024-08-20T22:17:47.1377405Z name: nvidia 2024-08-20T22:17:47.1378128Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:17:47.1378974Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:17:47.1379647Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:17:47.1380260Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:17:47.1380718Z parm: NVreg_RmLogonRC:int 2024-08-20T22:17:47.1381157Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:17:47.1381627Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:17:47.1382077Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:17:47.1382522Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:17:47.1383064Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:17:47.1383642Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:17:47.1384133Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:17:47.1384570Z parm: NVreg_EnableMSI:int 2024-08-20T22:17:47.1385004Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:17:47.1385472Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:17:47.1386042Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:17:47.1386669Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:17:47.1387247Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:17:47.1387869Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:17:47.1388480Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:17:47.1389104Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:17:47.1389714Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:17:47.1390216Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:17:47.1390764Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:17:47.1391319Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:17:47.1391827Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:17:47.1392311Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:17:47.1392809Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:17:47.1393295Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:17:47.1393755Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:17:47.1394271Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:17:47.1394804Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:17:47.1395297Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:17:47.1395798Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:17:47.1396308Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:17:47.1396803Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:17:47.1397315Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:17:47.1397804Z parm: NVreg_RmMsg:charp 2024-08-20T22:17:47.1398228Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:17:47.1398703Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:17:47.1399191Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:17:47.1399657Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:17:47.1400237Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:17:47.1400717Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:17:47.1401185Z parm: rm_firmware_active:charp 2024-08-20T22:17:47.1401610Z + HAS_NVIDIA_DRIVER=0 2024-08-20T22:17:47.1402015Z ++ command -v nvidia-smi 2024-08-20T22:17:47.1402421Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-20T22:17:47.1402799Z + set +e 2024-08-20T22:17:47.1403322Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-20T22:17:49.4063844Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-20T22:17:49.4064437Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:17:49.4065056Z + '[' 0 -ne 0 ']' 2024-08-20T22:17:49.4065438Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-20T22:17:49.4065831Z + HAS_NVIDIA_DRIVER=1 2024-08-20T22:17:49.4067007Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-20T22:17:49.4067736Z + set -e 2024-08-20T22:17:49.4068064Z + '[' 1 -eq 0 ']' 2024-08-20T22:17:49.4068663Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-20T22:17:49.4069358Z + post_install_nvidia_driver_common 2024-08-20T22:17:49.4070314Z + sudo modprobe nvidia 2024-08-20T22:17:49.5142034Z + echo 'After installing NVIDIA driver' 2024-08-20T22:17:49.5142576Z + lspci 2024-08-20T22:17:49.5142888Z After installing NVIDIA driver 2024-08-20T22:17:49.5248245Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:17:49.5249206Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:17:49.5250353Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:17:49.5251207Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:17:49.5252061Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:17:49.5252864Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:17:49.5253621Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:17:49.5254447Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:17:49.5255053Z + lsmod 2024-08-20T22:17:49.5280920Z Module Size Used by 2024-08-20T22:17:49.5281989Z nvidia_uvm 4706304 0 2024-08-20T22:17:49.5283082Z nvidia 54071296 1 nvidia_uvm 2024-08-20T22:17:49.5284208Z drm 602112 1 nvidia 2024-08-20T22:17:49.5285148Z drm_panel_orientation_quirks 28672 1 drm 2024-08-20T22:17:49.5286043Z backlight 24576 1 drm 2024-08-20T22:17:49.5286782Z i2c_core 106496 2 nvidia,drm 2024-08-20T22:17:49.5287212Z xt_conntrack 16384 1 2024-08-20T22:17:49.5287588Z nft_chain_nat 16384 3 2024-08-20T22:17:49.5287972Z xt_MASQUERADE 20480 1 2024-08-20T22:17:49.5288436Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-20T22:17:49.5288930Z nf_conntrack_netlink 57344 0 2024-08-20T22:17:49.5289513Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:17:49.5290156Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:17:49.5290623Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:17:49.5291063Z xfrm_user 57344 1 2024-08-20T22:17:49.5291455Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:17:49.5291882Z xt_addrtype 16384 2 2024-08-20T22:17:49.5292262Z nft_compat 20480 4 2024-08-20T22:17:49.5292715Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:17:49.5293337Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:17:49.5293888Z br_netfilter 36864 0 2024-08-20T22:17:49.5294321Z bridge 307200 1 br_netfilter 2024-08-20T22:17:49.5294773Z stp 16384 1 bridge 2024-08-20T22:17:49.5295189Z llc 16384 2 bridge,stp 2024-08-20T22:17:49.5295615Z overlay 167936 0 2024-08-20T22:17:49.5296324Z tls 114688 0 2024-08-20T22:17:49.5296688Z nls_ascii 16384 1 2024-08-20T22:17:49.5297063Z nls_cp437 20480 1 2024-08-20T22:17:49.5297434Z vfat 24576 1 2024-08-20T22:17:49.5297806Z fat 86016 1 vfat 2024-08-20T22:17:49.5298195Z sunrpc 692224 1 2024-08-20T22:17:49.5298580Z ghash_clmulni_intel 16384 0 2024-08-20T22:17:49.5299031Z ena 167936 0 2024-08-20T22:17:49.5299402Z ptp 36864 1 ena 2024-08-20T22:17:49.5299798Z pps_core 24576 1 ptp 2024-08-20T22:17:49.5300195Z aesni_intel 393216 0 2024-08-20T22:17:49.5300644Z i8042 45056 0 2024-08-20T22:17:49.5301140Z serio 28672 3 i8042 2024-08-20T22:17:49.5301896Z button 24576 0 2024-08-20T22:17:49.5302443Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:17:49.5303136Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:17:49.5303806Z sch_fq_codel 20480 17 2024-08-20T22:17:49.5304309Z dm_mod 188416 0 2024-08-20T22:17:49.5304813Z dax 45056 1 dm_mod 2024-08-20T22:17:49.5305342Z loop 36864 0 2024-08-20T22:17:49.5305815Z fuse 163840 1 2024-08-20T22:17:49.5306277Z configfs 57344 1 2024-08-20T22:17:49.5306656Z dmi_sysfs 20480 0 2024-08-20T22:17:49.5307119Z crc32_pclmul 16384 0 2024-08-20T22:17:49.5307621Z crc32c_intel 24576 0 2024-08-20T22:17:49.5308117Z efivarfs 24576 1 2024-08-20T22:17:49.5308495Z + modinfo nvidia 2024-08-20T22:17:49.5309205Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:17:49.5309927Z alias: char-major-195-* 2024-08-20T22:17:49.5310312Z version: 550.54.15 2024-08-20T22:17:49.5310662Z supported: external 2024-08-20T22:17:49.5311002Z license: NVIDIA 2024-08-20T22:17:49.5311391Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:17:49.5311881Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:17:49.5312344Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:17:49.5312806Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:17:49.5313309Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:17:49.5313800Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:17:49.5314298Z depends: i2c-core,drm 2024-08-20T22:17:49.5314658Z retpoline: Y 2024-08-20T22:17:49.5314969Z name: nvidia 2024-08-20T22:17:49.5315591Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:17:49.5316294Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:17:49.5316945Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:17:49.5317549Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:17:49.5318008Z parm: NVreg_RmLogonRC:int 2024-08-20T22:17:49.5318450Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:17:49.5318906Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:17:49.5319347Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:17:49.5319792Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:17:49.5320328Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:17:49.5320892Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:17:49.5321383Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:17:49.5321819Z parm: NVreg_EnableMSI:int 2024-08-20T22:17:49.5322238Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:17:49.5322706Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:17:49.5323244Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:17:49.5323842Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:17:49.5324404Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:17:49.5325158Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:17:49.5325765Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:17:49.5326389Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:17:49.5326981Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:17:49.5327528Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:17:49.5328067Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:17:49.5328620Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:17:49.5329118Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:17:49.5329591Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:17:49.5330084Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:17:49.5330560Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:17:49.5331099Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:17:49.5331611Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:17:49.5332150Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:17:49.5332632Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:17:49.5333119Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:17:49.5333624Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:17:49.5334115Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:17:49.5334612Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:17:49.5335098Z parm: NVreg_RmMsg:charp 2024-08-20T22:17:49.5335516Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:17:49.5335991Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:17:49.5336462Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:17:49.5336948Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:17:49.5337471Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:17:49.5337953Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:17:49.5338404Z parm: rm_firmware_active:charp 2024-08-20T22:17:49.5338911Z + set +e 2024-08-20T22:17:49.5339228Z + nvidia-smi 2024-08-20T22:17:51.1049950Z Tue Aug 20 22:17:51 2024 2024-08-20T22:17:51.1050986Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:17:51.1051934Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:17:51.1052745Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:17:51.1053557Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:17:51.1054466Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:17:51.1055180Z | | | MIG M. | 2024-08-20T22:17:51.1055769Z |=========================================+========================+======================| 2024-08-20T22:17:51.1151450Z | 0 NVIDIA A10G Off | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:17:51.1152431Z | 0% 29C P0 57W / 300W | 0MiB / 23028MiB | 5% Default | 2024-08-20T22:17:51.1153273Z | | | N/A | 2024-08-20T22:17:51.1154187Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:17:51.1154831Z 2024-08-20T22:17:51.1155484Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:17:51.1156110Z | Processes: | 2024-08-20T22:17:51.1156968Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:17:51.1157888Z | ID ID Usage | 2024-08-20T22:17:51.1159459Z |=========================================================================================| 2024-08-20T22:17:51.1160109Z | No running processes found | 2024-08-20T22:17:51.1160894Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:17:51.7284708Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-20T22:17:53.2965320Z NVIDIA A10G 2024-08-20T22:17:53.7424510Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:17:53.7425219Z + '[' 0 -eq 0 ']' 2024-08-20T22:17:53.7426283Z + echo 'INFO: Ignoring allowed status 0' 2024-08-20T22:17:53.7426868Z + set -e 2024-08-20T22:17:53.7427273Z INFO: Ignoring allowed status 0 2024-08-20T22:17:53.7434218Z == Installing nvidia container toolkit for amzn2023 == 2024-08-20T22:17:53.7438473Z + sudo yum install -y yum-utils 2024-08-20T22:17:54.2105139Z Last metadata expiration check: 0:09:35 ago on Tue Aug 20 22:08:19 2024. 2024-08-20T22:17:54.2323389Z Package dnf-utils-4.3.0-13.amzn2023.0.4.noarch is already installed. 2024-08-20T22:17:54.2622155Z Dependencies resolved. 2024-08-20T22:17:54.2745194Z Nothing to do. 2024-08-20T22:17:54.2745628Z Complete! 2024-08-20T22:17:54.3593455Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2024-08-20T22:17:54.3594779Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:17:54.3596104Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:17:54.6501580Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:17:54.7236134Z + sudo yum install -y nvidia-docker2 2024-08-20T22:17:55.2010061Z nvidia-container-toolkit 10 kB/s | 833 B 00:00 2024-08-20T22:17:55.2227190Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2024-08-20T22:17:55.2520717Z Dependencies resolved. 2024-08-20T22:17:55.2644235Z Nothing to do. 2024-08-20T22:17:55.2645077Z Complete! 2024-08-20T22:17:55.3408767Z + sudo systemctl restart docker 2024-08-20T22:18:51.4803189Z Tue Aug 20 22:18:51 2024 2024-08-20T22:18:51.4804055Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:18:51.4804901Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:18:51.4805711Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:18:51.4806516Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:18:51.4807458Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:18:51.4808186Z | | | MIG M. | 2024-08-20T22:18:51.4808755Z |=========================================+========================+======================| 2024-08-20T22:18:51.4923813Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:18:51.4925474Z | 0% 29C P0 61W / 300W | 0MiB / 23028MiB | 5% Default | 2024-08-20T22:18:51.4926749Z | | | N/A | 2024-08-20T22:18:51.4927971Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:18:51.4928565Z 2024-08-20T22:18:51.4929223Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:18:51.4929862Z | Processes: | 2024-08-20T22:18:51.4930568Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:18:51.4931601Z | ID ID Usage | 2024-08-20T22:18:51.4932162Z |=========================================================================================| 2024-08-20T22:18:51.4932826Z | No running processes found | 2024-08-20T22:18:51.4933589Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:18:52.5228984Z Command completed after 1 attempt(s). 2024-08-20T22:18:52.5312566Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:18:52.5313350Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:18:52.5314052Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-20T22:18:52.5314717Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:18:52.5330383Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:18:52.5330898Z env: 2024-08-20T22:18:52.5331181Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:18:52.5331643Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:18:52.5332128Z ##[endgroup] 2024-08-20T22:18:52.8084968Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:18:53.2217524Z Collecting psutil==5.9.1 2024-08-20T22:18:53.2601526Z Downloading psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB) 2024-08-20T22:18:53.3091012Z Collecting nvidia-ml-py==11.525.84 2024-08-20T22:18:53.3133298Z Downloading nvidia_ml_py-11.525.84-py3-none-any.whl (34 kB) 2024-08-20T22:18:53.3923592Z Installing collected packages: psutil, nvidia-ml-py 2024-08-20T22:18:53.5474183Z Successfully installed nvidia-ml-py-11.525.84 psutil-5.9.1 2024-08-20T22:18:53.7441555Z Prepare all required actions 2024-08-20T22:18:53.7442090Z Getting action download info 2024-08-20T22:18:53.8395151Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-20T22:18:54.0441838Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-20T22:18:54.1799615Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-20T22:18:54.1800109Z with: 2024-08-20T22:18:54.1800453Z name: linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-08-20T22:18:54.1800915Z s3-bucket: gha-artifacts 2024-08-20T22:18:54.1801263Z env: 2024-08-20T22:18:54.1801539Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:18:54.1801997Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:18:54.1802482Z ##[endgroup] 2024-08-20T22:18:54.1836945Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:18:54.1837386Z with: 2024-08-20T22:18:54.1837768Z name: linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-08-20T22:18:54.1838224Z s3-bucket: gha-artifacts 2024-08-20T22:18:54.1838580Z region: us-east-1 2024-08-20T22:18:54.1838897Z env: 2024-08-20T22:18:54.1839185Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:18:54.1839639Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:18:54.1840131Z ##[endgroup] 2024-08-20T22:18:54.6630979Z (node:52474) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:18:54.6631687Z 2024-08-20T22:18:54.6631953Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:18:54.6632693Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:18:54.6633583Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:18:54.7296753Z Found 1 objects with prefix pytorch/pytorch/10479310725/linux-focal-cuda12.4-py3.10-gcc9-sm86/ 2024-08-20T22:18:54.7298991Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:19:20.5886193Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:19:20.5892940Z Artifact download has finished successfully 2024-08-20T22:19:20.6244871Z ##[group]Run unzip -o artifacts.zip 2024-08-20T22:19:20.6245317Z unzip -o artifacts.zip 2024-08-20T22:19:20.6255458Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:20.6255963Z env: 2024-08-20T22:19:20.6256242Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:20.6256705Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:20.6257195Z ##[endgroup] 2024-08-20T22:19:20.6341903Z Archive: artifacts.zip 2024-08-20T22:19:20.6342776Z creating: dist/ 2024-08-20T22:19:22.7686742Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:19:22.7687633Z creating: build/custom_test_artifacts/ 2024-08-20T22:19:22.7688421Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-20T22:19:22.7689179Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-20T22:19:22.7690030Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:19:22.7696146Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:19:22.7697407Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-20T22:19:22.7698573Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:19:22.7699720Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:19:22.7700928Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:19:22.7702226Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:19:22.7704055Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:19:22.7705085Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:19:22.7706308Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:19:22.7707881Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:19:22.7709402Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:19:22.7710954Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:19:22.7712243Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:19:22.7713674Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:19:22.7715259Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:19:22.7716325Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:19:22.7717361Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:19:22.7757861Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:19:22.7799875Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:19:22.7801413Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:19:22.7847698Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:19:22.7849611Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:19:22.7851519Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:19:22.7853787Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:19:22.7855516Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:19:22.7857394Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:19:22.7859399Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:19:22.7861154Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:19:22.7862815Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:19:22.7864126Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:19:22.7865390Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:19:22.7866628Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:19:22.7867882Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:19:22.7869094Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:19:22.7870335Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:19:22.7937820Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:19:22.8010146Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:19:22.8011756Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:19:22.8013085Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:19:22.8014259Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:19:22.8015474Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-20T22:19:22.8016718Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-20T22:19:22.8018101Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-20T22:19:22.8019775Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-20T22:19:22.8021211Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-20T22:19:22.8022294Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-20T22:19:22.8023381Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-20T22:19:22.8024461Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-20T22:19:22.8025550Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-20T22:19:22.8026630Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-20T22:19:22.8027702Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-20T22:19:22.8042271Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-20T22:19:22.8185726Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-20T22:19:22.8187093Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-20T22:19:22.8188841Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-20T22:19:22.8190437Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-20T22:19:22.8192009Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-20T22:19:22.8193470Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-20T22:19:22.8194624Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-20T22:19:22.8195794Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-20T22:19:22.8196949Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-20T22:19:22.8198100Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-20T22:19:22.8199229Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-20T22:19:22.8214832Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-20T22:19:22.8300261Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-20T22:19:22.8303478Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:19:22.8306193Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:19:22.8308642Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-20T22:19:22.8309811Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-20T22:19:22.8310739Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-20T22:19:22.8311664Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-20T22:19:22.8312529Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-20T22:19:22.8313335Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-20T22:19:22.8314163Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-20T22:19:22.8429348Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-20T22:19:22.8492651Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-20T22:19:22.8493649Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-20T22:19:22.8494426Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-20T22:19:22.8495261Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:19:22.8501684Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:19:22.8502921Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-20T22:19:22.8504047Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:19:22.8505042Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:19:22.8506252Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:19:22.8507465Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:19:22.8508954Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:19:22.8509997Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:19:22.8511058Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:19:22.8512621Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:19:22.8514103Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:19:22.8515536Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:19:22.8516871Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:19:22.8518317Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:19:22.8519803Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:19:22.8521030Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:19:22.8522040Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:19:22.8562848Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:19:22.8603991Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:19:22.8605564Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:19:22.8652251Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:19:22.8654095Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:19:22.8656204Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:19:22.8658188Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:19:22.8660111Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:19:22.8661961Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:19:22.8663831Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:19:22.8665640Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:19:22.8667308Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:19:22.8668606Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:19:22.8669865Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:19:22.8671088Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:19:22.8672317Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:19:22.8673519Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:19:22.8674735Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:19:22.8740925Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:19:22.8815081Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:19:22.8816664Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:19:22.8818256Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:19:22.8819575Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:19:22.8820777Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-20T22:19:22.8822041Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-20T22:19:22.8823401Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-20T22:19:22.8824946Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-20T22:19:22.8826388Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-20T22:19:22.8827488Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-20T22:19:22.8828605Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-20T22:19:22.8829723Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-20T22:19:22.8830839Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-20T22:19:22.8831952Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-20T22:19:22.8833055Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-20T22:19:22.8848125Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-20T22:19:22.8912400Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-20T22:19:22.8913998Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:19:22.8915422Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:19:22.8916639Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-20T22:19:22.8917823Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-20T22:19:22.8918777Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-20T22:19:22.8919688Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-20T22:19:22.8920836Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-20T22:19:22.8921871Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-20T22:19:22.8922882Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-20T22:19:22.8972485Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-20T22:19:22.8973512Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-20T22:19:22.8974323Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-20T22:19:22.8975213Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:19:22.8981769Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:19:22.8983090Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-20T22:19:22.8984415Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:19:22.8985492Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:19:22.8986713Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:19:22.8988287Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:19:22.8989972Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:19:22.8991045Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:19:22.8992354Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:19:22.8993832Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:19:22.8995308Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:19:22.8996846Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:19:22.8998246Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:19:22.8999871Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:19:22.9001367Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:19:22.9002485Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:19:22.9003564Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:19:22.9043138Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:19:22.9084909Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:19:22.9086732Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:19:22.9134467Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:19:22.9138375Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:19:22.9140409Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:19:22.9142450Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:19:22.9144434Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:19:22.9146665Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:19:22.9148451Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:19:22.9149926Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:19:22.9151350Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:19:22.9152715Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:19:22.9154025Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:19:22.9155663Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:19:22.9157475Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:19:22.9159373Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:19:22.9161546Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:19:22.9222829Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:19:22.9295412Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:19:22.9297099Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:19:22.9298522Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:19:22.9299857Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:19:22.9301168Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-20T22:19:22.9302516Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-20T22:19:22.9304011Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-20T22:19:22.9305683Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-20T22:19:22.9307098Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-20T22:19:22.9308258Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-20T22:19:22.9309444Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-20T22:19:22.9311114Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-20T22:19:22.9312333Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-20T22:19:22.9313532Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-20T22:19:22.9314711Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-20T22:19:22.9315973Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-20T22:19:22.9436993Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-20T22:19:22.9439271Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-20T22:19:22.9442267Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-20T22:19:22.9445838Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-20T22:19:22.9448708Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-20T22:19:22.9449923Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-20T22:19:22.9451169Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-20T22:19:22.9452430Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-20T22:19:22.9453684Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-20T22:19:22.9455022Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-20T22:19:22.9456295Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-20T22:19:22.9467340Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-20T22:19:22.9524379Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-20T22:19:22.9527847Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:19:22.9529424Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:19:22.9530584Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-20T22:19:22.9531874Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-20T22:19:22.9533176Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-20T22:19:22.9534168Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-20T22:19:22.9535097Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-20T22:19:22.9535946Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-20T22:19:22.9536823Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-20T22:19:22.9637056Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-20T22:19:22.9682000Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-20T22:19:22.9683498Z creating: build/lib/ 2024-08-20T22:19:22.9770327Z inflating: build/lib/libprotobuf-lite.a 2024-08-20T22:19:23.0226962Z inflating: build/lib/libprotobuf.a 2024-08-20T22:19:23.0236757Z inflating: build/lib/libpthreadpool.a 2024-08-20T22:19:23.0245087Z inflating: build/lib/libcpuinfo.a 2024-08-20T22:19:23.0253753Z inflating: build/lib/libcpuinfo_internals.a 2024-08-20T22:19:23.0254423Z inflating: build/lib/libclog.a 2024-08-20T22:19:23.0272942Z inflating: build/lib/libnnpack.a 2024-08-20T22:19:23.0275611Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-20T22:19:23.0341147Z inflating: build/lib/libgtest.a 2024-08-20T22:19:23.0416486Z inflating: build/lib/libbenchmark.a 2024-08-20T22:19:23.0481045Z inflating: build/lib/libasmjit.a 2024-08-20T22:19:23.0488451Z inflating: build/lib/libittnotify.a 2024-08-20T22:19:23.0516167Z inflating: build/lib/libtensorpipe_uv.a 2024-08-20T22:19:23.0643950Z inflating: build/lib/libgloo.a 2024-08-20T22:19:23.0664929Z inflating: build/lib/libfmt.a 2024-08-20T22:19:23.0759957Z inflating: build/lib/libc10.so 2024-08-20T22:19:23.0761369Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-20T22:19:23.0763364Z inflating: build/lib/libtorch_global_deps.so 2024-08-20T22:19:23.1267012Z inflating: build/lib/libprotoc.a 2024-08-20T22:19:23.1285148Z inflating: build/lib/libgmock.a 2024-08-20T22:19:23.1286124Z inflating: build/lib/libgtest_main.a 2024-08-20T22:19:23.1286798Z inflating: build/lib/libbenchmark_main.a 2024-08-20T22:19:23.1306752Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-20T22:19:23.1695789Z inflating: build/lib/libgloo_cuda.a 2024-08-20T22:19:24.1886872Z inflating: build/lib/libdnnl.a 2024-08-20T22:19:24.2460423Z inflating: build/lib/libtensorpipe.a 2024-08-20T22:19:24.2518150Z inflating: build/lib/libc10_cuda.so 2024-08-20T22:19:24.2518909Z inflating: build/lib/libgmock_main.a 2024-08-20T22:19:24.3781054Z inflating: build/lib/libfbgemm.a 2024-08-20T22:19:24.4037334Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-20T22:19:24.4545203Z inflating: build/lib/libkineto.a 2024-08-20T22:19:24.4729825Z inflating: build/lib/libXNNPACK.a 2024-08-20T22:19:24.4772523Z inflating: build/lib/libonnx_proto.a 2024-08-20T22:19:24.5466496Z inflating: build/lib/libonnx.a 2024-08-20T22:19:26.9974346Z inflating: build/lib/libtorch_cpu.so 2024-08-20T22:19:26.9979156Z inflating: build/lib/libunbox_lib.a 2024-08-20T22:19:26.9983522Z inflating: build/lib/libshm.so 2024-08-20T22:19:29.0594012Z inflating: build/lib/libtorch_cuda.so 2024-08-20T22:19:29.0594902Z inflating: build/lib/libtorch.so 2024-08-20T22:19:29.0597931Z inflating: build/lib/libc10d_cuda_test.so 2024-08-20T22:19:29.9383689Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-20T22:19:30.1380729Z inflating: build/lib/libtorch_python.so 2024-08-20T22:19:30.1453440Z inflating: build/lib/libtorchbind_test.so 2024-08-20T22:19:30.1474473Z inflating: build/lib/libjitbackend_test.so 2024-08-20T22:19:30.1499928Z inflating: build/lib/libbackend_with_compiler.so 2024-08-20T22:19:30.1526089Z inflating: build/lib/libaoti_custom_ops.so 2024-08-20T22:19:30.1560934Z inflating: build/lib/libnnapi_backend.so 2024-08-20T22:19:30.1561565Z creating: build/bin/ 2024-08-20T22:19:30.1611836Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-20T22:19:30.1663754Z inflating: build/bin/c10_DeviceGuard_test 2024-08-20T22:19:30.1714857Z inflating: build/bin/c10_Device_test 2024-08-20T22:19:30.1774322Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-20T22:19:30.1828478Z inflating: build/bin/c10_Scalar_test 2024-08-20T22:19:30.1877898Z inflating: build/bin/c10_StreamGuard_test 2024-08-20T22:19:30.1929904Z inflating: build/bin/c10_SymInt_test 2024-08-20T22:19:30.1984135Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-20T22:19:30.2039932Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-20T22:19:30.2096676Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-20T22:19:30.2167845Z inflating: build/bin/c10_cow_test 2024-08-20T22:19:30.2221218Z inflating: build/bin/c10_Bitset_test 2024-08-20T22:19:30.2270283Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-20T22:19:30.2320412Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-20T22:19:30.2371275Z inflating: build/bin/c10_Half_test 2024-08-20T22:19:30.2428648Z inflating: build/bin/c10_LeftRight_test 2024-08-20T22:19:30.2484216Z inflating: build/bin/c10_Metaprogramming_test 2024-08-20T22:19:30.2534406Z inflating: build/bin/c10_Synchronized_test 2024-08-20T22:19:30.2590864Z inflating: build/bin/c10_ThreadLocal_test 2024-08-20T22:19:30.2643038Z inflating: build/bin/c10_TypeIndex_test 2024-08-20T22:19:30.2693835Z inflating: build/bin/c10_TypeList_test 2024-08-20T22:19:30.2743149Z inflating: build/bin/c10_TypeTraits_test 2024-08-20T22:19:30.2795267Z inflating: build/bin/c10_accumulate_test 2024-08-20T22:19:30.2851333Z inflating: build/bin/c10_bfloat16_test 2024-08-20T22:19:30.2901941Z inflating: build/bin/c10_bit_cast_test 2024-08-20T22:19:30.2958432Z inflating: build/bin/c10_complex_math_test 2024-08-20T22:19:30.3014433Z inflating: build/bin/c10_complex_test 2024-08-20T22:19:30.3067433Z inflating: build/bin/c10_exception_test 2024-08-20T22:19:30.3117836Z inflating: build/bin/c10_flags_test 2024-08-20T22:19:30.3168292Z inflating: build/bin/c10_generic_math_test 2024-08-20T22:19:30.3333899Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-20T22:19:30.3384506Z inflating: build/bin/c10_irange_test 2024-08-20T22:19:30.3438003Z inflating: build/bin/c10_lazy_test 2024-08-20T22:19:30.3496346Z inflating: build/bin/c10_logging_test 2024-08-20T22:19:30.3571611Z inflating: build/bin/c10_optional_test 2024-08-20T22:19:30.3634369Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-20T22:19:30.3689347Z inflating: build/bin/c10_registry_test 2024-08-20T22:19:30.3840219Z inflating: build/bin/c10_small_vector_test 2024-08-20T22:19:30.3893076Z inflating: build/bin/c10_ssize_test 2024-08-20T22:19:30.3944936Z inflating: build/bin/c10_string_util_test 2024-08-20T22:19:30.3995763Z inflating: build/bin/c10_tempfile_test 2024-08-20T22:19:30.4055146Z inflating: build/bin/c10_string_view_test 2024-08-20T22:19:30.4104561Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-20T22:19:30.4160384Z inflating: build/bin/c10_typeid_test 2024-08-20T22:19:30.4609359Z inflating: build/bin/protoc-3.13.0.0 2024-08-20T22:19:30.5055871Z inflating: build/bin/protoc 2024-08-20T22:19:30.5110009Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-20T22:19:30.5163205Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-20T22:19:30.5215598Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-20T22:19:30.5268793Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-20T22:19:30.5321048Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-20T22:19:30.5374188Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-20T22:19:30.5423930Z inflating: build/bin/c10_cuda_CUDATest 2024-08-20T22:19:30.5477389Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-20T22:19:30.5810016Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-20T22:19:30.6155544Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-20T22:19:30.6517589Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-20T22:19:30.6570791Z inflating: build/bin/BackoffTest 2024-08-20T22:19:30.6625453Z inflating: build/bin/FileStoreTest 2024-08-20T22:19:30.6681484Z inflating: build/bin/TCPStoreTest 2024-08-20T22:19:30.6735870Z inflating: build/bin/HashStoreTest 2024-08-20T22:19:30.6749343Z inflating: build/bin/ProcessGroupMPITest 2024-08-20T22:19:30.6804120Z inflating: build/bin/test_edge_op_registration 2024-08-20T22:19:30.6808441Z inflating: build/bin/torch_shm_manager 2024-08-20T22:19:30.6811004Z inflating: build/bin/example_allreduce 2024-08-20T22:19:30.6866847Z inflating: build/bin/test_dist_autograd 2024-08-20T22:19:30.6935826Z inflating: build/bin/test_cpp_rpc 2024-08-20T22:19:30.6938611Z inflating: build/bin/parallel_benchmark 2024-08-20T22:19:30.7006836Z inflating: build/bin/test_mobile_nnc 2024-08-20T22:19:30.7016053Z inflating: build/bin/aot_model_compiler_test 2024-08-20T22:19:30.7368095Z inflating: build/bin/test_lazy 2024-08-20T22:19:30.8554312Z inflating: build/bin/test_api 2024-08-20T22:19:30.8628252Z inflating: build/bin/Dict_test 2024-08-20T22:19:30.8680856Z inflating: build/bin/Dimname_test 2024-08-20T22:19:30.8746462Z inflating: build/bin/MaybeOwned_test 2024-08-20T22:19:30.8803914Z inflating: build/bin/NamedTensor_test 2024-08-20T22:19:30.8863518Z inflating: build/bin/apply_utils_test 2024-08-20T22:19:30.8923010Z inflating: build/bin/atest 2024-08-20T22:19:30.8986807Z inflating: build/bin/basic 2024-08-20T22:19:30.9042721Z inflating: build/bin/broadcast_test 2024-08-20T22:19:30.9093325Z inflating: build/bin/cpu_allocator_test 2024-08-20T22:19:30.9152113Z inflating: build/bin/cpu_generator_test 2024-08-20T22:19:30.9205775Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-20T22:19:30.9299440Z inflating: build/bin/cpu_rng_test 2024-08-20T22:19:30.9349768Z inflating: build/bin/dispatch_key_set_test 2024-08-20T22:19:30.9400565Z inflating: build/bin/dlconvertor_test 2024-08-20T22:19:30.9459585Z inflating: build/bin/extension_backend_test 2024-08-20T22:19:30.9515167Z inflating: build/bin/half_test 2024-08-20T22:19:30.9612041Z inflating: build/bin/ivalue_test 2024-08-20T22:19:30.9662216Z inflating: build/bin/lazy_tensor_test 2024-08-20T22:19:30.9716977Z inflating: build/bin/math_kernel_test 2024-08-20T22:19:30.9771990Z inflating: build/bin/memory_format_test 2024-08-20T22:19:30.9825000Z inflating: build/bin/memory_overlapping_test 2024-08-20T22:19:30.9878457Z inflating: build/bin/mobile_memory_cleanup 2024-08-20T22:19:30.9934905Z inflating: build/bin/native_test 2024-08-20T22:19:30.9986054Z inflating: build/bin/operator_name_test 2024-08-20T22:19:31.0037944Z inflating: build/bin/operators_test 2024-08-20T22:19:31.0090121Z inflating: build/bin/packedtensoraccessor_test 2024-08-20T22:19:31.0157801Z inflating: build/bin/pow_test 2024-08-20T22:19:31.0215308Z inflating: build/bin/quantized_test 2024-08-20T22:19:31.0266032Z inflating: build/bin/reduce_ops_test 2024-08-20T22:19:31.0317402Z inflating: build/bin/reportMemoryUsage_test 2024-08-20T22:19:31.0374432Z inflating: build/bin/scalar_tensor_test 2024-08-20T22:19:31.0433471Z inflating: build/bin/scalar_test 2024-08-20T22:19:31.0486387Z inflating: build/bin/StorageUtils_test 2024-08-20T22:19:31.0539223Z inflating: build/bin/stride_properties_test 2024-08-20T22:19:31.0619663Z inflating: build/bin/tensor_iterator_test 2024-08-20T22:19:31.0673387Z inflating: build/bin/test_parallel 2024-08-20T22:19:31.0676541Z inflating: build/bin/thread_init_test 2024-08-20T22:19:31.0733174Z inflating: build/bin/type_ptr_test 2024-08-20T22:19:31.0793481Z inflating: build/bin/type_test 2024-08-20T22:19:31.0846996Z inflating: build/bin/undefined_tensor_test 2024-08-20T22:19:31.0848369Z inflating: build/bin/verify_api_visibility 2024-08-20T22:19:31.0917445Z inflating: build/bin/legacy_vmap_test 2024-08-20T22:19:31.0969771Z inflating: build/bin/weakref_test 2024-08-20T22:19:31.1021584Z inflating: build/bin/wrapdim_test 2024-08-20T22:19:31.1082370Z inflating: build/bin/IListRef_test 2024-08-20T22:19:31.1135175Z inflating: build/bin/xla_tensor_test 2024-08-20T22:19:31.1241549Z inflating: build/bin/List_test 2024-08-20T22:19:31.1308896Z inflating: build/bin/KernelFunction_test 2024-08-20T22:19:31.1431837Z inflating: build/bin/kernel_function_legacy_test 2024-08-20T22:19:31.1530149Z inflating: build/bin/kernel_function_test 2024-08-20T22:19:31.1659512Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-20T22:19:31.1763816Z inflating: build/bin/kernel_lambda_test 2024-08-20T22:19:31.1826100Z inflating: build/bin/kernel_stackbased_test 2024-08-20T22:19:31.1923120Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-20T22:19:31.1974645Z inflating: build/bin/CppSignature_test 2024-08-20T22:19:31.2030796Z inflating: build/bin/backend_fallback_test 2024-08-20T22:19:31.2080289Z inflating: build/bin/op_allowlist_test 2024-08-20T22:19:31.2384602Z inflating: build/bin/op_registration_test 2024-08-20T22:19:31.2448726Z inflating: build/bin/inline_container_test 2024-08-20T22:19:31.2502073Z inflating: build/bin/cuda_apply_test 2024-08-20T22:19:31.2555245Z inflating: build/bin/cuda_allocator_test 2024-08-20T22:19:31.2610823Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-20T22:19:31.2669679Z inflating: build/bin/cuda_atomic_ops_test 2024-08-20T22:19:31.2739877Z inflating: build/bin/cuda_complex_math_test 2024-08-20T22:19:31.2799628Z inflating: build/bin/cuda_complex_test 2024-08-20T22:19:31.2850482Z inflating: build/bin/cuda_device_test 2024-08-20T22:19:31.2909101Z inflating: build/bin/cuda_cub_test 2024-08-20T22:19:31.2960598Z inflating: build/bin/cuda_dlconvertor_test 2024-08-20T22:19:31.3025476Z inflating: build/bin/cuda_distributions_test 2024-08-20T22:19:31.3082615Z inflating: build/bin/cuda_generator_test 2024-08-20T22:19:31.3132506Z inflating: build/bin/cuda_half_test 2024-08-20T22:19:31.3184250Z inflating: build/bin/cuda_integer_divider_test 2024-08-20T22:19:31.3233826Z inflating: build/bin/cuda_optional_test 2024-08-20T22:19:31.3286088Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-20T22:19:31.3339879Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-20T22:19:31.3389556Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-20T22:19:31.3450653Z inflating: build/bin/cuda_stream_test 2024-08-20T22:19:31.3500587Z inflating: build/bin/cuda_cudnn_test 2024-08-20T22:19:31.3553109Z inflating: build/bin/cuda_vectorized_test 2024-08-20T22:19:31.3567628Z inflating: build/bin/tutorial_tensorexpr 2024-08-20T22:19:31.3633582Z inflating: build/bin/ProcessGroupGlooTest 2024-08-20T22:19:31.3691671Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-20T22:19:31.3755692Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-20T22:19:31.3818986Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-20T22:19:31.4649898Z inflating: build/bin/test_tensorexpr 2024-08-20T22:19:31.5232157Z inflating: build/bin/test_jit 2024-08-20T22:19:31.5232755Z creating: .additional_ci_files/ 2024-08-20T22:19:31.5293706Z inflating: .additional_ci_files/test-times.json 2024-08-20T22:19:31.5535293Z inflating: .additional_ci_files/test-class-times.json 2024-08-20T22:19:31.5570132Z ##[group]Run rm artifacts.zip 2024-08-20T22:19:31.5570527Z rm artifacts.zip 2024-08-20T22:19:31.5580278Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:31.5580784Z env: 2024-08-20T22:19:31.5581063Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:31.5581522Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:31.5582022Z ##[endgroup] 2024-08-20T22:19:31.6858597Z ##[group]Run df -H 2024-08-20T22:19:31.6859219Z df -H 2024-08-20T22:19:31.6870806Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:31.6871560Z env: 2024-08-20T22:19:31.6871953Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:31.6872623Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:31.6873367Z ##[endgroup] 2024-08-20T22:19:31.6932123Z Filesystem Size Used Avail Use% Mounted on 2024-08-20T22:19:31.6932740Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-20T22:19:31.6933393Z tmpfs 34G 0 34G 0% /dev/shm 2024-08-20T22:19:31.6934004Z tmpfs 14G 553k 14G 1% /run 2024-08-20T22:19:31.6934556Z /dev/nvme0n1p1 161G 53G 109G 33% / 2024-08-20T22:19:31.6935272Z tmpfs 34G 4.1k 34G 1% /tmp 2024-08-20T22:19:31.6936116Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-20T22:19:31.6936788Z tmpfs 6.7G 0 6.7G 0% /run/user/0 2024-08-20T22:19:31.6980830Z Prepare all required actions 2024-08-20T22:19:31.6981286Z Getting action download info 2024-08-20T22:19:31.8276406Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-20T22:19:31.8276884Z with: 2024-08-20T22:19:31.8277142Z env: 2024-08-20T22:19:31.8277423Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:31.8277883Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:31.8278375Z ##[endgroup] 2024-08-20T22:19:31.8311374Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:19:31.8311824Z with: 2024-08-20T22:19:31.8312091Z name: td_results 2024-08-20T22:19:31.8312416Z s3-bucket: gha-artifacts 2024-08-20T22:19:31.8312769Z region: us-east-1 2024-08-20T22:19:31.8313078Z env: 2024-08-20T22:19:31.8313352Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:31.8313816Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:31.8314301Z ##[endgroup] 2024-08-20T22:19:32.3019419Z (node:52500) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:19:32.3020257Z 2024-08-20T22:19:32.3020594Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:19:32.3021507Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:19:32.3022446Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:19:32.3539637Z Found 0 objects with prefix pytorch/pytorch/10479310725/td_results/ 2024-08-20T22:19:32.3546401Z Artifact download has finished successfully 2024-08-20T22:19:32.3863704Z ##[group]Run mkdir -p .additional_ci_files 2024-08-20T22:19:32.3864196Z mkdir -p .additional_ci_files 2024-08-20T22:19:32.3864768Z mv td_results.json .additional_ci_files/td_results.json 2024-08-20T22:19:32.3874654Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:32.3875165Z env: 2024-08-20T22:19:32.3875458Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:32.3876120Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:32.3876620Z ##[endgroup] 2024-08-20T22:19:32.3933528Z mv: cannot stat 'td_results.json': No such file or directory 2024-08-20T22:19:32.3948407Z ##[error]Process completed with exit code 1. 2024-08-20T22:19:32.3983075Z ##[group]Run .github/scripts/parse_ref.py 2024-08-20T22:19:32.3983567Z .github/scripts/parse_ref.py 2024-08-20T22:19:32.3991742Z shell: /usr/bin/bash -e {0} 2024-08-20T22:19:32.3992125Z env: 2024-08-20T22:19:32.3992410Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:32.3992876Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:32.3993366Z ##[endgroup] 2024-08-20T22:19:32.4281273Z Prepare all required actions 2024-08-20T22:19:32.4321534Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-20T22:19:32.4321997Z with: 2024-08-20T22:19:32.4322594Z github-token: *** 2024-08-20T22:19:32.4322911Z env: 2024-08-20T22:19:32.4323189Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:32.4323676Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:32.4324172Z ##[endgroup] 2024-08-20T22:19:32.4341227Z ##[group]Run set -eux 2024-08-20T22:19:32.4341655Z set -eux 2024-08-20T22:19:32.4342340Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-20T22:19:32.4351736Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:32.4352238Z env: 2024-08-20T22:19:32.4352532Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:32.4352999Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:32.4353639Z GITHUB_TOKEN: *** 2024-08-20T22:19:32.4353956Z ##[endgroup] 2024-08-20T22:19:32.4382179Z + python3 .github/scripts/get_workflow_job_id.py 10479310725 i-0aa26b6dbaf56f3e9 2024-08-20T22:19:33.9292751Z setting job-id=29026426405 2024-08-20T22:19:33.9294033Z setting job-name=cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:19:33.9517928Z Prepare all required actions 2024-08-20T22:19:33.9518384Z Getting action download info 2024-08-20T22:19:34.0921486Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-20T22:19:34.0921956Z with: 2024-08-20T22:19:34.0922415Z github-token: *** 2024-08-20T22:19:34.0923582Z test-matrix: {"include": [{"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:19:34.0925154Z job-name: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:19:34.0925891Z env: 2024-08-20T22:19:34.0926187Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:34.0926657Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:34.0927151Z ##[endgroup] 2024-08-20T22:19:34.0969766Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:19:34.0970352Z with: 2024-08-20T22:19:34.0970631Z shell: bash 2024-08-20T22:19:34.0970933Z timeout_minutes: 10 2024-08-20T22:19:34.0971265Z max_attempts: 5 2024-08-20T22:19:34.0971586Z retry_wait_seconds: 30 2024-08-20T22:19:34.0972705Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:19:34.0973875Z polling_interval_seconds: 1 2024-08-20T22:19:34.0974259Z warning_on_retry: true 2024-08-20T22:19:34.0974612Z continue_on_error: false 2024-08-20T22:19:34.0974947Z env: 2024-08-20T22:19:34.0975229Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:34.0975682Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:34.0976325Z GITHUB_TOKEN: *** 2024-08-20T22:19:34.0976643Z ##[endgroup] 2024-08-20T22:19:34.1726433Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:19:34.4085797Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:19:34.5546657Z Collecting requests==2.27.1 2024-08-20T22:19:34.5893675Z Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB) 2024-08-20T22:19:34.7836038Z Collecting pyyaml==6.0.1 2024-08-20T22:19:34.7877449Z Downloading PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (738 kB) 2024-08-20T22:19:34.8190727Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-20T22:19:34.8199506Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-20T22:19:35.1193377Z Collecting charset-normalizer~=2.0.0 2024-08-20T22:19:35.1233800Z Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 2024-08-20T22:19:35.1731647Z Collecting certifi>=2017.4.17 2024-08-20T22:19:35.1772060Z Downloading certifi-2024.7.4-py3-none-any.whl (162 kB) 2024-08-20T22:19:35.2571797Z Installing collected packages: charset-normalizer, certifi, requests, pyyaml 2024-08-20T22:19:35.3774148Z Successfully installed certifi-2024.7.4 charset-normalizer-2.0.12 pyyaml-6.0.1 requests-2.27.1 2024-08-20T22:19:36.1518047Z Command completed after 1 attempt(s). 2024-08-20T22:19:36.1581440Z ##[group]Run set -x 2024-08-20T22:19:36.1606078Z set -x 2024-08-20T22:19:36.1606397Z  2024-08-20T22:19:36.1606961Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:19:36.1607641Z # in runner workspace 2024-08-20T22:19:36.1608171Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-20T22:19:36.1617209Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:36.1617712Z env: 2024-08-20T22:19:36.1617993Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:36.1618457Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:36.1619010Z ##[endgroup] 2024-08-20T22:19:36.1650269Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-20T22:19:36.1895285Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:19:36.1895833Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:19:36.1896303Z echo "Job name: ${JOB_NAME}" 2024-08-20T22:19:36.1896711Z  2024-08-20T22:19:36.1897269Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:19:36.1897961Z # in runner workspace 2024-08-20T22:19:36.1898550Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-20T22:19:36.1899350Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-20T22:19:36.1899823Z  --job-name "${JOB_NAME}" \ 2024-08-20T22:19:36.1901146Z  --test-matrix "{"include": [{"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-08-20T22:19:36.1902482Z  --selected-test-configs "" \ 2024-08-20T22:19:36.1902947Z  --pr-number "${PR_NUMBER}" \ 2024-08-20T22:19:36.1903381Z  --tag "${TAG}" \ 2024-08-20T22:19:36.1903791Z  --event-name "${EVENT_NAME}" \ 2024-08-20T22:19:36.1904248Z  --schedule "${SCHEDULE}" \ 2024-08-20T22:19:36.1904688Z  --branch "${HEAD_BRANCH}" 2024-08-20T22:19:36.1913747Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:36.1914255Z env: 2024-08-20T22:19:36.1914546Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:36.1915014Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:36.1915695Z GITHUB_TOKEN: *** 2024-08-20T22:19:36.1916369Z JOB_NAME: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:19:36.1917114Z PR_NUMBER: 2024-08-20T22:19:36.1917437Z TAG: ciflow/inductor/133712 2024-08-20T22:19:36.1917817Z EVENT_NAME: push 2024-08-20T22:19:36.1918303Z SCHEDULE: 2024-08-20T22:19:36.1918601Z HEAD_BRANCH: 2024-08-20T22:19:36.1918907Z ##[endgroup] 2024-08-20T22:19:36.1947471Z Workflow: inductor 2024-08-20T22:19:36.1948441Z Job name: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:19:36.4344184Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-20T22:19:36.6023650Z ##[group]Run echo "Filtered matrix:" 2024-08-20T22:19:36.6024115Z echo "Filtered matrix:" 2024-08-20T22:19:36.6025351Z echo "{"include": [{"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]}" 2024-08-20T22:19:36.6026572Z  2024-08-20T22:19:36.6026852Z echo 2024-08-20T22:19:36.6027256Z echo "Is the current job unstable? False" 2024-08-20T22:19:36.6027731Z  2024-08-20T22:19:36.6028013Z echo 2024-08-20T22:19:36.6028382Z echo "Is keep-going label set? False" 2024-08-20T22:19:36.6028836Z  2024-08-20T22:19:36.6029112Z echo 2024-08-20T22:19:36.6029436Z echo "Renabled issues? " 2024-08-20T22:19:36.6038303Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:36.6038813Z env: 2024-08-20T22:19:36.6039101Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:36.6039570Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:36.6040068Z ##[endgroup] 2024-08-20T22:19:36.6070653Z Filtered matrix: 2024-08-20T22:19:36.6072196Z {include: [{config: inductor_timm, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}]} 2024-08-20T22:19:36.6073397Z 2024-08-20T22:19:36.6073570Z Is the current job unstable? False 2024-08-20T22:19:36.6074182Z 2024-08-20T22:19:36.6074555Z Is keep-going label set? False 2024-08-20T22:19:36.6074818Z 2024-08-20T22:19:36.6074944Z Renabled issues? 2024-08-20T22:19:36.6129419Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:19:36.6130145Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:19:36.6139212Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:36.6139719Z env: 2024-08-20T22:19:36.6140019Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:36.6140488Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:36.6140997Z JOB_TIMEOUT: 240 2024-08-20T22:19:36.6141315Z ##[endgroup] 2024-08-20T22:19:36.6228443Z ##[group]Run set -x 2024-08-20T22:19:36.6228850Z set -x 2024-08-20T22:19:36.6229157Z  2024-08-20T22:19:36.6229526Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-20T22:19:36.6230085Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-20T22:19:36.6230662Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-20T22:19:36.6231207Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-20T22:19:36.6231634Z else 2024-08-20T22:19:36.6231989Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:19:36.6232422Z fi 2024-08-20T22:19:36.6232714Z  2024-08-20T22:19:36.6233198Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-20T22:19:36.6233966Z # TODO: Stop building test binaries as part of the build phase 2024-08-20T22:19:36.6234660Z # Used for GPU_FLAG since that doesn't play nice 2024-08-20T22:19:36.6235255Z # shellcheck disable=SC2086,SC2090 2024-08-20T22:19:36.6235731Z container_name=$(docker run \ 2024-08-20T22:19:36.6236165Z  ${GPU_FLAG:-} \ 2024-08-20T22:19:36.6236548Z  -e BUILD_ENVIRONMENT \ 2024-08-20T22:19:36.6236961Z  -e PR_NUMBER \ 2024-08-20T22:19:36.6237345Z  -e GITHUB_ACTIONS \ 2024-08-20T22:19:36.6237751Z  -e GITHUB_REPOSITORY \ 2024-08-20T22:19:36.6238349Z  -e GITHUB_WORKFLOW \ 2024-08-20T22:19:36.6238752Z  -e GITHUB_JOB \ 2024-08-20T22:19:36.6239130Z  -e GITHUB_RUN_ID \ 2024-08-20T22:19:36.6239528Z  -e GITHUB_RUN_NUMBER \ 2024-08-20T22:19:36.6239950Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-20T22:19:36.6240361Z  -e JOB_ID \ 2024-08-20T22:19:36.6240710Z  -e JOB_NAME \ 2024-08-20T22:19:36.6241079Z  -e BASE_SHA \ 2024-08-20T22:19:36.6241437Z  -e BRANCH \ 2024-08-20T22:19:36.6241778Z  -e SHA1 \ 2024-08-20T22:19:36.6242139Z  -e AWS_DEFAULT_REGION \ 2024-08-20T22:19:36.6242564Z  -e IN_WHEEL_TEST \ 2024-08-20T22:19:36.6242962Z  -e SHARD_NUMBER \ 2024-08-20T22:19:36.6243341Z  -e TEST_CONFIG \ 2024-08-20T22:19:36.6243727Z  -e NUM_TEST_SHARDS \ 2024-08-20T22:19:36.6244147Z  -e REENABLED_ISSUES \ 2024-08-20T22:19:36.6244579Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-20T22:19:36.6245028Z  -e VERBOSE_TEST_LOGS \ 2024-08-20T22:19:36.6245449Z  -e TEST_SHOWLOCALS \ 2024-08-20T22:19:36.6245863Z  -e NO_TEST_TIMEOUT \ 2024-08-20T22:19:36.6246504Z  -e NO_TD \ 2024-08-20T22:19:36.6246889Z  -e TD_DISTRIBUTED \ 2024-08-20T22:19:36.6247277Z  -e PR_LABELS \ 2024-08-20T22:19:36.6247696Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-20T22:19:36.6248157Z  -e SCCACHE_BUCKET \ 2024-08-20T22:19:36.6248562Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-20T22:19:36.6248983Z  -e XLA_CUDA \ 2024-08-20T22:19:36.6249401Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-20T22:19:36.6249917Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-20T22:19:36.6250449Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-20T22:19:36.6250968Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-20T22:19:36.6251447Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-20T22:19:36.6251913Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-20T22:19:36.6252359Z  -e DASHBOARD_TAG \ 2024-08-20T22:19:36.6252843Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-20T22:19:36.6253401Z  --security-opt seccomp=unconfined \ 2024-08-20T22:19:36.6253906Z  --cap-add=SYS_PTRACE \ 2024-08-20T22:19:36.6254329Z  --ipc=host \ 2024-08-20T22:19:36.6254695Z  --shm-size="${SHM_SIZE}" \ 2024-08-20T22:19:36.6255097Z  --tty \ 2024-08-20T22:19:36.6255412Z  --detach \ 2024-08-20T22:19:36.6255957Z  --name="${container_name}" \ 2024-08-20T22:19:36.6256400Z  --user jenkins \ 2024-08-20T22:19:36.6256905Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-20T22:19:36.6257479Z  -w /var/lib/jenkins/workspace \ 2024-08-20T22:19:36.6257933Z  "${DOCKER_IMAGE}" 2024-08-20T22:19:36.6258298Z ) 2024-08-20T22:19:36.6258788Z # Propagate download.pytorch.org IP to container 2024-08-20T22:19:36.6259717Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-20T22:19:36.6260694Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-20T22:19:36.6261614Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-20T22:19:36.6270293Z shell: /usr/bin/bash -e {0} 2024-08-20T22:19:36.6270658Z env: 2024-08-20T22:19:36.6270948Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:36.6271434Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:36.6272050Z BUILD_ENVIRONMENT: linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-08-20T22:19:36.6272566Z PR_NUMBER: 2024-08-20T22:19:36.6272906Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-20T22:19:36.6273336Z GITHUB_WORKFLOW: inductor 2024-08-20T22:19:36.6273696Z GITHUB_JOB: test 2024-08-20T22:19:36.6274175Z GITHUB_RUN_ID: 10479310725 2024-08-20T22:19:36.6274558Z GITHUB_RUN_NUMBER: 88082 2024-08-20T22:19:36.6274924Z GITHUB_RUN_ATTEMPT: 1 2024-08-20T22:19:36.6275278Z JOB_ID: 29026426405 2024-08-20T22:19:36.6275957Z JOB_NAME: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:19:36.6276698Z BRANCH: 2024-08-20T22:19:36.6277042Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:19:36.6277593Z BASE_SHA: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:19:36.6278087Z TEST_CONFIG: inductor_timm 2024-08-20T22:19:36.6278462Z SHARD_NUMBER: 2 2024-08-20T22:19:36.6278787Z NUM_TEST_SHARDS: 2 2024-08-20T22:19:36.6279127Z REENABLED_ISSUES: 2024-08-20T22:19:36.6279483Z CONTINUE_THROUGH_ERROR: False 2024-08-20T22:19:36.6279889Z VERBOSE_TEST_LOGS: False 2024-08-20T22:19:36.6280257Z TEST_SHOWLOCALS: False 2024-08-20T22:19:36.6280624Z NO_TEST_TIMEOUT: False 2024-08-20T22:19:36.6280975Z NO_TD: False 2024-08-20T22:19:36.6281295Z TD_DISTRIBUTED: False 2024-08-20T22:19:36.6281733Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-20T22:19:36.6282244Z SCCACHE_S3_KEY_PREFIX: inductor 2024-08-20T22:19:36.6282644Z SHM_SIZE: 2g 2024-08-20T22:19:36.6283820Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:19:36.6285008Z XLA_CUDA: 2024-08-20T22:19:36.6285515Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:19:36.6286148Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-20T22:19:36.6286607Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-20T22:19:36.6287025Z DASHBOARD_TAG: 2024-08-20T22:19:36.6287549Z HUGGING_FACE_HUB_TOKEN: *** 2024-08-20T22:19:36.6287948Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-20T22:19:36.6288335Z ##[endgroup] 2024-08-20T22:19:36.6317532Z + [[ inductor_timm == \m\u\l\t\i\g\p\u ]] 2024-08-20T22:19:36.6318342Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *onnx* ]] 2024-08-20T22:19:36.6318861Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:19:36.6326661Z +++ nproc --ignore=2 2024-08-20T22:19:36.6357085Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479310725 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:19:48.1040444Z + container_name=5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T22:19:48.1042677Z + grep download.pytorch.org /etc/hosts 2024-08-20T22:19:48.1044615Z + docker exec -i 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-20T22:19:48.2415878Z + echo DOCKER_CONTAINER_ID=5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T22:19:48.2419636Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:19:48.2422597Z + docker exec -t 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-20T22:19:48.6394451Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-20T22:19:48.9655676Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-20T22:19:48.9658162Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-20T22:19:48.9660156Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-20T22:19:48.9662539Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-20T22:19:48.9665316Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-20T22:19:48.9669998Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.13.1) 2024-08-20T22:19:48.9686017Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-20T22:19:48.9699050Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-20T22:19:48.9716251Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.22.4) 2024-08-20T22:19:49.0102731Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-20T22:19:49.5802741Z Installing collected packages: torch 2024-08-20T22:19:59.3345442Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T22:19:59.3347188Z timm 1.0.9.dev0 requires torchvision, which is not installed. 2024-08-20T22:19:59.3348373Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-20T22:19:59.4237560Z + export TERM=vt100 2024-08-20T22:19:59.4237967Z + TERM=vt100 2024-08-20T22:19:59.4241619Z ++ dirname .ci/pytorch/test.sh 2024-08-20T22:19:59.4252987Z + source .ci/pytorch/common.sh 2024-08-20T22:19:59.4256116Z +++ dirname .ci/pytorch/common.sh 2024-08-20T22:19:59.4265751Z ++ source .ci/pytorch/common_utils.sh 2024-08-20T22:19:59.4267490Z +++ declare -f -t trap_add 2024-08-20T22:19:59.4272905Z ++ set -ex 2024-08-20T22:19:59.4273474Z ++ [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:19:59.4274112Z ++ BUILD_TEST_LIBTORCH=0 2024-08-20T22:19:59.4274645Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *rocm* ]] 2024-08-20T22:19:59.4277332Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-20T22:19:59.4296928Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-20T22:19:59.4297366Z + trap_add cleanup_workspace EXIT 2024-08-20T22:19:59.4297773Z + trap_add_cmd=cleanup_workspace 2024-08-20T22:19:59.4298146Z + shift 2024-08-20T22:19:59.4300396Z + for trap_add_name in "$@" 2024-08-20T22:19:59.4304548Z +++ trap -p EXIT 2024-08-20T22:19:59.4307355Z ++ eval 'extract_trap_cmd ' 2024-08-20T22:19:59.4307735Z +++ extract_trap_cmd 2024-08-20T22:19:59.4308095Z +++ printf '%s\n' '' 2024-08-20T22:19:59.4308491Z ++ printf '%s\n' cleanup_workspace 2024-08-20T22:19:59.4310308Z + trap -- ' 2024-08-20T22:19:59.4311064Z cleanup_workspace' EXIT 2024-08-20T22:19:59.4311590Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-20T22:20:00.1436232Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-20T22:20:00.1457631Z + echo 'Environment variables:' 2024-08-20T22:20:00.1458200Z Environment variables: 2024-08-20T22:20:00.1458627Z + env 2024-08-20T22:20:00.1468045Z INSTALLED_DB=yes 2024-08-20T22:20:00.1468603Z NV_LIBCUBLAS_VERSION=12.4.2.65-1 2024-08-20T22:20:00.1469075Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:20:00.1469487Z NV_NVML_DEV_VERSION=12.4.99-1 2024-08-20T22:20:00.1470128Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:20:00.1470853Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:20:00.1471451Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.20.5-1+cuda12.4 2024-08-20T22:20:00.1472003Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.20.5-1 2024-08-20T22:20:00.1472794Z BUILD_ENVIRONMENT=linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-08-20T22:20:00.1473482Z HOSTNAME=5d65ee9831b2 2024-08-20T22:20:00.1474629Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_8ab20202-3186-4adb-a884-108604e7b639 2024-08-20T22:20:00.1475723Z GITHUB_ACTION=__self 2024-08-20T22:20:00.1476207Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:20:00.1482279Z NVIDIA_REQUIRE_CUDA=cuda>=12.4 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 brand=tesla,driver>=535,driver<536 brand=unknown,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=geforce,driver>=535,driver<536 brand=geforcertx,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=titan,driver>=535,driver<536 brand=titanrtx,driver>=535,driver<536 2024-08-20T22:20:00.1488529Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-4=12.4.2.65-1 2024-08-20T22:20:00.1489049Z NV_NVTX_VERSION=12.4.99-1 2024-08-20T22:20:00.1489393Z GITHUB_RUN_NUMBER=88082 2024-08-20T22:20:00.1490093Z TEST_CONFIG=inductor_timm 2024-08-20T22:20:00.1490469Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:20:00.1490958Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:20:00.1491415Z NV_CUDA_CUDART_DEV_VERSION=12.4.99-1 2024-08-20T22:20:00.1491859Z NV_LIBCUSPARSE_VERSION=12.3.0.142-1 2024-08-20T22:20:00.1492266Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:20:00.1492668Z NV_LIBNPP_VERSION=12.2.5.2-1 2024-08-20T22:20:00.1493114Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:20:00.1493610Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:20:00.1494073Z GITHUB_REF_TYPE=tag 2024-08-20T22:20:00.1494398Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:20:00.1494778Z NCCL_VERSION=2.20.5-1 2024-08-20T22:20:00.1495167Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:20:00.1495609Z XLA_CUDA= 2024-08-20T22:20:00.1496095Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:20:00.1498446Z *** 2024-08-20T22:20:00.1498854Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:20:00.1499269Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:20:00.1499629Z GITHUB_ACTIONS=true 2024-08-20T22:20:00.1499967Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:00.1500494Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-4=12.4.99-1 2024-08-20T22:20:00.1501031Z NV_LIBNPP_PACKAGE=libnpp-12-4=12.2.5.2-1 2024-08-20T22:20:00.1501496Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:20:00.1502203Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:20:00.1502687Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:20:00.1503480Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:20:00.1504189Z UCC_HOME=/usr 2024-08-20T22:20:00.1504527Z NV_LIBCUBLAS_DEV_VERSION=12.4.2.65-1 2024-08-20T22:20:00.1504932Z VERBOSE_TEST_LOGS=False 2024-08-20T22:20:00.1505276Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:20:00.1505733Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-4 2024-08-20T22:20:00.1506232Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:20:00.1506708Z NV_CUDA_CUDART_VERSION=12.4.99-1 2024-08-20T22:20:00.1507075Z SHARD_NUMBER=2 2024-08-20T22:20:00.1507385Z GITHUB_REF_PROTECTED=false 2024-08-20T22:20:00.1507737Z HOME=/var/lib/jenkins 2024-08-20T22:20:00.1508102Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:20:00.1508534Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:20:00.1509008Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:20:00.1509494Z SCCACHE_S3_KEY_PREFIX=inductor 2024-08-20T22:20:00.1509862Z CUDA_VERSION=12.4.0 2024-08-20T22:20:00.1510286Z NV_LIBCUBLAS_PACKAGE=libcublas-12-4=12.4.2.65-1 2024-08-20T22:20:00.1510744Z NUM_TEST_SHARDS=2 2024-08-20T22:20:00.1511080Z UCX_HOME=/usr 2024-08-20T22:20:00.1511595Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-4=12.4.0-1 2024-08-20T22:20:00.1512671Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_8ab20202-3186-4adb-a884-108604e7b639 2024-08-20T22:20:00.1513911Z JOB_NAME=cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:20:00.1515132Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_8ab20202-3186-4adb-a884-108604e7b639 2024-08-20T22:20:00.1516238Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:20:00.1516883Z GITHUB_EVENT_NAME=push 2024-08-20T22:20:00.1517225Z DASHBOARD_TAG= 2024-08-20T22:20:00.1517529Z GITHUB_RUN_ID=10479310725 2024-08-20T22:20:00.1517990Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-4=12.2.5.2-1 2024-08-20T22:20:00.1518524Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-4 2024-08-20T22:20:00.1519514Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_8ab20202-3186-4adb-a884-108604e7b639 2024-08-20T22:20:00.1520423Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:20:00.1520837Z NV_LIBNPP_DEV_VERSION=12.2.5.2-1 2024-08-20T22:20:00.1521206Z PR_NUMBER= 2024-08-20T22:20:00.1521493Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:20:00.1521829Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:20:00.1522374Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:20:00.1522837Z TERM=vt100 2024-08-20T22:20:00.1523177Z NV_LIBCUSPARSE_DEV_VERSION=12.3.0.142-1 2024-08-20T22:20:00.1523592Z INSTALLED_VISION=yes 2024-08-20T22:20:00.1523899Z BRANCH= 2024-08-20T22:20:00.1524181Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:20:00.1524581Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:20:00.1525007Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:20:00.1525761Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:20:00.1526511Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:20:00.1526996Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:20:00.1527459Z REENABLED_ISSUES= 2024-08-20T22:20:00.1527758Z SHLVL=1 2024-08-20T22:20:00.1528026Z MAX_JOBS=14 2024-08-20T22:20:00.1528340Z NV_CUDA_LIB_VERSION=12.4.0-1 2024-08-20T22:20:00.1528693Z NVARCH=x86_64 2024-08-20T22:20:00.1528996Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:20:00.1529456Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:20:00.1529984Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:20:00.1530481Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4 2024-08-20T22:20:00.1531199Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:20:00.1531766Z GITHUB_JOB=test 2024-08-20T22:20:00.1532259Z NV_LIBNCCL_PACKAGE=libnccl2=2.20.5-1+cuda12.4 2024-08-20T22:20:00.1532812Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:20:00.1533321Z NO_TEST_TIMEOUT=False 2024-08-20T22:20:00.1533653Z TD_DISTRIBUTED=False 2024-08-20T22:20:00.1534025Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.4.0-1 2024-08-20T22:20:00.1534467Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:20:00.1534907Z NV_NVPROF_VERSION=12.4.99-1 2024-08-20T22:20:00.1535270Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:20:00.1535618Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:20:00.1535974Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:20:00.1537025Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:20:00.1538096Z GITHUB_BASE_REF= 2024-08-20T22:20:00.1538418Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:20:00.1538883Z CI=true 2024-08-20T22:20:00.1539207Z NV_LIBNCCL_PACKAGE_VERSION=2.20.5-1 2024-08-20T22:20:00.1539628Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:20:00.1539995Z JOB_ID=29026426405 2024-08-20T22:20:00.1540311Z INSTALLED_PROTOBUF=yes 2024-08-20T22:20:00.1540645Z GITHUB_HEAD_REF= 2024-08-20T22:20:00.1540979Z GITHUB_ACTION_REF= 2024-08-20T22:20:00.1541428Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:20:00.1541890Z TEST_SHOWLOCALS=False 2024-08-20T22:20:00.1542224Z GITHUB_WORKFLOW=inductor 2024-08-20T22:20:00.1542576Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:20:00.1543492Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_8ab20202-3186-4adb-a884-108604e7b639 2024-08-20T22:20:00.1544301Z NO_TD=False 2024-08-20T22:20:00.1544602Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:20:00.1544961Z _=/usr/bin/env 2024-08-20T22:20:00.1545451Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-20T22:20:00.1704051Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-20T22:20:00.1705263Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-20T22:20:00.1706293Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-20T22:20:00.1707265Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-20T22:20:00.1707871Z + BUILD_DIR=build 2024-08-20T22:20:00.1708198Z + BUILD_RENAMED_DIR=build_renamed 2024-08-20T22:20:00.1708582Z + BUILD_BIN_DIR=build/bin 2024-08-20T22:20:00.1708922Z + SHARD_NUMBER=2 2024-08-20T22:20:00.1709225Z + NUM_TEST_SHARDS=2 2024-08-20T22:20:00.1709545Z + export VALGRIND=ON 2024-08-20T22:20:00.1709858Z + VALGRIND=ON 2024-08-20T22:20:00.1710524Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *clang9* ]] 2024-08-20T22:20:00.1711087Z + [[ 0 == \1 ]] 2024-08-20T22:20:00.1711403Z + [[ False == \1 ]] 2024-08-20T22:20:00.1711897Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *bazel* ]] 2024-08-20T22:20:00.1712588Z ++ realpath build/custom_test_artifacts 2024-08-20T22:20:00.1725905Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-20T22:20:00.1726853Z + [[ -n '' ]] 2024-08-20T22:20:00.1727211Z + echo 'Environment variables' 2024-08-20T22:20:00.1727592Z Environment variables 2024-08-20T22:20:00.1728057Z + env 2024-08-20T22:20:00.1736227Z INSTALLED_DB=yes 2024-08-20T22:20:00.1736742Z NV_LIBCUBLAS_VERSION=12.4.2.65-1 2024-08-20T22:20:00.1737274Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:20:00.1737789Z NV_NVML_DEV_VERSION=12.4.99-1 2024-08-20T22:20:00.1738853Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:20:00.1739612Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:20:00.1740293Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.20.5-1+cuda12.4 2024-08-20T22:20:00.1740976Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.20.5-1 2024-08-20T22:20:00.1741554Z BUILD_ENVIRONMENT=linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-08-20T22:20:00.1742049Z HOSTNAME=5d65ee9831b2 2024-08-20T22:20:00.1743202Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_8ab20202-3186-4adb-a884-108604e7b639 2024-08-20T22:20:00.1744268Z GITHUB_ACTION=__self 2024-08-20T22:20:00.1744686Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:20:00.1751406Z NVIDIA_REQUIRE_CUDA=cuda>=12.4 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 brand=tesla,driver>=535,driver<536 brand=unknown,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=geforce,driver>=535,driver<536 brand=geforcertx,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=titan,driver>=535,driver<536 brand=titanrtx,driver>=535,driver<536 2024-08-20T22:20:00.1757398Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-4=12.4.2.65-1 2024-08-20T22:20:00.1757919Z NV_NVTX_VERSION=12.4.99-1 2024-08-20T22:20:00.1758264Z GITHUB_RUN_NUMBER=88082 2024-08-20T22:20:00.1758757Z TEST_CONFIG=inductor_timm 2024-08-20T22:20:00.1759127Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:20:00.1759611Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:20:00.1760076Z NV_CUDA_CUDART_DEV_VERSION=12.4.99-1 2024-08-20T22:20:00.1760522Z NV_LIBCUSPARSE_VERSION=12.3.0.142-1 2024-08-20T22:20:00.1760981Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:20:00.1761371Z NV_LIBNPP_VERSION=12.2.5.2-1 2024-08-20T22:20:00.1761818Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:20:00.1762324Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:20:00.1762790Z GITHUB_REF_TYPE=tag 2024-08-20T22:20:00.1763113Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:20:00.1763497Z NCCL_VERSION=2.20.5-1 2024-08-20T22:20:00.1763889Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:20:00.1764328Z XLA_CUDA= 2024-08-20T22:20:00.1764770Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:20:00.1765157Z *** 2024-08-20T22:20:00.1765440Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:20:00.1765837Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:20:00.1766195Z GITHUB_ACTIONS=true 2024-08-20T22:20:00.1766700Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:00.1767205Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-4=12.4.99-1 2024-08-20T22:20:00.1767737Z NV_LIBNPP_PACKAGE=libnpp-12-4=12.2.5.2-1 2024-08-20T22:20:00.1768203Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:20:00.1768723Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:20:00.1769205Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:20:00.1769997Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:20:00.1770711Z UCC_HOME=/usr 2024-08-20T22:20:00.1771051Z NV_LIBCUBLAS_DEV_VERSION=12.4.2.65-1 2024-08-20T22:20:00.1771455Z VERBOSE_TEST_LOGS=False 2024-08-20T22:20:00.1771794Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:20:00.1772258Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-4 2024-08-20T22:20:00.1772762Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:20:00.1773232Z NV_CUDA_CUDART_VERSION=12.4.99-1 2024-08-20T22:20:00.1773600Z SHARD_NUMBER=2 2024-08-20T22:20:00.1773917Z GITHUB_REF_PROTECTED=false 2024-08-20T22:20:00.1774272Z HOME=/var/lib/jenkins 2024-08-20T22:20:00.1774632Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:20:00.1775069Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:20:00.1775545Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:20:00.1776025Z SCCACHE_S3_KEY_PREFIX=inductor 2024-08-20T22:20:00.1776544Z CUDA_VERSION=12.4.0 2024-08-20T22:20:00.1776982Z NV_LIBCUBLAS_PACKAGE=libcublas-12-4=12.4.2.65-1 2024-08-20T22:20:00.1777426Z NUM_TEST_SHARDS=2 2024-08-20T22:20:00.1777728Z UCX_HOME=/usr 2024-08-20T22:20:00.1778242Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-4=12.4.0-1 2024-08-20T22:20:00.1779429Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_8ab20202-3186-4adb-a884-108604e7b639 2024-08-20T22:20:00.1780660Z JOB_NAME=cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:20:00.1781888Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_8ab20202-3186-4adb-a884-108604e7b639 2024-08-20T22:20:00.1782995Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:20:00.1783650Z GITHUB_EVENT_NAME=push 2024-08-20T22:20:00.1783979Z DASHBOARD_TAG= 2024-08-20T22:20:00.1784280Z GITHUB_RUN_ID=10479310725 2024-08-20T22:20:00.1784746Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-4=12.2.5.2-1 2024-08-20T22:20:00.1785283Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-4 2024-08-20T22:20:00.1786279Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_8ab20202-3186-4adb-a884-108604e7b639 2024-08-20T22:20:00.1787182Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:20:00.1787588Z NV_LIBNPP_DEV_VERSION=12.2.5.2-1 2024-08-20T22:20:00.1787963Z PR_NUMBER= 2024-08-20T22:20:00.1788248Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:20:00.1788559Z VALGRIND=ON 2024-08-20T22:20:00.1788860Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:20:00.1789308Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:20:00.1789769Z TERM=vt100 2024-08-20T22:20:00.1790106Z NV_LIBCUSPARSE_DEV_VERSION=12.3.0.142-1 2024-08-20T22:20:00.1790522Z INSTALLED_VISION=yes 2024-08-20T22:20:00.1790845Z BRANCH= 2024-08-20T22:20:00.1791178Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:20:00.1791573Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:20:00.1792002Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:20:00.1792758Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:20:00.1793502Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:20:00.1793986Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:20:00.1794451Z REENABLED_ISSUES= 2024-08-20T22:20:00.1794747Z SHLVL=1 2024-08-20T22:20:00.1795001Z MAX_JOBS=14 2024-08-20T22:20:00.1795325Z NV_CUDA_LIB_VERSION=12.4.0-1 2024-08-20T22:20:00.1795676Z NVARCH=x86_64 2024-08-20T22:20:00.1795974Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:20:00.1796526Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:20:00.1797064Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:20:00.1797558Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4 2024-08-20T22:20:00.1798216Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:20:00.1798787Z GITHUB_JOB=test 2024-08-20T22:20:00.1799189Z NV_LIBNCCL_PACKAGE=libnccl2=2.20.5-1+cuda12.4 2024-08-20T22:20:00.1799750Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:20:00.1800258Z NO_TEST_TIMEOUT=False 2024-08-20T22:20:00.1800607Z TD_DISTRIBUTED=False 2024-08-20T22:20:00.1801039Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.4.0-1 2024-08-20T22:20:00.1801477Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:20:00.1801892Z NV_NVPROF_VERSION=12.4.99-1 2024-08-20T22:20:00.1802256Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:20:00.1802603Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:20:00.1802957Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:20:00.1803996Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:20:00.1805069Z GITHUB_BASE_REF= 2024-08-20T22:20:00.1805399Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:20:00.1805766Z CI=true 2024-08-20T22:20:00.1806080Z NV_LIBNCCL_PACKAGE_VERSION=2.20.5-1 2024-08-20T22:20:00.1806498Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:20:00.1806968Z JOB_ID=29026426405 2024-08-20T22:20:00.1807277Z INSTALLED_PROTOBUF=yes 2024-08-20T22:20:00.1807607Z GITHUB_HEAD_REF= 2024-08-20T22:20:00.1807909Z GITHUB_ACTION_REF= 2024-08-20T22:20:00.1808347Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:20:00.1808809Z TEST_SHOWLOCALS=False 2024-08-20T22:20:00.1809142Z GITHUB_WORKFLOW=inductor 2024-08-20T22:20:00.1809500Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:20:00.1810398Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_8ab20202-3186-4adb-a884-108604e7b639 2024-08-20T22:20:00.1811205Z NO_TD=False 2024-08-20T22:20:00.1811513Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:20:00.1811874Z _=/usr/bin/env 2024-08-20T22:20:00.1812211Z + echo 'Testing pytorch' 2024-08-20T22:20:00.1812556Z Testing pytorch 2024-08-20T22:20:00.1812884Z + export LANG=C.UTF-8 2024-08-20T22:20:00.1813223Z + LANG=C.UTF-8 2024-08-20T22:20:00.1813521Z + PR_NUMBER= 2024-08-20T22:20:00.1813845Z + [[ inductor_timm == \d\e\f\a\u\l\t ]] 2024-08-20T22:20:00.1814313Z + [[ inductor_timm == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-20T22:20:00.1814771Z + [[ inductor_timm == \s\l\o\w ]] 2024-08-20T22:20:00.1815370Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-08-20T22:20:00.1816071Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *cuda* ]] 2024-08-20T22:20:00.1816617Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:20:00.1817090Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:20:00.1817520Z + [[ inductor_timm == *crossref* ]] 2024-08-20T22:20:00.1818067Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:20:00.1818812Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *xpu* ]] 2024-08-20T22:20:00.1819470Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:20:00.1820035Z + pip_install --user ninja==1.10.2 2024-08-20T22:20:00.1820580Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-20T22:20:01.4278101Z Collecting ninja==1.10.2 2024-08-20T22:20:01.4456754Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-20T22:20:01.4973765Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-20T22:20:02.0932189Z Installing collected packages: ninja 2024-08-20T22:20:02.1011605Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-20T22:20:02.1012975Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-20T22:20:02.1474621Z Successfully installed ninja-1.10.2 2024-08-20T22:20:02.2274981Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:20:02.2277159Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:20:02.2278773Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:20:02.2279293Z + install_tlparse 2024-08-20T22:20:02.2279693Z + pip_install --user tlparse==0.3.25 2024-08-20T22:20:02.2280275Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-20T22:20:02.6481013Z Collecting tlparse==0.3.25 2024-08-20T22:20:02.6665366Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-20T22:20:02.7140349Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-20T22:20:03.3522136Z Installing collected packages: tlparse 2024-08-20T22:20:03.4264391Z Successfully installed tlparse-0.3.25 2024-08-20T22:20:03.5075437Z ++ python -m site --user-base 2024-08-20T22:20:03.5323530Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:20:03.5325730Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *asan* ]] 2024-08-20T22:20:03.5326514Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *-debug* ]] 2024-08-20T22:20:03.5327308Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:20:03.5328332Z + echo 'We are not in debug mode: linux-focal-cuda12.4-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-08-20T22:20:03.5329505Z We are not in debug mode: linux-focal-cuda12.4-py3.10-gcc9-sm86. Expect the assertion to pass 2024-08-20T22:20:03.5330220Z + cd test 2024-08-20T22:20:03.5330776Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-20T22:20:05.1703481Z + [[ inductor_timm == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-20T22:20:05.1704057Z + [[ inductor_timm == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-20T22:20:05.1707526Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-20T22:20:05.1708188Z + [[ inductor_timm == *pr_time_benchmarks* ]] 2024-08-20T22:20:05.1708827Z + [[ inductor_timm == *dynamo_eager* ]] 2024-08-20T22:20:05.1709346Z + [[ inductor_timm == *aot_eager* ]] 2024-08-20T22:20:05.1709808Z + [[ inductor_timm == *aot_inductor* ]] 2024-08-20T22:20:05.1710282Z + [[ inductor_timm == *inductor* ]] 2024-08-20T22:20:05.1710755Z + [[ inductor_timm != *perf* ]] 2024-08-20T22:20:05.1711416Z + DYNAMO_BENCHMARK_FLAGS+=(--inductor) 2024-08-20T22:20:05.1711908Z + [[ inductor_timm == *dynamic* ]] 2024-08-20T22:20:05.1712330Z + [[ inductor_timm == *cpu* ]] 2024-08-20T22:20:05.1712858Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-20T22:20:05.1741128Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:20:05.1741826Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-08-20T22:20:05.1744140Z + cd test 2024-08-20T22:20:05.1744654Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-20T22:20:06.6483330Z PyTorch built with: 2024-08-20T22:20:06.6484132Z - GCC 9.4 2024-08-20T22:20:06.6484568Z - C++ Version: 201703 2024-08-20T22:20:06.6485546Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:20:06.6486647Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:20:06.6487331Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:20:06.6487861Z - LAPACK is enabled (usually provided by MKL) 2024-08-20T22:20:06.6488358Z - NNPACK is enabled 2024-08-20T22:20:06.6488749Z - CPU capability usage: AVX2 2024-08-20T22:20:06.6489174Z - CUDA Runtime 12.4 2024-08-20T22:20:06.6490082Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-08-20T22:20:06.6490660Z - CuDNN 90.1 2024-08-20T22:20:06.6490990Z - Magma 2.6.1 2024-08-20T22:20:06.6497868Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.4, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-20T22:20:06.6504650Z 2024-08-20T22:20:06.9715061Z + cd test 2024-08-20T22:20:06.9715805Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-20T22:20:08.3157152Z ATen/Parallel: 2024-08-20T22:20:08.3157684Z at::get_num_threads() : 8 2024-08-20T22:20:08.3158215Z at::get_num_interop_threads() : 16 2024-08-20T22:20:08.3158704Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:20:08.3159106Z omp_get_max_threads() : 8 2024-08-20T22:20:08.3160172Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:20:08.3161000Z mkl_get_max_threads() : 8 2024-08-20T22:20:08.3161629Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:20:08.3162256Z std::thread::hardware_concurrency() : 16 2024-08-20T22:20:08.3162691Z Environment variables: 2024-08-20T22:20:08.3163045Z OMP_NUM_THREADS : [not set] 2024-08-20T22:20:08.3163423Z MKL_NUM_THREADS : [not set] 2024-08-20T22:20:08.3163810Z ATen parallel backend: OpenMP 2024-08-20T22:20:08.3164071Z 2024-08-20T22:20:08.5968913Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:20:08.5969683Z + [[ inductor_timm == *backward* ]] 2024-08-20T22:20:08.5970113Z + [[ inductor_timm == *xla* ]] 2024-08-20T22:20:08.5970508Z + [[ inductor_timm == *executorch* ]] 2024-08-20T22:20:08.5970986Z + [[ inductor_timm == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-20T22:20:08.5971857Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:20:08.5972451Z + [[ inductor_timm == distributed ]] 2024-08-20T22:20:08.5972970Z + [[ inductor_timm == *inductor_distributed* ]] 2024-08-20T22:20:08.5973559Z + [[ inductor_timm == *inductor-halide* ]] 2024-08-20T22:20:08.5974180Z + [[ inductor_timm == *inductor-micro-benchmark* ]] 2024-08-20T22:20:08.5974671Z + [[ inductor_timm == *huggingface* ]] 2024-08-20T22:20:08.5975098Z + [[ inductor_timm == *timm* ]] 2024-08-20T22:20:08.5975477Z + install_torchvision 2024-08-20T22:20:08.5975820Z + local orig_preload 2024-08-20T22:20:08.5976144Z + local commit 2024-08-20T22:20:08.5976451Z ++ get_pinned_commit vision 2024-08-20T22:20:08.5976840Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-20T22:20:08.5990607Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:08.5991084Z + orig_preload= 2024-08-20T22:20:08.5991521Z + '[' -n '' ']' 2024-08-20T22:20:08.5992363Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:08.5994178Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:08.9332647Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:08.9336940Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-5asi1b2n 2024-08-20T22:20:08.9367873Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-5asi1b2n 2024-08-20T22:20:10.4908098Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-20T22:20:10.4936680Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:11.8187344Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:12.1167839Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:14.5965929Z Preparing metadata (setup.py) ... [?25l- \ done 2024-08-20T22:20:14.5998634Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.22.4) 2024-08-20T22:20:14.6003065Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git40ec5f6) 2024-08-20T22:20:14.6007554Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-20T22:20:14.6080046Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-20T22:20:14.6084083Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-20T22:20:14.6086946Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-20T22:20:14.6090108Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-20T22:20:14.6093395Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-20T22:20:14.6098807Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.1) 2024-08-20T22:20:14.6113410Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-20T22:20:14.6588407Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-20T22:20:14.6651131Z Building wheels for collected packages: torchvision 2024-08-20T22:21:29.1007076Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / done 2024-08-20T22:21:29.1039829Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2025106 sha256=24990982cec169bc473584a7515532ba66a59150a6be0eb76cd6aded7b396e15 2024-08-20T22:21:29.1042227Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-08-20T22:21:29.1076594Z Successfully built torchvision 2024-08-20T22:21:29.5846535Z Installing collected packages: torchvision 2024-08-20T22:21:29.9961928Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-20T22:21:30.1358372Z + '[' -n '' ']' 2024-08-20T22:21:30.1358837Z + id=1 2024-08-20T22:21:30.1359746Z + test_dynamo_benchmark timm_models 1 2024-08-20T22:21:30.1360520Z ++ pwd 2024-08-20T22:21:30.1364048Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:21:30.1364817Z + local suite=timm_models 2024-08-20T22:21:30.1365264Z + shift 2024-08-20T22:21:30.1365546Z + local shard_id=1 2024-08-20T22:21:30.1365853Z + shift 2024-08-20T22:21:30.1366160Z + [[ inductor_timm == *perf_compare* ]] 2024-08-20T22:21:30.1366611Z + [[ inductor_timm == *perf* ]] 2024-08-20T22:21:30.1367057Z + [[ inductor_timm == *cpu* ]] 2024-08-20T22:21:30.1367454Z + [[ inductor_timm == *aot_inductor* ]] 2024-08-20T22:21:30.1368156Z + test_single_dynamo_benchmark inference timm_models 1 --inference --bfloat16 2024-08-20T22:21:30.1370179Z ++ pwd 2024-08-20T22:21:30.1373607Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:21:30.1374296Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:21:30.1398500Z + local name=inference 2024-08-20T22:21:30.1398889Z + shift 2024-08-20T22:21:30.1399280Z + local suite=timm_models 2024-08-20T22:21:30.1399740Z + shift 2024-08-20T22:21:30.1400086Z + local shard_id=1 2024-08-20T22:21:30.1400386Z + shift 2024-08-20T22:21:30.1400658Z + partition_flags=() 2024-08-20T22:21:30.1400993Z + local partition_flags 2024-08-20T22:21:30.1401379Z + [[ -n 2 ]] 2024-08-20T22:21:30.1401697Z + [[ -n 1 ]] 2024-08-20T22:21:30.1402301Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-08-20T22:21:30.1403368Z + [[ inductor_timm == *perf_compare* ]] 2024-08-20T22:21:30.1403822Z + [[ inductor_timm == *perf* ]] 2024-08-20T22:21:30.1404229Z + [[ inductor_timm == *aot_inductor* ]] 2024-08-20T22:21:30.1404661Z + [[ inductor_timm == *_avx2* ]] 2024-08-20T22:21:30.1405058Z + [[ inductor_timm == *_avx512* ]] 2024-08-20T22:21:30.1406637Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --inductor --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 1 --output /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv 2024-08-20T22:21:36.5629666Z 2024-08-20T22:21:36.7106944Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:21:36.7107302Z 2024-08-20T22:21:36.7945630Z model.safetensors: 0% 0.00/29.6M [00:00 will be ignored 2024-08-20T22:45:20.3103527Z pass 2024-08-20T22:45:20.3151215Z TIMING: entire_frame_compile:69.59658 code_gen:30.60324 inductor_compile:48.59121 backend_compile:54.53665 total_wall_time:80.7495 2024-08-20T22:45:20.3152636Z STATS: call_* op count: 700 | FakeTensorMode.__torch_dispatch__:44300 | FakeTensor.__torch_dispatch__:16141 | ProxyTorchDispatchMode.__torch_dispatch__:17697 2024-08-20T22:45:20.3154139Z Dynamo produced 4 graphs covering 700 ops with 7 graph breaks (5 unique) 2024-08-20T22:45:27.5912582Z 2024-08-20T22:45:28.5241847Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:45:28.5242342Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:45:28.5242797Z cuda train mobilenetv2_100 2024-08-20T22:46:14.4361053Z W0820 22:46:14.435000 16045 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:46:54.6106366Z pass 2024-08-20T22:46:54.6152576Z TIMING: entire_frame_compile:67.60488 code_gen:29.7638 inductor_compile:48.64209 backend_compile:52.7677 total_wall_time:80.46176 2024-08-20T22:46:54.6153971Z STATS: call_* op count: 699 | FakeTensorMode.__torch_dispatch__:45004 | FakeTensor.__torch_dispatch__:17124 | ProxyTorchDispatchMode.__torch_dispatch__:17713 2024-08-20T22:46:54.6155137Z Dynamo produced 3 graphs covering 699 ops with 7 graph breaks (5 unique) 2024-08-20T22:47:01.9388091Z 2024-08-20T22:47:02.9302841Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:47:02.9303484Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:47:02.9304062Z cuda train mobilenetv3_large_100 2024-08-20T22:47:52.9636856Z W0820 22:47:52.962000 16831 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:48:39.6526152Z pass 2024-08-20T22:48:39.6597292Z TIMING: entire_frame_compile:75.65838 code_gen:35.47505 inductor_compile:55.88379 backend_compile:59.44121 total_wall_time:90.76551 2024-08-20T22:48:39.6599900Z STATS: call_* op count: 775 | FakeTensorMode.__torch_dispatch__:48411 | FakeTensor.__torch_dispatch__:18376 | ProxyTorchDispatchMode.__torch_dispatch__:19569 2024-08-20T22:48:39.6601976Z Dynamo produced 3 graphs covering 775 ops with 7 graph breaks (5 unique) 2024-08-20T22:48:47.4386275Z 2024-08-20T22:48:48.4024373Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:48:48.4025322Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:48:48.4026188Z cuda train mobilevit_s 2024-08-20T22:50:04.7952240Z pass 2024-08-20T22:50:04.8035656Z TIMING: entire_frame_compile:37.76565 code_gen:36.0958 inductor_compile:52.2869 backend_compile:29.52703 total_wall_time:70.31671 2024-08-20T22:50:04.8037062Z STATS: call_* op count: 363 | FakeTensorMode.__torch_dispatch__:39430 | FakeTensor.__torch_dispatch__:10260 | ProxyTorchDispatchMode.__torch_dispatch__:16472 2024-08-20T22:50:04.8038537Z Dynamo produced 3 graphs covering 363 ops with 6 graph breaks (5 unique) 2024-08-20T22:50:10.9619384Z 2024-08-20T22:50:12.3148976Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:50:12.3149704Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:50:12.3150396Z cuda train nfnet_l0 2024-08-20T22:50:59.5248110Z W0820 22:50:59.523000 19086 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:52:00.9362825Z pass 2024-08-20T22:52:00.9602091Z TIMING: entire_frame_compile:85.52329 code_gen:33.92527 inductor_compile:55.38332 backend_compile:63.51744 total_wall_time:101.21789 2024-08-20T22:52:00.9603556Z STATS: call_* op count: 1205 | FakeTensorMode.__torch_dispatch__:60165 | FakeTensor.__torch_dispatch__:20670 | ProxyTorchDispatchMode.__torch_dispatch__:25953 2024-08-20T22:52:00.9605180Z Dynamo produced 3 graphs covering 1205 ops with 7 graph breaks (5 unique) 2024-08-20T22:52:08.8779754Z 2024-08-20T22:52:10.0417266Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:52:10.0417765Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:52:10.0418219Z cuda train pit_b_224 2024-08-20T22:52:45.1885765Z pass 2024-08-20T22:52:45.2379639Z TIMING: entire_frame_compile:17.97238 code_gen:11.38685 inductor_compile:18.80072 backend_compile:13.57943 total_wall_time:29.30056 2024-08-20T22:52:45.2381428Z STATS: call_* op count: 310 | FakeTensorMode.__torch_dispatch__:28801 | FakeTensor.__torch_dispatch__:6181 | ProxyTorchDispatchMode.__torch_dispatch__:12561 2024-08-20T22:52:45.2382548Z Dynamo produced 2 graphs covering 310 ops with 6 graph breaks (5 unique) 2024-08-20T22:52:50.3216615Z 2024-08-20T22:52:52.1417227Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:52:52.1417740Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:52:52.1421347Z cuda train pnasnet5large 2024-08-20T22:58:12.2693114Z pass 2024-08-20T22:58:12.4130049Z TIMING: entire_frame_compile:147.97027 code_gen:185.89618 inductor_compile:228.54899 backend_compile:112.55365 total_wall_time:299.77871 2024-08-20T22:58:12.4131526Z STATS: call_* op count: 1161 | FakeTensorMode.__torch_dispatch__:129488 | FakeTensor.__torch_dispatch__:34521 | ProxyTorchDispatchMode.__torch_dispatch__:47606 2024-08-20T22:58:12.4134217Z Dynamo produced 2 graphs covering 1161 ops with 5 graph breaks (4 unique) 2024-08-20T22:58:25.9352612Z 2024-08-20T22:58:27.4714333Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:58:27.4714963Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:58:27.4715472Z cuda train poolformer_m36 2024-08-20T23:02:07.8888988Z pass 2024-08-20T23:02:07.9292652Z TIMING: entire_frame_compile:62.12064 code_gen:164.68455 inductor_compile:186.37936 backend_compile:51.36171 total_wall_time:211.73149 2024-08-20T23:02:07.9294830Z STATS: call_* op count: 554 | FakeTensorMode.__torch_dispatch__:50540 | ProxyTorchDispatchMode.__torch_dispatch__:22827 | FakeTensor.__torch_dispatch__:15814 2024-08-20T23:02:07.9296550Z Dynamo produced 2 graphs covering 554 ops with 6 graph breaks (5 unique) 2024-08-20T23:02:16.4090230Z 2024-08-20T23:02:17.3641279Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:02:17.3641879Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:02:17.3645224Z cuda train regnety_002 2024-08-20T23:03:00.9541594Z pass 2024-08-20T23:03:00.9578563Z TIMING: entire_frame_compile:28.92904 code_gen:13.83034 inductor_compile:23.37085 backend_compile:20.19933 total_wall_time:40.39178 2024-08-20T23:03:00.9579984Z STATS: call_* op count: 271 | FakeTensorMode.__torch_dispatch__:29920 | FakeTensor.__torch_dispatch__:8391 | ProxyTorchDispatchMode.__torch_dispatch__:12209 2024-08-20T23:03:00.9581119Z Dynamo produced 2 graphs covering 271 ops with 6 graph breaks (5 unique) 2024-08-20T23:03:06.6443161Z 2024-08-20T23:03:07.8793451Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:03:07.8793939Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:03:07.8794389Z cuda train repvgg_a2 2024-08-20T23:03:55.1925052Z W0820 23:03:55.191000 24188 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:04:38.2135803Z pass 2024-08-20T23:04:38.2302972Z TIMING: entire_frame_compile:72.38409 code_gen:30.80825 inductor_compile:49.90979 backend_compile:56.22793 total_wall_time:83.21096 2024-08-20T23:04:38.2304715Z STATS: call_* op count: 756 | FakeTensorMode.__torch_dispatch__:46838 | FakeTensor.__torch_dispatch__:17199 | ProxyTorchDispatchMode.__torch_dispatch__:18975 2024-08-20T23:04:38.2306140Z Dynamo produced 3 graphs covering 756 ops with 7 graph breaks (5 unique) 2024-08-20T23:04:45.7158732Z 2024-08-20T23:04:47.3056039Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:04:47.3057001Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:04:47.3057873Z cuda train res2net101_26w_4s 2024-08-20T23:07:26.5472871Z pass 2024-08-20T23:07:26.5813308Z TIMING: entire_frame_compile:101.76612 code_gen:62.44654 inductor_compile:97.39462 backend_compile:79.34839 total_wall_time:147.656 2024-08-20T23:07:26.5814907Z STATS: call_* op count: 977 | FakeTensorMode.__torch_dispatch__:100624 | FakeTensor.__torch_dispatch__:30426 | ProxyTorchDispatchMode.__torch_dispatch__:37834 2024-08-20T23:07:26.5816181Z Dynamo produced 2 graphs covering 977 ops with 6 graph breaks (5 unique) 2024-08-20T23:07:37.5414329Z 2024-08-20T23:07:38.8766941Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:07:38.8767436Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:07:38.8767881Z cuda train res2net50_14w_8s 2024-08-20T23:10:06.0160422Z pass 2024-08-20T23:10:06.0364049Z TIMING: entire_frame_compile:93.39617 code_gen:60.97652 inductor_compile:91.63512 backend_compile:72.49395 total_wall_time:138.00919 2024-08-20T23:10:06.0366856Z STATS: call_* op count: 852 | FakeTensorMode.__torch_dispatch__:90422 | FakeTensor.__torch_dispatch__:26327 | ProxyTorchDispatchMode.__torch_dispatch__:32715 2024-08-20T23:10:06.0369125Z Dynamo produced 2 graphs covering 852 ops with 6 graph breaks (5 unique) 2024-08-20T23:10:15.9941602Z 2024-08-20T23:10:17.1909209Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:10:17.1909690Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:10:17.1910140Z cuda train res2next50 2024-08-20T23:11:33.9474224Z pass 2024-08-20T23:11:33.9651585Z TIMING: entire_frame_compile:46.45582 code_gen:28.75914 inductor_compile:45.53822 backend_compile:34.45179 total_wall_time:71.21479 2024-08-20T23:11:33.9653206Z STATS: call_* op count: 484 | FakeTensorMode.__torch_dispatch__:49960 | FakeTensor.__torch_dispatch__:15105 | ProxyTorchDispatchMode.__torch_dispatch__:18879 2024-08-20T23:11:33.9654489Z Dynamo produced 2 graphs covering 484 ops with 6 graph breaks (5 unique) 2024-08-20T23:11:40.8040730Z 2024-08-20T23:11:41.5832908Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:11:41.5833400Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:11:41.5833846Z cuda train resmlp_12_224 2024-08-20T23:12:04.0967388Z pass 2024-08-20T23:12:04.1095271Z TIMING: entire_frame_compile:12.59109 code_gen:6.70145 inductor_compile:11.12915 backend_compile:9.18988 total_wall_time:19.60607 2024-08-20T23:12:04.1098050Z STATS: call_* op count: 179 | FakeTensorMode.__torch_dispatch__:18242 | ProxyTorchDispatchMode.__torch_dispatch__:8345 | FakeTensor.__torch_dispatch__:3939 2024-08-20T23:12:04.1100405Z Dynamo produced 2 graphs covering 179 ops with 6 graph breaks (5 unique) 2024-08-20T23:12:08.8584468Z 2024-08-20T23:12:10.5615572Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:12:10.5616052Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:12:10.5618939Z cuda train resnest101e 2024-08-20T23:14:28.6883257Z pass 2024-08-20T23:14:28.7296390Z TIMING: entire_frame_compile:83.31643 code_gen:47.39741 inductor_compile:79.42119 backend_compile:61.77834 total_wall_time:126.61495 2024-08-20T23:14:28.7298594Z STATS: call_* op count: 1028 | FakeTensorMode.__torch_dispatch__:94317 | FakeTensor.__torch_dispatch__:26591 | ProxyTorchDispatchMode.__torch_dispatch__:38715 2024-08-20T23:14:28.7300013Z Dynamo produced 2 graphs covering 1028 ops with 6 graph breaks (5 unique) 2024-08-20T23:14:38.1959422Z 2024-08-20T23:14:39.3665861Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:14:39.3666469Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:14:39.3666922Z cuda train rexnet_100 2024-08-20T23:15:51.0551373Z W0820 23:15:51.054000 29092 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:16:51.5417483Z pass 2024-08-20T23:16:51.5499444Z TIMING: entire_frame_compile:101.35575 code_gen:50.70182 inductor_compile:78.68637 backend_compile:79.31309 total_wall_time:124.40401 2024-08-20T23:16:51.5501236Z STATS: call_* op count: 1034 | FakeTensorMode.__torch_dispatch__:64940 | FakeTensor.__torch_dispatch__:24077 | ProxyTorchDispatchMode.__torch_dispatch__:26051 2024-08-20T23:16:51.5502587Z Dynamo produced 3 graphs covering 1034 ops with 7 graph breaks (5 unique) 2024-08-20T23:17:00.6756705Z 2024-08-20T23:17:01.8639117Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:17:01.8639717Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:17:01.8641886Z cuda train sebotnet33ts_256 2024-08-20T23:18:08.7800444Z pass 2024-08-20T23:18:08.7927257Z TIMING: entire_frame_compile:39.41093 code_gen:28.66858 inductor_compile:41.76106 backend_compile:31.96317 total_wall_time:60.76103 2024-08-20T23:18:08.7930726Z STATS: call_* op count: 372 | FakeTensorMode.__torch_dispatch__:34241 | FakeTensor.__torch_dispatch__:9073 | ProxyTorchDispatchMode.__torch_dispatch__:13430 2024-08-20T23:18:08.7932990Z Dynamo produced 3 graphs covering 372 ops with 6 graph breaks (5 unique) 2024-08-20T23:18:14.8727597Z 2024-08-20T23:18:16.1602863Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:18:16.1603560Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:18:16.1604258Z cuda train selecsls42b 2024-08-20T23:18:50.9545354Z pass 2024-08-20T23:18:50.9816562Z TIMING: entire_frame_compile:22.04689 code_gen:11.12758 inductor_compile:19.71411 backend_compile:16.60193 total_wall_time:31.25167 2024-08-20T23:18:50.9818648Z STATS: call_* op count: 178 | FakeTensorMode.__torch_dispatch__:22496 | FakeTensor.__torch_dispatch__:6628 | ProxyTorchDispatchMode.__torch_dispatch__:8917 2024-08-20T23:18:50.9820427Z Dynamo produced 2 graphs covering 178 ops with 6 graph breaks (5 unique) 2024-08-20T23:18:56.2814611Z 2024-08-20T23:18:57.2614787Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:18:57.2615409Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:18:57.2617190Z cuda train spnasnet_100 2024-08-20T23:19:49.7840865Z W0820 23:19:49.783000 33499 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:20:40.0541238Z pass 2024-08-20T23:20:40.0581246Z TIMING: entire_frame_compile:83.10119 code_gen:35.19306 inductor_compile:57.12545 backend_compile:64.89846 total_wall_time:96.32598 2024-08-20T23:20:40.0582692Z STATS: call_* op count: 855 | FakeTensorMode.__torch_dispatch__:54124 | FakeTensor.__torch_dispatch__:19775 | ProxyTorchDispatchMode.__torch_dispatch__:21710 2024-08-20T23:20:40.0583828Z Dynamo produced 3 graphs covering 855 ops with 7 graph breaks (5 unique) 2024-08-20T23:20:48.0936420Z 2024-08-20T23:20:50.0712795Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:20:50.0713548Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:20:50.0714239Z cuda train swin_base_patch4_window7_224 2024-08-20T23:22:30.0789334Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-20T23:22:30.0790501Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 404, in torch_dynamo_resume_in_forward_and_backward_pass_at_402 2024-08-20T23:22:30.0792253Z pred = mod(*cloned_inputs) 2024-08-20T23:22:30.0793513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 838, in forward 2024-08-20T23:22:30.0794856Z x = self.forward_features(x) 2024-08-20T23:22:30.0795951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 830, in forward_features 2024-08-20T23:22:30.0797073Z x = self.layers(x) 2024-08-20T23:22:30.0798201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 559, in forward 2024-08-20T23:22:30.0799123Z x = self.blocks(x) 2024-08-20T23:22:30.0800097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 406, in forward 2024-08-20T23:22:30.0800989Z x = x + self.drop_path1(self._attn(self.norm1(x))) 2024-08-20T23:22:30.0801966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 390, in _attn 2024-08-20T23:22:30.0803001Z attn_windows = self.attn(x_windows, mask=attn_mask) # nW*B, window_size*window_size, C 2024-08-20T23:22:30.0804209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 194, in forward 2024-08-20T23:22:30.0805048Z attn = attn + self._get_rel_pos_bias() 2024-08-20T23:22:30.0806038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 165, in _get_rel_pos_bias 2024-08-20T23:22:30.0807020Z relative_position_bias = self.relative_position_bias_table[ 2024-08-20T23:22:30.0807687Z 2024-08-20T23:22:30.6067322Z W0820 23:22:30.605000 33827 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:23:48.9014277Z pass 2024-08-20T23:23:48.9431502Z TIMING: entire_frame_compile:132.54542 code_gen:66.38051 inductor_compile:102.11959 backend_compile:107.59006 total_wall_time:166.82803 2024-08-20T23:23:48.9434187Z STATS: call_* op count: 2446 | FakeTensorMode.__torch_dispatch__:115807 | FakeTensor.__torch_dispatch__:29780 | ProxyTorchDispatchMode.__torch_dispatch__:50142 2024-08-20T23:23:48.9435414Z Dynamo produced 3 graphs covering 2446 ops with 7 graph breaks (5 unique) 2024-08-20T23:23:59.2549711Z 2024-08-20T23:24:02.5441426Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:24:02.5442335Z loading model: 0it [00:03, ?it/s] 2024-08-20T23:24:02.5443217Z cuda train swsl_resnext101_32x16d 2024-08-20T23:25:25.4940135Z pass 2024-08-20T23:25:25.6309392Z TIMING: entire_frame_compile:49.92686 code_gen:21.13437 inductor_compile:38.68566 backend_compile:35.85231 total_wall_time:68.30647 2024-08-20T23:25:25.6310943Z STATS: call_* op count: 453 | FakeTensorMode.__torch_dispatch__:54554 | FakeTensor.__torch_dispatch__:16128 | ProxyTorchDispatchMode.__torch_dispatch__:22001 2024-08-20T23:25:25.6313454Z Dynamo produced 2 graphs covering 453 ops with 6 graph breaks (5 unique) 2024-08-20T23:25:32.5386463Z 2024-08-20T23:25:33.5434207Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:25:33.5434791Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:25:33.5435257Z cuda train tf_efficientnet_b0 2024-08-20T23:26:33.5136702Z pass 2024-08-20T23:26:33.5220919Z TIMING: entire_frame_compile:37.29706 code_gen:23.23777 inductor_compile:36.76926 backend_compile:28.27229 total_wall_time:55.75768 2024-08-20T23:26:33.5222487Z STATS: call_* op count: 297 | FakeTensorMode.__torch_dispatch__:36675 | FakeTensor.__torch_dispatch__:11256 | ProxyTorchDispatchMode.__torch_dispatch__:14649 2024-08-20T23:26:33.5223868Z Dynamo produced 2 graphs covering 297 ops with 6 graph breaks (5 unique) 2024-08-20T23:26:39.9542090Z 2024-08-20T23:26:41.0701343Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:26:41.0701975Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:26:41.0702554Z cuda train tf_mixnet_l 2024-08-20T23:28:06.7771782Z pass 2024-08-20T23:28:06.7863216Z TIMING: entire_frame_compile:54.57025 code_gen:33.96399 inductor_compile:51.51515 backend_compile:39.88721 total_wall_time:80.06962 2024-08-20T23:28:06.7865171Z STATS: call_* op count: 598 | FakeTensorMode.__torch_dispatch__:53845 | FakeTensor.__torch_dispatch__:14221 | ProxyTorchDispatchMode.__torch_dispatch__:19823 2024-08-20T23:28:06.7866329Z Dynamo produced 2 graphs covering 598 ops with 6 graph breaks (5 unique) 2024-08-20T23:28:14.2564701Z 2024-08-20T23:28:15.3271798Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:28:15.3272420Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:28:15.3272928Z cuda train tinynet_a 2024-08-20T23:29:21.7506640Z pass 2024-08-20T23:29:21.7590572Z TIMING: entire_frame_compile:42.06175 code_gen:24.44573 inductor_compile:40.00554 backend_compile:31.54591 total_wall_time:61.96981 2024-08-20T23:29:21.7592161Z STATS: call_* op count: 346 | FakeTensorMode.__torch_dispatch__:43017 | FakeTensor.__torch_dispatch__:13254 | ProxyTorchDispatchMode.__torch_dispatch__:17283 2024-08-20T23:29:21.7593297Z Dynamo produced 2 graphs covering 346 ops with 6 graph breaks (5 unique) 2024-08-20T23:29:28.3056799Z 2024-08-20T23:29:29.3917191Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:29:29.3917697Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:29:29.3918174Z cuda train tnt_s_patch16_224 2024-08-20T23:30:44.8934159Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-20T23:30:44.8935425Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 404, in torch_dynamo_resume_in_forward_and_backward_pass_at_402 2024-08-20T23:30:44.8936504Z pred = mod(*cloned_inputs) 2024-08-20T23:30:44.8938399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/tnt.py", line 335, in forward 2024-08-20T23:30:44.8939454Z x = self.forward_features(x) 2024-08-20T23:30:44.8940587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/tnt.py", line 311, in forward_features 2024-08-20T23:30:44.8941478Z pixel_embed = self.pixel_embed(x, self.pixel_pos) 2024-08-20T23:30:44.8942392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/tnt.py", line 183, in forward 2024-08-20T23:30:44.8943140Z x = self.unfold(x) 2024-08-20T23:30:44.8943360Z 2024-08-20T23:30:45.2164966Z W0820 23:30:45.215000 37990 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:32:10.6418008Z pass 2024-08-20T23:32:10.6666177Z TIMING: entire_frame_compile:126.55334 code_gen:56.94659 inductor_compile:91.9055 backend_compile:104.03443 total_wall_time:152.02178 2024-08-20T23:32:10.6667659Z STATS: call_* op count: 1849 | FakeTensorMode.__torch_dispatch__:109443 | FakeTensor.__torch_dispatch__:30233 | ProxyTorchDispatchMode.__torch_dispatch__:49261 2024-08-20T23:32:10.6668814Z Dynamo produced 3 graphs covering 1849 ops with 7 graph breaks (5 unique) 2024-08-20T23:32:21.3530933Z 2024-08-20T23:32:22.7909227Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:32:22.7909794Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:32:22.7910255Z cuda train twins_pcpvt_base 2024-08-20T23:33:49.1182388Z W0820 23:33:49.117000 38614 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:35:59.9570061Z pass 2024-08-20T23:35:59.9875580Z TIMING: entire_frame_compile:168.79666 code_gen:75.15625 inductor_compile:122.79655 backend_compile:134.85035 total_wall_time:201.36201 2024-08-20T23:35:59.9877208Z STATS: call_* op count: 2432 | FakeTensorMode.__torch_dispatch__:137141 | FakeTensor.__torch_dispatch__:43685 | ProxyTorchDispatchMode.__torch_dispatch__:59071 2024-08-20T23:35:59.9878747Z Dynamo produced 3 graphs covering 2432 ops with 7 graph breaks (5 unique) 2024-08-20T23:36:12.6269341Z 2024-08-20T23:36:13.9892316Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:36:13.9892879Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:36:13.9893335Z cuda train visformer_small 2024-08-20T23:36:50.7867054Z W0820 23:36:50.785000 39562 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:37:24.1187634Z pass 2024-08-20T23:37:24.1441112Z TIMING: entire_frame_compile:56.01434 code_gen:20.56574 inductor_compile:34.23203 backend_compile:43.30013 total_wall_time:64.2884 2024-08-20T23:37:24.1442608Z STATS: call_* op count: 703 | FakeTensorMode.__torch_dispatch__:39704 | FakeTensor.__torch_dispatch__:13021 | ProxyTorchDispatchMode.__torch_dispatch__:15868 2024-08-20T23:37:24.1443747Z Dynamo produced 3 graphs covering 703 ops with 7 graph breaks (5 unique) 2024-08-20T23:37:30.5946515Z 2024-08-20T23:37:32.2254391Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:37:32.2255124Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:37:32.2255839Z cuda train vit_base_patch16_224 2024-08-20T23:38:00.9434526Z W0820 23:38:00.942000 40163 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:38:36.8058874Z pass 2024-08-20T23:38:36.8581613Z TIMING: entire_frame_compile:47.95349 code_gen:20.41323 inductor_compile:33.90858 backend_compile:38.61527 total_wall_time:56.22878 2024-08-20T23:38:36.8583827Z STATS: call_* op count: 740 | FakeTensorMode.__torch_dispatch__:39863 | FakeTensor.__torch_dispatch__:12925 | ProxyTorchDispatchMode.__torch_dispatch__:17807 2024-08-20T23:38:36.8585474Z Dynamo produced 3 graphs covering 740 ops with 7 graph breaks (5 unique) 2024-08-20T23:38:43.1881443Z 2024-08-20T23:38:44.4151500Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:38:44.4152883Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:38:44.4153476Z cuda train volo_d1_224 2024-08-20T23:39:18.9216477Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-20T23:39:18.9217655Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 404, in torch_dynamo_resume_in_forward_and_backward_pass_at_402 2024-08-20T23:39:18.9219827Z pred = mod(*cloned_inputs) 2024-08-20T23:39:18.9221061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 822, in forward 2024-08-20T23:39:18.9221926Z x = self.forward_features(x) 2024-08-20T23:39:18.9222936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 795, in forward_features 2024-08-20T23:39:18.9223845Z x = self.forward_tokens(x) 2024-08-20T23:39:18.9224790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 642, in forward_tokens 2024-08-20T23:39:18.9225651Z x = block(x) 2024-08-20T23:39:18.9226479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 135, in forward 2024-08-20T23:39:18.9227405Z x = x + self.drop_path1(self.attn(self.norm1(x))) 2024-08-20T23:39:18.9228431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 90, in forward 2024-08-20T23:39:18.9229569Z x = F.fold(x, output_size=(H, W), kernel_size=self.kernel_size, padding=self.padding, stride=self.stride) 2024-08-20T23:39:18.9230235Z 2024-08-20T23:39:44.8996890Z W0820 23:39:44.898000 40472 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:40:42.8086766Z pass 2024-08-20T23:40:42.8198230Z TIMING: entire_frame_compile:87.64443 code_gen:44.61409 inductor_compile:68.72204 backend_compile:71.39137 total_wall_time:111.44325 2024-08-20T23:40:42.8199652Z STATS: call_* op count: 1282 | FakeTensorMode.__torch_dispatch__:71999 | FakeTensor.__torch_dispatch__:21124 | ProxyTorchDispatchMode.__torch_dispatch__:32632 2024-08-20T23:40:42.8200818Z Dynamo produced 3 graphs covering 1282 ops with 7 graph breaks (5 unique) 2024-08-20T23:40:51.2412696Z 2024-08-20T23:40:54.7517861Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:40:54.7518360Z loading model: 0it [00:03, ?it/s] 2024-08-20T23:40:54.7520806Z cuda train xcit_large_24_p8_224 2024-08-20T23:40:59.0837728Z WARNING:common:fp64 golden ref were not generated for xcit_large_24_p8_224. Setting accuracy check to cosine 2024-08-20T23:42:58.4558108Z W0820 23:42:58.454000 41106 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:45:39.9441327Z pass_due_to_skip 2024-08-20T23:45:39.9870333Z TIMING: entire_frame_compile:219.50223 code_gen:95.96317 inductor_compile:156.39321 backend_compile:173.47462 total_wall_time:261.36724 2024-08-20T23:45:39.9871918Z STATS: call_* op count: 3042 | FakeTensorMode.__torch_dispatch__:173695 | FakeTensor.__torch_dispatch__:57343 | ProxyTorchDispatchMode.__torch_dispatch__:78438 2024-08-20T23:45:39.9873087Z Dynamo produced 3 graphs covering 3042 ops with 7 graph breaks (5 unique) 2024-08-20T23:45:52.0212636Z accuracy pass_rate=96.77% 2024-08-20T23:45:52.0214390Z calls_captured gmean=691.18x mean=887.871x 2024-08-20T23:45:52.0218647Z unique_graphs gmean=2.49x mean=2.548x 2024-08-20T23:45:52.0222600Z graph_breaks gmean=6.39x mean=6.419x 2024-08-20T23:45:52.0226438Z unique_graph_breaks gmean=4.96x mean=4.968x 2024-08-20T23:45:52.0230316Z autograd_captures gmean=0.00x mean=0.000x 2024-08-20T23:45:52.0234038Z autograd_compiles gmean=0.00x mean=0.000x 2024-08-20T23:45:52.0237758Z cudagraph_skips gmean=0.00x mean=0.097x 2024-08-20T23:45:54.9574222Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_timm_training.csv 2024-08-20T23:45:55.2193565Z mixnet_l PASS 2024-08-20T23:45:55.2196778Z mnasnet_100 PASS 2024-08-20T23:45:55.2201158Z mobilenetv2_100 PASS 2024-08-20T23:45:55.2205766Z mobilenetv3_large_100 PASS 2024-08-20T23:45:55.2210370Z mobilevit_s PASS 2024-08-20T23:45:55.2214947Z nfnet_l0 PASS 2024-08-20T23:45:55.2219577Z pit_b_224 PASS 2024-08-20T23:45:55.2224099Z pnasnet5large PASS 2024-08-20T23:45:55.2228645Z poolformer_m36 PASS 2024-08-20T23:45:55.2233208Z regnety_002 PASS 2024-08-20T23:45:55.2237788Z repvgg_a2 PASS 2024-08-20T23:45:55.2242014Z res2net101_26w_4s PASS 2024-08-20T23:45:55.2246913Z res2net50_14w_8s PASS 2024-08-20T23:45:55.2252554Z res2next50 PASS 2024-08-20T23:45:55.2257023Z resmlp_12_224 PASS 2024-08-20T23:45:55.2261872Z resnest101e PASS 2024-08-20T23:45:55.2266541Z rexnet_100 PASS 2024-08-20T23:45:55.2271079Z sebotnet33ts_256 PASS 2024-08-20T23:45:55.2275900Z selecsls42b PASS 2024-08-20T23:45:55.2280368Z spnasnet_100 PASS 2024-08-20T23:45:55.2285058Z swin_base_patch4_window7_224 PASS 2024-08-20T23:45:55.2289660Z swsl_resnext101_32x16d PASS 2024-08-20T23:45:55.2294301Z tf_efficientnet_b0 PASS 2024-08-20T23:45:55.2298900Z tf_mixnet_l PASS 2024-08-20T23:45:55.2303736Z tinynet_a PASS 2024-08-20T23:45:55.2308268Z tnt_s_patch16_224 PASS 2024-08-20T23:45:55.2312819Z twins_pcpvt_base PASS 2024-08-20T23:45:55.2317341Z visformer_small PASS 2024-08-20T23:45:55.2322025Z vit_base_patch16_224 PASS 2024-08-20T23:45:55.2326645Z volo_d1_224 PASS 2024-08-20T23:45:55.2331185Z xcit_large_24_p8_224 XFAIL 2024-08-20T23:45:55.2748359Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_timm_training.csv 2024-08-20T23:45:55.5406487Z mixnet_l PASS 2024-08-20T23:45:55.5410737Z mnasnet_100 PASS 2024-08-20T23:45:55.5415549Z mobilenetv2_100 PASS 2024-08-20T23:45:55.5420738Z mobilenetv3_large_100 PASS 2024-08-20T23:45:55.5425866Z mobilevit_s PASS 2024-08-20T23:45:55.5430411Z nfnet_l0 PASS 2024-08-20T23:45:55.5435201Z pit_b_224 PASS 2024-08-20T23:45:55.5440178Z pnasnet5large PASS 2024-08-20T23:45:55.5445228Z poolformer_m36 PASS 2024-08-20T23:45:55.5451018Z regnety_002 PASS 2024-08-20T23:45:55.5455656Z repvgg_a2 PASS 2024-08-20T23:45:55.5460945Z res2net101_26w_4s PASS 2024-08-20T23:45:55.5465904Z res2net50_14w_8s PASS 2024-08-20T23:45:55.5470908Z res2next50 PASS 2024-08-20T23:45:55.5475789Z resmlp_12_224 PASS 2024-08-20T23:45:55.5480804Z resnest101e PASS 2024-08-20T23:45:55.5485816Z rexnet_100 PASS 2024-08-20T23:45:55.5490763Z sebotnet33ts_256 PASS 2024-08-20T23:45:55.5495721Z selecsls42b PASS 2024-08-20T23:45:55.5500917Z spnasnet_100 PASS 2024-08-20T23:45:55.5505729Z swin_base_patch4_window7_224 PASS 2024-08-20T23:45:55.5510693Z swsl_resnext101_32x16d PASS 2024-08-20T23:45:55.5515512Z tf_efficientnet_b0 PASS 2024-08-20T23:45:55.5520395Z tf_mixnet_l PASS 2024-08-20T23:45:55.5525394Z tinynet_a PASS 2024-08-20T23:45:55.5530289Z tnt_s_patch16_224 PASS 2024-08-20T23:45:55.5535225Z twins_pcpvt_base PASS 2024-08-20T23:45:55.5540319Z visformer_small PASS 2024-08-20T23:45:55.5545257Z vit_base_patch16_224 PASS 2024-08-20T23:45:55.5550939Z volo_d1_224 PASS 2024-08-20T23:45:55.5555628Z xcit_large_24_p8_224 PASS 2024-08-20T23:45:55.5993436Z + cleanup_workspace 2024-08-20T23:45:55.5994521Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-20T23:45:55.5995770Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-20T23:45:55.5996959Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-20T23:45:55.5997814Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-20T23:45:55.5998676Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-20T23:45:55.5999584Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-20T23:45:55.6000302Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-20T23:45:56.3449147Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-20T23:45:56.3449686Z cat test/**/*_toprint.log || true 2024-08-20T23:45:56.3463069Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:45:56.3463587Z env: 2024-08-20T23:45:56.3463879Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:56.3464347Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:56.3465080Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:56.3465720Z ##[endgroup] 2024-08-20T23:45:56.3560678Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-20T23:45:56.3593741Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:45:56.3594223Z kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:45:56.3602929Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:45:56.3603436Z env: 2024-08-20T23:45:56.3603721Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:56.3604190Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:56.3604914Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:56.3605567Z MONITOR_SCRIPT_PID: 52457 2024-08-20T23:45:56.3605926Z ##[endgroup] 2024-08-20T23:45:56.3804166Z Prepare all required actions 2024-08-20T23:45:56.3804624Z Getting action download info 2024-08-20T23:45:56.5717192Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-20T23:45:56.7336355Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-20T23:45:56.7336842Z with: 2024-08-20T23:45:56.7337391Z file-suffix: test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426405 2024-08-20T23:45:56.7338072Z s3-bucket: gha-artifacts 2024-08-20T23:45:56.7338430Z env: 2024-08-20T23:45:56.7338808Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:56.7339273Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:56.7340009Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:56.7340640Z ##[endgroup] 2024-08-20T23:45:56.7367879Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-20T23:45:56.7368544Z # Remove any previous test jsons if they exist 2024-08-20T23:45:56.7369115Z rm -f test-jsons-*.zip 2024-08-20T23:45:56.7369647Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-20T23:45:56.7379493Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:45:56.7380002Z env: 2024-08-20T23:45:56.7380298Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:56.7380761Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:56.7381499Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:56.7382578Z FILE_SUFFIX: test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426405 2024-08-20T23:45:56.7383242Z ##[endgroup] 2024-08-20T23:45:56.7754854Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-20T23:45:56.7783714Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-20T23:45:56.7784426Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-20T23:45:56.7790802Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-20T23:45:56.7796590Z adding: test/slow_tests.json (deflated 82%) 2024-08-20T23:45:56.7831578Z ##[group]Run # Remove any previous test reports if they exist 2024-08-20T23:45:56.7832225Z # Remove any previous test reports if they exist 2024-08-20T23:45:56.7832757Z rm -f test-reports-*.zip 2024-08-20T23:45:56.7833470Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-20T23:45:56.7842956Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:45:56.7843452Z env: 2024-08-20T23:45:56.7843743Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:56.7844212Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:56.7845023Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:56.7845914Z FILE_SUFFIX: test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426405 2024-08-20T23:45:56.7847022Z ##[endgroup] 2024-08-20T23:45:56.8111184Z adding: test/test-reports/inference_timm_models.csv (deflated 69%) 2024-08-20T23:45:56.8112016Z adding: test/test-reports/training_timm_models.csv (deflated 66%) 2024-08-20T23:45:56.8116055Z adding: test/test-reports/training_timm_model_graph_breaks.csv (deflated 97%) 2024-08-20T23:45:56.8117019Z adding: test/test-reports/training_timm_model_graph_break_deduped.csv (deflated 74%) 2024-08-20T23:45:56.8150417Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-20T23:45:56.8151074Z # Remove any previous usage logs if they exist 2024-08-20T23:45:56.8151585Z rm -f logs-*.zip 2024-08-20T23:45:56.8152260Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-20T23:45:56.8153057Z # so check to see if the file exists first 2024-08-20T23:45:56.8153566Z if [ -f 'usage_log.txt' ]; then 2024-08-20T23:45:56.8154327Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-20T23:45:56.8154826Z fi 2024-08-20T23:45:56.8155196Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-20T23:45:56.8155778Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-20T23:45:56.8156275Z fi 2024-08-20T23:45:56.8165027Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:45:56.8165542Z env: 2024-08-20T23:45:56.8165822Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:56.8166287Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:56.8167038Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:56.8167939Z FILE_SUFFIX: test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426405 2024-08-20T23:45:56.8168591Z ##[endgroup] 2024-08-20T23:45:56.8260924Z adding: usage_log.txt (deflated 92%) 2024-08-20T23:45:56.8356990Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-20T23:45:56.8357966Z # Remove any previous debugging artifacts if they exist 2024-08-20T23:45:56.8358706Z rm -f debug-*.zip 2024-08-20T23:45:56.8359225Z if [ -d 'test/debug' ]; then 2024-08-20T23:45:56.8359900Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-20T23:45:56.8360534Z fi 2024-08-20T23:45:56.8371321Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:45:56.8371982Z env: 2024-08-20T23:45:56.8372349Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:56.8372949Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:56.8374160Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:56.8375353Z FILE_SUFFIX: test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426405 2024-08-20T23:45:56.8376206Z ##[endgroup] 2024-08-20T23:45:56.8473656Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:45:56.8474101Z with: 2024-08-20T23:45:56.8474391Z s3-bucket: gha-artifacts 2024-08-20T23:45:56.8474849Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:45:56.8475331Z retention-days: 14 2024-08-20T23:45:56.8475673Z if-no-files-found: warn 2024-08-20T23:45:56.8476042Z path: test-jsons-*.zip 2024-08-20T23:45:56.8476390Z name: artifact 2024-08-20T23:45:56.8476696Z region: us-east-1 2024-08-20T23:45:56.8477006Z env: 2024-08-20T23:45:56.8477292Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:56.8477749Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:56.8478493Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:56.8479132Z ##[endgroup] 2024-08-20T23:45:57.2029687Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:45:57.2030416Z With the provided path, there will be 1 file uploaded 2024-08-20T23:45:57.2031047Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:45:57.3195335Z Starting upload of test-jsons-test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426405.zip 2024-08-20T23:45:57.4313577Z Finished upload of test-jsons-test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426405.zip 2024-08-20T23:45:57.4592209Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:45:57.4592662Z with: 2024-08-20T23:45:57.4592956Z s3-bucket: gha-artifacts 2024-08-20T23:45:57.4593399Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:45:57.4593886Z retention-days: 14 2024-08-20T23:45:57.4594231Z if-no-files-found: error 2024-08-20T23:45:57.4594624Z path: test-reports-*.zip 2024-08-20T23:45:57.4594985Z name: artifact 2024-08-20T23:45:57.4595291Z region: us-east-1 2024-08-20T23:45:57.4595605Z env: 2024-08-20T23:45:57.4595898Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:57.4596363Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:57.4597097Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:57.4597739Z ##[endgroup] 2024-08-20T23:45:57.8165552Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:45:57.8166285Z With the provided path, there will be 1 file uploaded 2024-08-20T23:45:57.8166965Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:45:57.8223407Z Starting upload of test-reports-test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426405.zip 2024-08-20T23:45:57.8999398Z Finished upload of test-reports-test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426405.zip 2024-08-20T23:45:57.9279835Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:45:57.9280277Z with: 2024-08-20T23:45:57.9280581Z s3-bucket: gha-artifacts 2024-08-20T23:45:57.9281027Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:45:57.9281512Z retention-days: 14 2024-08-20T23:45:57.9281853Z if-no-files-found: ignore 2024-08-20T23:45:57.9282219Z path: logs-*.zip 2024-08-20T23:45:57.9282539Z name: artifact 2024-08-20T23:45:57.9282848Z region: us-east-1 2024-08-20T23:45:57.9283170Z env: 2024-08-20T23:45:57.9283462Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:57.9283929Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:57.9284658Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:57.9285292Z ##[endgroup] 2024-08-20T23:45:58.2813604Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:45:58.2814407Z With the provided path, there will be 1 file uploaded 2024-08-20T23:45:58.2815148Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:45:58.2870548Z Starting upload of logs-test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426405.zip 2024-08-20T23:45:58.3897276Z Finished upload of logs-test-inductor_timm-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426405.zip 2024-08-20T23:45:58.4174654Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:45:58.4175099Z with: 2024-08-20T23:45:58.4175389Z s3-bucket: gha-artifacts 2024-08-20T23:45:58.4175848Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:45:58.4176330Z retention-days: 14 2024-08-20T23:45:58.4176671Z if-no-files-found: ignore 2024-08-20T23:45:58.4177044Z path: debug-*.zip 2024-08-20T23:45:58.4177364Z name: artifact 2024-08-20T23:45:58.4177673Z region: us-east-1 2024-08-20T23:45:58.4177984Z env: 2024-08-20T23:45:58.4178264Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:58.4178809Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:58.4179560Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:58.4180233Z ##[endgroup] 2024-08-20T23:45:58.7630142Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-20T23:45:58.7909015Z ##[group]Run # shellcheck disable=SC2156 2024-08-20T23:45:58.7909510Z # shellcheck disable=SC2156 2024-08-20T23:45:58.7910311Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-20T23:45:58.7919498Z shell: /usr/bin/bash -e {0} 2024-08-20T23:45:58.7919857Z env: 2024-08-20T23:45:58.7920147Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:58.7920614Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:58.7921347Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:58.7921986Z ##[endgroup] 2024-08-20T23:45:59.0562931Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-20T23:45:59.0563523Z with: 2024-08-20T23:45:59.0563791Z env: 2024-08-20T23:45:59.0564081Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:59.0564542Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:59.0565283Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:59.0565922Z ##[endgroup] 2024-08-20T23:45:59.0585508Z ##[group]Run set -eou pipefail 2024-08-20T23:45:59.0585928Z set -eou pipefail 2024-08-20T23:45:59.0586297Z  2024-08-20T23:45:59.0586819Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-20T23:45:59.0587466Z for _ in $(seq 1440); do 2024-08-20T23:45:59.0587949Z  # Break if no ssh session exists anymore 2024-08-20T23:45:59.0588461Z  if [ "$(who)" = "" ]; then 2024-08-20T23:45:59.0588883Z  break 2024-08-20T23:45:59.0589229Z  fi 2024-08-20T23:45:59.0589536Z  echo "." 2024-08-20T23:45:59.0589929Z  sleep 5 2024-08-20T23:45:59.0590257Z done 2024-08-20T23:45:59.0599050Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:45:59.0599558Z env: 2024-08-20T23:45:59.0599850Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:59.0600333Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:59.0601083Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:59.0601720Z ##[endgroup] 2024-08-20T23:45:59.0632136Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-20T23:45:59.0711105Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:45:59.0711887Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:45:59.0712498Z # shellcheck disable=SC2046 2024-08-20T23:45:59.0712994Z docker stop $(docker ps -q) || true 2024-08-20T23:45:59.0713492Z # Prune all of the docker images 2024-08-20T23:45:59.0714135Z docker system prune -af 2024-08-20T23:45:59.0722286Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:45:59.0722800Z env: 2024-08-20T23:45:59.0723086Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:45:59.0723554Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:45:59.0724290Z DOCKER_CONTAINER_ID: 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:45:59.0724929Z ##[endgroup] 2024-08-20T23:45:59.5859641Z 5d65ee9831b2 2024-08-20T23:46:01.0383913Z Deleted Containers: 2024-08-20T23:46:01.0384470Z 5d65ee9831b20e57c3633c336ea168c7b2490b59e83bd03a28c5e063f3827516 2024-08-20T23:46:01.0384898Z 2024-08-20T23:46:12.2044762Z Deleted Images: 2024-08-20T23:46:12.2047091Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T23:46:12.2050069Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks@sha256:4753b1ce6a27c0af5c669dd74e9e16618d0a1a015eb474ef14f6fd51409fc5f7 2024-08-20T23:46:12.2051902Z deleted: sha256:cee8d41a96d3e7eccb178c1529633c2dd768035bec694588cf53029b783ac292 2024-08-20T23:46:12.2052735Z deleted: sha256:387912ed740fe741bc7f78f2754e4da03169e3d9785348eeef65657209e6f778 2024-08-20T23:46:12.2053581Z deleted: sha256:fdabcba8f8fb280e19a152aa25e47e9651627e1f9f1bc23153068fdf8c7d630a 2024-08-20T23:46:12.2054435Z deleted: sha256:5c719bfedda0fa42f3cc857a021fb6d9166275ef47e6b7854bdf79cbd94371a7 2024-08-20T23:46:12.2055273Z deleted: sha256:1a7fa8fef08db4647f773032825124b3c7765761cdb2514100129cd118b53f37 2024-08-20T23:46:12.2056147Z deleted: sha256:f2e444a1c58fb82e012d6be36af6acb9d1f137330877be768e65749c12c6eb7a 2024-08-20T23:46:12.2056971Z deleted: sha256:33704a494ac956cb2ec1430c9126595eaac4741e5d6ae47ee20d64306ac05764 2024-08-20T23:46:12.2058150Z deleted: sha256:6a4ca9ff7195505d66fd28b827e4d52c95309ca34d455fe52bec2a3dfb6bf973 2024-08-20T23:46:12.2059088Z deleted: sha256:32a556656b7c0c0fedabcb4be518df66ae4b707ac63b52e7e5c256a151fd586f 2024-08-20T23:46:12.2059926Z deleted: sha256:21ffd21aadcd3065d2bd133ce57077e37347bb948615c4c71aa475d0fc9ed89b 2024-08-20T23:46:12.2060761Z deleted: sha256:d653a3b9ba44f6be8e819fcbd32a14c4a36ed536a5ef86f8bdc794ea46cef755 2024-08-20T23:46:12.2061601Z deleted: sha256:d345bcdfdea1caa63503188719bb1c5fa135b18bb0a8e816aefe3d2ded5c996a 2024-08-20T23:46:12.2062442Z deleted: sha256:a251f1284cbe4dfb7c5afb0f89272046796e77fb91a71471d89acca62f25a6db 2024-08-20T23:46:12.2063268Z deleted: sha256:f0189edd8227eb15aea7e1bfe18525327b825ee314882ebf3357b5093835dac3 2024-08-20T23:46:12.2064311Z deleted: sha256:5eacbc0fafbd39bf65cadd1e29a52ee9fd4222765cb56fcb878bda0344598406 2024-08-20T23:46:12.2065397Z deleted: sha256:7eddd13a99b3708e985984c95fc0c52cba171ff24a9c0fa45d57cb665d4e221f 2024-08-20T23:46:12.2066350Z deleted: sha256:bc8cac8e2879f26cb90afbbeb7e64ba11581b57648745399021fa45df7c2f7ae 2024-08-20T23:46:12.2067305Z deleted: sha256:57532a5c1bacb2650908c672b75fae5c8781cb7ee701ac1784967e50bfceaae5 2024-08-20T23:46:12.2068137Z deleted: sha256:21509edb6a37130a50af8008f246f00373a5bc735d4d1afa0a3fdbda622fbbbd 2024-08-20T23:46:12.2068961Z deleted: sha256:4db1e29af6ad7a5f28276ef7bdb6ac5a60244101f36d955aa34fb217ea4e45d4 2024-08-20T23:46:12.2069784Z deleted: sha256:5b97f357f15208f34242b48ed3cf670775a8fb58e04eb24ef8c8d9552848a918 2024-08-20T23:46:12.2070595Z deleted: sha256:968988c362f1c9b3b08fec85cf7062d8b29858681660ddfb3afd56840c40e25e 2024-08-20T23:46:12.2071418Z deleted: sha256:67b5029b1c3c8aa88fd34383f8ffc6e5eab1aa392fd01030d946abc467a47b59 2024-08-20T23:46:12.2072246Z deleted: sha256:a30fda61fc6efa33e06722afb4f61b07780cb21ca0668d135bbe551841828c65 2024-08-20T23:46:12.2073087Z deleted: sha256:bafaf280fd9d77b6d6cf03b502768cb7c77f215dad925746be1c31b7a58e48ef 2024-08-20T23:46:12.2073940Z deleted: sha256:2cec74a4397496f01bfa49616ffa41f1cfb9bfd83f333f1eabb5f35e82480699 2024-08-20T23:46:12.2074809Z deleted: sha256:44e532372f95c9294bef0fae8cb8a240437327fe41a7868842b9068423f1e9f2 2024-08-20T23:46:12.2075837Z deleted: sha256:9d3868b219eb14fb39fe0b22202580a15871a0ba15447569a89c096eb9a2cd06 2024-08-20T23:46:12.2076645Z deleted: sha256:033b1e94d1741cdceb2e0052d884cf58a98e7634482a5490385768dd53708747 2024-08-20T23:46:12.2077452Z deleted: sha256:11f9b7ea949d1065d500b6c408f8313c13734d192dbb2f6230e9b63fabe94ebf 2024-08-20T23:46:12.2078271Z deleted: sha256:10f56d39f97b51ff7fe2e2df9b80e1cfde2422660ac722c1d541139dbe1a704c 2024-08-20T23:46:12.2079106Z deleted: sha256:03a7c0c48c9a83e65ba5156403bc1fe60356ab4b1fc7e075f9141603833246f3 2024-08-20T23:46:12.2079930Z deleted: sha256:7fe9c7528f9ef7341782ed285475cb83eccc61ce0ab97d1e7cbae56ed0e01d42 2024-08-20T23:46:12.2080816Z deleted: sha256:0efbe5060cc721d5a732d3e34de1b9dd40573d304803cbd0b2437ca8872ce549 2024-08-20T23:46:12.2081650Z deleted: sha256:e6adbe3e7465b1b08f494e3561c42837c0116bc71ab409a7c62c56328fb9e1ef 2024-08-20T23:46:12.2082480Z deleted: sha256:2058c6a1cea3f868fabc404608ed1d273db9f63b5e1d3211d765e41ae51f8d61 2024-08-20T23:46:12.2083307Z deleted: sha256:33d25e35e9a4f1bfa5d8373c18e9f3449358445c62dc2708bc5036ed14c7e9bd 2024-08-20T23:46:12.2084136Z deleted: sha256:5f854f6a610a0bf5198cefce78fcb96dadf7ce7fdb02bfe6685b326f6cd3406b 2024-08-20T23:46:12.2084948Z deleted: sha256:98634d529353f4e760c15592834942206298353e233a2937a836a73c6b5e4168 2024-08-20T23:46:12.2085728Z deleted: sha256:071ec252097583d609a084e0f43a70395c334fe8d9b3e56a8d97135c49a3c3a5 2024-08-20T23:46:12.2086530Z deleted: sha256:5f818bbf177d0e18a7c5e5ce3470c6f6a2cc11017bed953cd5b78b2b26a35558 2024-08-20T23:46:12.2087366Z deleted: sha256:d2cce52317ffa022e42fdd14dc33292b40b1a273cc139c96d3a206a2173725a4 2024-08-20T23:46:12.2088215Z deleted: sha256:d5ce7ab9e4b134eb4efd1e5712bd0c87df578bdb5be9a1cbf5dcdb583044e9f8 2024-08-20T23:46:12.2089052Z deleted: sha256:7bf36330750a896b5b5581463f0e9cfdcd3920e3c5c20d728ee553c7f8cf6eeb 2024-08-20T23:46:12.2089980Z deleted: sha256:7f743fed415e07aca752031384d8262c5ea4c80e239710986b93e7118f67f1c2 2024-08-20T23:46:12.2090796Z deleted: sha256:aa2f660d8384dd4660a1080d54c22db28b4c8502c92255e97bac64b8172153b7 2024-08-20T23:46:12.2091697Z deleted: sha256:f872fb5e9a43d29fb03685b648ea38679b88586c783e5ec6815bb69895c0103c 2024-08-20T23:46:12.2092515Z deleted: sha256:2b39b70836b2330fe60c8a8e809f97b5deff7e1b3868fa4a7bf0e00e5f6540c4 2024-08-20T23:46:12.2093352Z deleted: sha256:fffd71ce3df39233b4a9d1e1c6a3e78fe0d22539f03b6f5e46ef323be746bb1b 2024-08-20T23:46:12.2094171Z deleted: sha256:8fa7767d8015b2f308374dc2573904630aff8009ae15ac5cd5b99986a9e5d38c 2024-08-20T23:46:12.2094974Z deleted: sha256:6d39ae4f4f81796c79703a115341abc214ea6402b95d7c4481b0860cca5411db 2024-08-20T23:46:12.2095798Z deleted: sha256:d635bd598f9c6aa8a28f9a417861a82ceddd0c28fb9e006ac1bdeffb6da92dc8 2024-08-20T23:46:12.2096621Z deleted: sha256:08458dac742f18d448f988793bd19c27c6129457b8b2ff2743ccdf2b705b86e3 2024-08-20T23:46:12.2097433Z deleted: sha256:7715bcb4ef88525b2b34380f2b81ceae25e87fb53a8c97e99dd319765db02f6f 2024-08-20T23:46:12.2098260Z deleted: sha256:ca9a40d26179de2d38203500cbd75fa2dbcb7da4b0badb3f02861714e1ee0e20 2024-08-20T23:46:12.2099194Z deleted: sha256:5076e56d3cbdded7799738aff3b50afcb3cb8f05b3aa4a9aa4a6e6fe6bb49465 2024-08-20T23:46:12.2100004Z deleted: sha256:c01100978013295b52ecbb0a196c82106506b48935a9c0a1d1dd66432d9b64b7 2024-08-20T23:46:12.2100795Z deleted: sha256:8747f391b7489623fc54b3813b0f79e3474c779d402b53e0c557e9b5b5f8e492 2024-08-20T23:46:12.2101600Z deleted: sha256:987974f3c4e180d9f3086ad4e1df30220d5ea68bafe243c0966d80cb45a0176d 2024-08-20T23:46:12.2102422Z deleted: sha256:1d2d9d89274b1d01c7b76fbc66fccafea8414d85482cb5c43d4b6631a09c1f4e 2024-08-20T23:46:12.2103251Z deleted: sha256:b5e92a2b04c427229eb25b5013dc2062bb64d7f751d3ea26292a7465f351a56b 2024-08-20T23:46:12.2104101Z deleted: sha256:6cf2fd73cc6550401f1b569cadf927d047ae3ff9624ed49a52c37e2f7e664346 2024-08-20T23:46:12.2104962Z deleted: sha256:dc09bad6e139f1dbf77f033a8bb913b50348ff30cca0459a1b65291309712b5a 2024-08-20T23:46:12.2105788Z deleted: sha256:69f8b41d1f6e2a349f9c4292928d43aa3c8750feff87e713ef27c9473db7e4bd 2024-08-20T23:46:12.2106688Z deleted: sha256:a88196e571b2b4c038c05542017cff4cf98568990a3483e4454739f789202a28 2024-08-20T23:46:12.2107511Z deleted: sha256:29f9f89c9d02abbd1cc7ea3a2e67211ffb77fa4fd4a6013dc8ac1328ed390133 2024-08-20T23:46:12.2108339Z deleted: sha256:fd5d6a109c3a5a89ed53f8b23d96494982c39df97ac3220edc4c4e4ba43c9af1 2024-08-20T23:46:12.2109177Z deleted: sha256:14436cfe80ca6ea0d09ba450a071ce28d8bc87fc3602d4c9f8b7cc3b474cbcc5 2024-08-20T23:46:12.2110009Z deleted: sha256:433544e4353c9e0269dc71fa16c00ddcecdcd5676ed0df618c7b51ac78e23053 2024-08-20T23:46:12.2110851Z deleted: sha256:fc24af9fd5121adfbc7b75df0abb1ef8c2d3868f33b7b6b1e8bf4e214fac4bd6 2024-08-20T23:46:12.2111684Z deleted: sha256:ce1492ad94b71204f8b3fb8903d0ad48a96652635e7fc9347e430639ca15a737 2024-08-20T23:46:12.2112492Z deleted: sha256:96f2ae9435e6e12f68a88dc404c8079b2796639589970c34894a86ccff3f3732 2024-08-20T23:46:12.2113319Z deleted: sha256:5310d2bc0952e7c6d8cbade7ecafd88ce3a04d9da303fa6419583e13a02b58a9 2024-08-20T23:46:12.2114166Z deleted: sha256:099076d608a55a027d60f5b572a3991a8d690a01500acf2c696933a7e6d38650 2024-08-20T23:46:12.2114996Z deleted: sha256:862b05d0f8f7e3dff2d04bd49784a9901f4e22c5dd6544ae72eb3be05e23fffe 2024-08-20T23:46:12.2115823Z deleted: sha256:f7a888e5f7904827f6d71cebcba5169dbc7a78b20f228b9b9b625f38e0c52f24 2024-08-20T23:46:12.2116658Z deleted: sha256:ab9f75a4a7a636aae480f630eaddc05133dda64f1c42445b617c97d3d209991c 2024-08-20T23:46:12.2117500Z deleted: sha256:56dda0c9cee64db5c10d5dd5f08c6f6707263f5ee98e8a075d635e5be996e855 2024-08-20T23:46:12.2118341Z deleted: sha256:5faf9c0a9efe4675ecd21a4ec417d51077d5e75da9e673161a94e7d6cd43f92c 2024-08-20T23:46:12.2118848Z 2024-08-20T23:46:12.2118999Z Total reclaimed space: 45.11GB 2024-08-20T23:46:12.2186169Z Post job cleanup. 2024-08-20T23:46:12.2238160Z Post job cleanup. 2024-08-20T23:46:12.3113050Z [command]/usr/bin/git version 2024-08-20T23:46:12.3162296Z git version 2.40.1 2024-08-20T23:46:12.3200720Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/4e5b870b-c20d-4d7d-9801-c9e2a41aa96f' before making global git config changes 2024-08-20T23:46:12.3201990Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T23:46:12.3205710Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T23:46:12.3255102Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T23:46:12.3299564Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T23:46:12.3669446Z Entering 'android/libs/fbjni' 2024-08-20T23:46:12.3737898Z Entering 'third_party/FP16' 2024-08-20T23:46:12.3808182Z Entering 'third_party/FXdiv' 2024-08-20T23:46:12.3876138Z Entering 'third_party/NNPACK' 2024-08-20T23:46:12.3943600Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:46:12.4011991Z Entering 'third_party/XNNPACK' 2024-08-20T23:46:12.4097017Z Entering 'third_party/benchmark' 2024-08-20T23:46:12.4169590Z Entering 'third_party/cpp-httplib' 2024-08-20T23:46:12.4239542Z Entering 'third_party/cpuinfo' 2024-08-20T23:46:12.4308472Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:46:12.4377753Z Entering 'third_party/cutlass' 2024-08-20T23:46:12.4453659Z Entering 'third_party/eigen' 2024-08-20T23:46:12.4528051Z Entering 'third_party/fbgemm' 2024-08-20T23:46:12.4596328Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:46:12.4664708Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:46:12.4732656Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:46:12.4806864Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:46:12.4873124Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:46:12.4942175Z Entering 'third_party/flatbuffers' 2024-08-20T23:46:12.5016116Z Entering 'third_party/fmt' 2024-08-20T23:46:12.5090471Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:46:12.5158543Z Entering 'third_party/gloo' 2024-08-20T23:46:12.5228716Z Entering 'third_party/googletest' 2024-08-20T23:46:12.5297546Z Entering 'third_party/ideep' 2024-08-20T23:46:12.5367702Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:46:12.5444718Z Entering 'third_party/ittapi' 2024-08-20T23:46:12.5514096Z Entering 'third_party/kineto' 2024-08-20T23:46:12.5581934Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:46:12.5647837Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:46:12.5718568Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:46:12.5788050Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:46:12.5856948Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:46:12.5922603Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:46:12.5994294Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:46:12.6062549Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:46:12.6129863Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:46:12.6198811Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:46:12.6268643Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:46:12.6335430Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:46:12.6410865Z Entering 'third_party/mimalloc' 2024-08-20T23:46:12.6480381Z Entering 'third_party/nccl/nccl' 2024-08-20T23:46:12.6548998Z Entering 'third_party/nlohmann' 2024-08-20T23:46:12.6618647Z Entering 'third_party/onnx' 2024-08-20T23:46:12.6700509Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:46:12.6771538Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:46:12.6846516Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:46:12.6919341Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:46:12.6985473Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:46:12.7060392Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:46:12.7126355Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:46:12.7194509Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:46:12.7261131Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:46:12.7327789Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:46:12.7395174Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:46:12.7466128Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:46:12.7536717Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:46:12.7625292Z Entering 'third_party/pocketfft' 2024-08-20T23:46:12.7695847Z Entering 'third_party/protobuf' 2024-08-20T23:46:12.7766396Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:46:12.7832980Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:46:12.7904796Z Entering 'third_party/psimd' 2024-08-20T23:46:12.7973900Z Entering 'third_party/pthreadpool' 2024-08-20T23:46:12.8042600Z Entering 'third_party/pybind11' 2024-08-20T23:46:12.8112262Z Entering 'third_party/python-peachpy' 2024-08-20T23:46:12.8182064Z Entering 'third_party/sleef' 2024-08-20T23:46:12.8250902Z Entering 'third_party/tensorpipe' 2024-08-20T23:46:12.8319178Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:46:12.8385928Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:46:12.8452565Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:46:12.8519558Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:46:12.8582904Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:46:12.8679602Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T23:46:12.8716062Z http.https://github.com/.extraheader 2024-08-20T23:46:12.8725680Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-20T23:46:12.8776530Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T23:46:12.9146091Z Entering 'android/libs/fbjni' 2024-08-20T23:46:12.9188605Z http.https://github.com/.extraheader 2024-08-20T23:46:12.9232593Z Entering 'third_party/FP16' 2024-08-20T23:46:12.9276806Z http.https://github.com/.extraheader 2024-08-20T23:46:12.9319594Z Entering 'third_party/FXdiv' 2024-08-20T23:46:12.9362878Z http.https://github.com/.extraheader 2024-08-20T23:46:12.9406385Z Entering 'third_party/NNPACK' 2024-08-20T23:46:12.9450614Z http.https://github.com/.extraheader 2024-08-20T23:46:12.9496117Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:46:12.9538539Z http.https://github.com/.extraheader 2024-08-20T23:46:12.9581590Z Entering 'third_party/XNNPACK' 2024-08-20T23:46:12.9625346Z http.https://github.com/.extraheader 2024-08-20T23:46:12.9685732Z Entering 'third_party/benchmark' 2024-08-20T23:46:12.9729152Z http.https://github.com/.extraheader 2024-08-20T23:46:12.9773751Z Entering 'third_party/cpp-httplib' 2024-08-20T23:46:12.9825552Z http.https://github.com/.extraheader 2024-08-20T23:46:12.9870561Z Entering 'third_party/cpuinfo' 2024-08-20T23:46:12.9914963Z http.https://github.com/.extraheader 2024-08-20T23:46:12.9960970Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:46:13.0003844Z http.https://github.com/.extraheader 2024-08-20T23:46:13.0048493Z Entering 'third_party/cutlass' 2024-08-20T23:46:13.0095899Z http.https://github.com/.extraheader 2024-08-20T23:46:13.0149418Z Entering 'third_party/eigen' 2024-08-20T23:46:13.0194391Z http.https://github.com/.extraheader 2024-08-20T23:46:13.0241247Z Entering 'third_party/fbgemm' 2024-08-20T23:46:13.0286335Z http.https://github.com/.extraheader 2024-08-20T23:46:13.0329809Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:46:13.0375990Z http.https://github.com/.extraheader 2024-08-20T23:46:13.0419592Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:46:13.0463761Z http.https://github.com/.extraheader 2024-08-20T23:46:13.0507816Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:46:13.0550443Z http.https://github.com/.extraheader 2024-08-20T23:46:13.0599800Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:46:13.0641534Z http.https://github.com/.extraheader 2024-08-20T23:46:13.0685566Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:46:13.0728367Z http.https://github.com/.extraheader 2024-08-20T23:46:13.0780296Z Entering 'third_party/flatbuffers' 2024-08-20T23:46:13.0825676Z http.https://github.com/.extraheader 2024-08-20T23:46:13.0872385Z Entering 'third_party/fmt' 2024-08-20T23:46:13.0916423Z http.https://github.com/.extraheader 2024-08-20T23:46:13.0961491Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:46:13.1005999Z http.https://github.com/.extraheader 2024-08-20T23:46:13.1051702Z Entering 'third_party/gloo' 2024-08-20T23:46:13.1096898Z http.https://github.com/.extraheader 2024-08-20T23:46:13.1140688Z Entering 'third_party/googletest' 2024-08-20T23:46:13.1185630Z http.https://github.com/.extraheader 2024-08-20T23:46:13.1229813Z Entering 'third_party/ideep' 2024-08-20T23:46:13.1275474Z http.https://github.com/.extraheader 2024-08-20T23:46:13.1318044Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:46:13.1361644Z http.https://github.com/.extraheader 2024-08-20T23:46:13.1413413Z Entering 'third_party/ittapi' 2024-08-20T23:46:13.1459982Z http.https://github.com/.extraheader 2024-08-20T23:46:13.1504182Z Entering 'third_party/kineto' 2024-08-20T23:46:13.1548961Z http.https://github.com/.extraheader 2024-08-20T23:46:13.1592301Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:46:13.1635216Z http.https://github.com/.extraheader 2024-08-20T23:46:13.1678575Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:46:13.1722611Z http.https://github.com/.extraheader 2024-08-20T23:46:13.1769613Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:46:13.1813253Z http.https://github.com/.extraheader 2024-08-20T23:46:13.1858256Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:46:13.1901855Z http.https://github.com/.extraheader 2024-08-20T23:46:13.1947014Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:46:13.1990913Z http.https://github.com/.extraheader 2024-08-20T23:46:13.2033212Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:46:13.2077789Z http.https://github.com/.extraheader 2024-08-20T23:46:13.2126003Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:46:13.2170804Z http.https://github.com/.extraheader 2024-08-20T23:46:13.2215687Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:46:13.2263755Z http.https://github.com/.extraheader 2024-08-20T23:46:13.2307427Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:46:13.2351166Z http.https://github.com/.extraheader 2024-08-20T23:46:13.2397072Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:46:13.2440038Z http.https://github.com/.extraheader 2024-08-20T23:46:13.2489690Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:46:13.2532195Z http.https://github.com/.extraheader 2024-08-20T23:46:13.2575607Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:46:13.2623236Z http.https://github.com/.extraheader 2024-08-20T23:46:13.2669485Z Entering 'third_party/mimalloc' 2024-08-20T23:46:13.2714112Z http.https://github.com/.extraheader 2024-08-20T23:46:13.2760325Z Entering 'third_party/nccl/nccl' 2024-08-20T23:46:13.2804670Z http.https://github.com/.extraheader 2024-08-20T23:46:13.2849740Z Entering 'third_party/nlohmann' 2024-08-20T23:46:13.2894153Z http.https://github.com/.extraheader 2024-08-20T23:46:13.2938593Z Entering 'third_party/onnx' 2024-08-20T23:46:13.2982576Z http.https://github.com/.extraheader 2024-08-20T23:46:13.3040385Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:46:13.3086009Z http.https://github.com/.extraheader 2024-08-20T23:46:13.3128311Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:46:13.3170579Z http.https://github.com/.extraheader 2024-08-20T23:46:13.3217318Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:46:13.3267096Z http.https://github.com/.extraheader 2024-08-20T23:46:13.3311747Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:46:13.3358370Z http.https://github.com/.extraheader 2024-08-20T23:46:13.3400280Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:46:13.3441511Z http.https://github.com/.extraheader 2024-08-20T23:46:13.3485394Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:46:13.3526439Z http.https://github.com/.extraheader 2024-08-20T23:46:13.3569488Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:46:13.3610637Z http.https://github.com/.extraheader 2024-08-20T23:46:13.3654528Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:46:13.3695303Z http.https://github.com/.extraheader 2024-08-20T23:46:13.3736462Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:46:13.3777834Z http.https://github.com/.extraheader 2024-08-20T23:46:13.3819263Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:46:13.3860729Z http.https://github.com/.extraheader 2024-08-20T23:46:13.3900937Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:46:13.3946656Z http.https://github.com/.extraheader 2024-08-20T23:46:13.3991427Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:46:13.4033137Z http.https://github.com/.extraheader 2024-08-20T23:46:13.4079148Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:46:13.4120545Z http.https://github.com/.extraheader 2024-08-20T23:46:13.4188107Z Entering 'third_party/pocketfft' 2024-08-20T23:46:13.4232486Z http.https://github.com/.extraheader 2024-08-20T23:46:13.4279604Z Entering 'third_party/protobuf' 2024-08-20T23:46:13.4323149Z http.https://github.com/.extraheader 2024-08-20T23:46:13.4373423Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:46:13.4416686Z http.https://github.com/.extraheader 2024-08-20T23:46:13.4460056Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:46:13.4502319Z http.https://github.com/.extraheader 2024-08-20T23:46:13.4548665Z Entering 'third_party/psimd' 2024-08-20T23:46:13.4593053Z http.https://github.com/.extraheader 2024-08-20T23:46:13.4636416Z Entering 'third_party/pthreadpool' 2024-08-20T23:46:13.4680816Z http.https://github.com/.extraheader 2024-08-20T23:46:13.4723506Z Entering 'third_party/pybind11' 2024-08-20T23:46:13.4768544Z http.https://github.com/.extraheader 2024-08-20T23:46:13.4812746Z Entering 'third_party/python-peachpy' 2024-08-20T23:46:13.4857904Z http.https://github.com/.extraheader 2024-08-20T23:46:13.4901391Z Entering 'third_party/sleef' 2024-08-20T23:46:13.4944962Z http.https://github.com/.extraheader 2024-08-20T23:46:13.4989531Z Entering 'third_party/tensorpipe' 2024-08-20T23:46:13.5032889Z http.https://github.com/.extraheader 2024-08-20T23:46:13.5075524Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:46:13.5117867Z http.https://github.com/.extraheader 2024-08-20T23:46:13.5161274Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:46:13.5203353Z http.https://github.com/.extraheader 2024-08-20T23:46:13.5247024Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:46:13.5289825Z http.https://github.com/.extraheader 2024-08-20T23:46:13.5332694Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:46:13.5376953Z http.https://github.com/.extraheader 2024-08-20T23:46:13.5417857Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:46:13.5464597Z http.https://github.com/.extraheader 2024-08-20T23:46:13.5622066Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-20T23:46:13.5642274Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-20T23:46:13.5650882Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:46:13.5651402Z ##[endgroup] 2024-08-20T23:46:20.4241071Z Cleaning up orphan processes