2024-08-20T22:08:38.8388044Z Current runner version: '2.319.1' 2024-08-20T22:08:38.8394566Z Runner name: 'i-0ab7c6e949d6545da' 2024-08-20T22:08:38.8395499Z Runner group name: 'Default' 2024-08-20T22:08:38.8396385Z Machine name: 'ip-10-0-37-113' 2024-08-20T22:08:38.8413503Z Testing runner upgrade compatibility 2024-08-20T22:08:39.0243769Z ##[group]GITHUB_TOKEN Permissions 2024-08-20T22:08:39.0245899Z Actions: read 2024-08-20T22:08:39.0246288Z Attestations: read 2024-08-20T22:08:39.0246895Z Checks: read 2024-08-20T22:08:39.0247342Z Contents: read 2024-08-20T22:08:39.0247764Z Deployments: read 2024-08-20T22:08:39.0248249Z Discussions: read 2024-08-20T22:08:39.0248645Z Issues: read 2024-08-20T22:08:39.0249027Z Metadata: read 2024-08-20T22:08:39.0249451Z Packages: read 2024-08-20T22:08:39.0249855Z Pages: read 2024-08-20T22:08:39.0250234Z PullRequests: read 2024-08-20T22:08:39.0250726Z RepositoryProjects: read 2024-08-20T22:08:39.0251134Z SecurityEvents: read 2024-08-20T22:08:39.0251546Z Statuses: read 2024-08-20T22:08:39.0252225Z ##[endgroup] 2024-08-20T22:08:39.0255428Z Secret source: Actions 2024-08-20T22:08:39.0256230Z Prepare workflow directory 2024-08-20T22:08:39.1184174Z Prepare all required actions 2024-08-20T22:08:39.1350962Z Getting action download info 2024-08-20T22:08:39.3190204Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-20T22:08:39.7677357Z Download action repository 'pytorch/pytorch@main' (SHA:1ae5d5bb62141d7c9b1b0b66c66a462b4e10b1f2) 2024-08-20T22:08:43.8917249Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-20T22:08:44.0814597Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-20T22:08:44.3691732Z Getting action download info 2024-08-20T22:08:44.4776044Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-20T22:08:44.6775907Z Getting action download info 2024-08-20T22:08:44.7666900Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-20T22:08:44.9082234Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/133712 (40ec5f6ddd9787aca0449b24128343ff4c4a88b3) 2024-08-20T22:08:44.9084383Z ##[group] Inputs 2024-08-20T22:08:44.9084888Z build-environment: linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-08-20T22:08:44.9086213Z test-matrix: {"include": [{"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:08:44.9088289Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:44.9089568Z sync-tag: linux-focal-cuda12_4-py3_10-gcc9-inductor-test 2024-08-20T22:08:44.9090592Z timeout-minutes: 240 2024-08-20T22:08:44.9090924Z use-gha: 2024-08-20T22:08:44.9091221Z dashboard-tag: 2024-08-20T22:08:44.9091542Z s3-bucket: gha-artifacts 2024-08-20T22:08:44.9091901Z aws-role-to-assume: 2024-08-20T22:08:44.9092230Z ##[endgroup] 2024-08-20T22:08:44.9093094Z Complete job name: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:08:44.9824073Z A job started hook has been configured by the self-hosted runner administrator 2024-08-20T22:08:44.9968105Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-20T22:08:44.9979882Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:44.9980407Z ##[endgroup] 2024-08-20T22:08:46.1561648Z Runner Type: amz2023.linux.g5.4xlarge.nvidia.gpu 2024-08-20T22:08:46.1562200Z Instance Type: g5.4xlarge 2024-08-20T22:08:46.1562894Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-20T22:08:46.1563433Z AMI ID: ami-06c68f701d8090592 2024-08-20T22:08:51.9118286Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-20T22:08:51.9118831Z with: 2024-08-20T22:08:51.9119514Z github-secret: *** 2024-08-20T22:08:51.9120431Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:08:51.9121419Z activate-with-label: false 2024-08-20T22:08:51.9121777Z label: with-ssh 2024-08-20T22:08:51.9122099Z remove-existing-keys: true 2024-08-20T22:08:51.9122468Z fail-silently: true 2024-08-20T22:08:51.9122776Z env: 2024-08-20T22:08:51.9123049Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:51.9123393Z ##[endgroup] 2024-08-20T22:08:52.0024472Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-20T22:08:52.0026933Z ciflow reference detected, attempting to extract PR number 2024-08-20T22:08:52.2847630Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-08-20T22:08:52.3471378Z No SSH keys found for user pytorch-bot[bot] 2024-08-20T22:08:52.3471990Z Grabbing public ssh keys from https://github.com/XuehaiPan.keys 2024-08-20T22:08:52.4244961Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-20T22:08:52.4260658Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-20T22:08:52.4284165Z Login using: ssh ec2-user@ec2-34-230-37-37.compute-1.amazonaws.com 2024-08-20T22:08:52.4284942Z All testing is done inside the container, to start an interactive session run: 2024-08-20T22:08:52.4285752Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:08:52.4417613Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-20T22:08:52.4418168Z with: 2024-08-20T22:08:52.4418451Z submodules: recursive 2024-08-20T22:08:52.4418783Z fetch-depth: 0 2024-08-20T22:08:52.4419074Z env: 2024-08-20T22:08:52.4419349Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:52.4419702Z ##[endgroup] 2024-08-20T22:08:52.4622579Z ##[group]Run retry () { 2024-08-20T22:08:52.4622975Z retry () { 2024-08-20T22:08:52.4623494Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-20T22:08:52.4624047Z } 2024-08-20T22:08:52.4624357Z echo "${GITHUB_WORKSPACE}" 2024-08-20T22:08:52.4624781Z if [ -z "${NO_SUDO}" ]; then 2024-08-20T22:08:52.4625258Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:08:52.4625717Z else 2024-08-20T22:08:52.4626060Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:08:52.4626486Z fi 2024-08-20T22:08:52.4626829Z mkdir "${GITHUB_WORKSPACE}" 2024-08-20T22:08:52.4638814Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:52.4639318Z env: 2024-08-20T22:08:52.4639597Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:52.4639935Z NO_SUDO: 2024-08-20T22:08:52.4640210Z ##[endgroup] 2024-08-20T22:08:52.4672945Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:08:52.5557567Z ##[group]Run malfet/checkout@silent-checkout 2024-08-20T22:08:52.5558001Z with: 2024-08-20T22:08:52.5558333Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:08:52.5558775Z fetch-depth: 0 2024-08-20T22:08:52.5559220Z submodules: recursive 2024-08-20T22:08:52.5559561Z quiet-checkout: true 2024-08-20T22:08:52.5559913Z repository: pytorch/pytorch 2024-08-20T22:08:52.5560393Z token: *** 2024-08-20T22:08:52.5560688Z ssh-strict: true 2024-08-20T22:08:52.5561009Z persist-credentials: true 2024-08-20T22:08:52.5561366Z clean: true 2024-08-20T22:08:52.5561701Z sparse-checkout-cone-mode: true 2024-08-20T22:08:52.5562086Z lfs: false 2024-08-20T22:08:52.5562385Z set-safe-directory: true 2024-08-20T22:08:52.5562724Z env: 2024-08-20T22:08:52.5563000Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:52.5563340Z ##[endgroup] 2024-08-20T22:08:52.6545029Z Syncing repository: pytorch/pytorch 2024-08-20T22:08:52.6546963Z ##[group]Getting Git version info 2024-08-20T22:08:52.6547750Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:08:52.6548680Z [command]/usr/bin/git version 2024-08-20T22:08:52.6549174Z git version 2.40.1 2024-08-20T22:08:52.6552432Z ##[endgroup] 2024-08-20T22:08:52.6567677Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/e5000c6b-5335-4522-9c8c-e2f52719bc20' before making global git config changes 2024-08-20T22:08:52.6568996Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T22:08:52.6573464Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:08:52.6626068Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:08:52.6630170Z ##[group]Initializing the repository 2024-08-20T22:08:52.6633247Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:08:52.6675644Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-20T22:08:52.6676739Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-20T22:08:52.6677819Z hint: of your new repositories, which will suppress this warning, call: 2024-08-20T22:08:52.6678548Z hint: 2024-08-20T22:08:52.6679144Z hint: git config --global init.defaultBranch 2024-08-20T22:08:52.6679707Z hint: 2024-08-20T22:08:52.6680234Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-20T22:08:52.6681080Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-20T22:08:52.6682145Z hint: 2024-08-20T22:08:52.6682601Z hint: git branch -m 2024-08-20T22:08:52.6683347Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-20T22:08:52.6690139Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-20T22:08:52.6732356Z ##[endgroup] 2024-08-20T22:08:52.6733044Z ##[group]Disabling automatic garbage collection 2024-08-20T22:08:52.6735554Z [command]/usr/bin/git config --local gc.auto 0 2024-08-20T22:08:52.6776441Z ##[endgroup] 2024-08-20T22:08:52.6777075Z ##[group]Setting up auth 2024-08-20T22:08:52.6782752Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T22:08:52.6825290Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T22:08:52.7197316Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T22:08:52.7238558Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T22:08:52.7588460Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:08:52.7643197Z ##[endgroup] 2024-08-20T22:08:52.7643910Z ##[group]Fetching the repository 2024-08-20T22:08:52.7650011Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-20T22:08:55.7438991Z remote: Enumerating objects: 1008509 2024-08-20T22:08:55.7439603Z remote: Enumerating objects: 1010208, done. 2024-08-20T22:08:55.7440293Z remote: Counting objects: 0% (1/1699) 2024-08-20T22:08:55.7440810Z remote: Counting objects: 1% (17/1699) 2024-08-20T22:08:55.7441340Z remote: Counting objects: 2% (34/1699) 2024-08-20T22:08:55.7441843Z remote: Counting objects: 3% (51/1699) 2024-08-20T22:08:55.7442362Z remote: Counting objects: 4% (68/1699) 2024-08-20T22:08:55.7442854Z remote: Counting objects: 5% (85/1699) 2024-08-20T22:08:55.7443357Z remote: Counting objects: 6% (102/1699) 2024-08-20T22:08:55.7444226Z remote: Counting objects: 7% (119/1699) 2024-08-20T22:08:55.7444720Z remote: Counting objects: 8% (136/1699) 2024-08-20T22:08:55.7445215Z remote: Counting objects: 9% (153/1699) 2024-08-20T22:08:55.7445716Z remote: Counting objects: 10% (170/1699) 2024-08-20T22:08:55.7446209Z remote: Counting objects: 11% (187/1699) 2024-08-20T22:08:55.7446692Z remote: Counting objects: 12% (204/1699) 2024-08-20T22:08:55.7447310Z remote: Counting objects: 13% (221/1699) 2024-08-20T22:08:55.7447808Z remote: Counting objects: 14% (238/1699) 2024-08-20T22:08:55.7448307Z remote: Counting objects: 15% (255/1699) 2024-08-20T22:08:55.7448789Z remote: Counting objects: 16% (272/1699) 2024-08-20T22:08:55.7449303Z remote: Counting objects: 17% (289/1699) 2024-08-20T22:08:55.7449822Z remote: Counting objects: 18% (306/1699) 2024-08-20T22:08:55.7450320Z remote: Counting objects: 19% (323/1699) 2024-08-20T22:08:55.7450805Z remote: Counting objects: 20% (340/1699) 2024-08-20T22:08:55.7451293Z remote: Counting objects: 21% (357/1699) 2024-08-20T22:08:55.7451828Z remote: Counting objects: 22% (374/1699) 2024-08-20T22:08:55.7452315Z remote: Counting objects: 23% (391/1699) 2024-08-20T22:08:55.7452803Z remote: Counting objects: 24% (408/1699) 2024-08-20T22:08:55.7453283Z remote: Counting objects: 25% (425/1699) 2024-08-20T22:08:55.7453770Z remote: Counting objects: 26% (442/1699) 2024-08-20T22:08:55.7454258Z remote: Counting objects: 27% (459/1699) 2024-08-20T22:08:55.7454927Z remote: Counting objects: 28% (476/1699) 2024-08-20T22:08:55.7455413Z remote: Counting objects: 29% (493/1699) 2024-08-20T22:08:55.7455900Z remote: Counting objects: 30% (510/1699) 2024-08-20T22:08:55.7456389Z remote: Counting objects: 31% (527/1699) 2024-08-20T22:08:55.7456882Z remote: Counting objects: 32% (544/1699) 2024-08-20T22:08:55.7457369Z remote: Counting objects: 33% (561/1699) 2024-08-20T22:08:55.7457861Z remote: Counting objects: 34% (578/1699) 2024-08-20T22:08:55.7458348Z remote: Counting objects: 35% (595/1699) 2024-08-20T22:08:55.7458837Z remote: Counting objects: 36% (612/1699) 2024-08-20T22:08:55.7459317Z remote: Counting objects: 37% (629/1699) 2024-08-20T22:08:55.7459806Z remote: Counting objects: 38% (646/1699) 2024-08-20T22:08:55.7460293Z remote: Counting objects: 39% (663/1699) 2024-08-20T22:08:55.7460779Z remote: Counting objects: 40% (680/1699) 2024-08-20T22:08:55.7461268Z remote: Counting objects: 41% (697/1699) 2024-08-20T22:08:55.7461760Z remote: Counting objects: 42% (714/1699) 2024-08-20T22:08:55.7462251Z remote: Counting objects: 43% (731/1699) 2024-08-20T22:08:55.7462738Z remote: Counting objects: 44% (748/1699) 2024-08-20T22:08:55.7463222Z remote: Counting objects: 45% (765/1699) 2024-08-20T22:08:55.7463719Z remote: Counting objects: 46% (782/1699) 2024-08-20T22:08:55.7464208Z remote: Counting objects: 47% (799/1699) 2024-08-20T22:08:55.7464691Z remote: Counting objects: 48% (816/1699) 2024-08-20T22:08:55.7465178Z remote: Counting objects: 49% (833/1699) 2024-08-20T22:08:55.7465665Z remote: Counting objects: 50% (850/1699) 2024-08-20T22:08:55.7466153Z remote: Counting objects: 51% (867/1699) 2024-08-20T22:08:55.7466635Z remote: Counting objects: 52% (884/1699) 2024-08-20T22:08:55.7467124Z remote: Counting objects: 53% (901/1699) 2024-08-20T22:08:55.7467618Z remote: Counting objects: 54% (918/1699) 2024-08-20T22:08:55.7468127Z remote: Counting objects: 55% (935/1699) 2024-08-20T22:08:55.7468615Z remote: Counting objects: 56% (952/1699) 2024-08-20T22:08:55.7469098Z remote: Counting objects: 57% (969/1699) 2024-08-20T22:08:55.7469735Z remote: Counting objects: 58% (986/1699) 2024-08-20T22:08:55.7470233Z remote: Counting objects: 59% (1003/1699) 2024-08-20T22:08:55.7470747Z remote: Counting objects: 60% (1020/1699) 2024-08-20T22:08:55.7471252Z remote: Counting objects: 61% (1037/1699) 2024-08-20T22:08:55.7471763Z remote: Counting objects: 62% (1054/1699) 2024-08-20T22:08:55.7472285Z remote: Counting objects: 63% (1071/1699) 2024-08-20T22:08:55.7472797Z remote: Counting objects: 64% (1088/1699) 2024-08-20T22:08:55.7473310Z remote: Counting objects: 65% (1105/1699) 2024-08-20T22:08:55.7473813Z remote: Counting objects: 66% (1122/1699) 2024-08-20T22:08:55.7474324Z remote: Counting objects: 67% (1139/1699) 2024-08-20T22:08:55.7474829Z remote: Counting objects: 68% (1156/1699) 2024-08-20T22:08:55.7475333Z remote: Counting objects: 69% (1173/1699) 2024-08-20T22:08:55.7475831Z remote: Counting objects: 70% (1190/1699) 2024-08-20T22:08:55.7476349Z remote: Counting objects: 71% (1207/1699) 2024-08-20T22:08:55.7476902Z remote: Counting objects: 72% (1224/1699) 2024-08-20T22:08:55.7477562Z remote: Counting objects: 73% (1241/1699) 2024-08-20T22:08:55.7478176Z remote: Counting objects: 74% (1258/1699) 2024-08-20T22:08:55.7478725Z remote: Counting objects: 75% (1275/1699) 2024-08-20T22:08:55.7479232Z remote: Counting objects: 76% (1292/1699) 2024-08-20T22:08:55.7479742Z remote: Counting objects: 77% (1309/1699) 2024-08-20T22:08:55.7480242Z remote: Counting objects: 78% (1326/1699) 2024-08-20T22:08:55.7480861Z remote: Counting objects: 79% (1343/1699) 2024-08-20T22:08:55.7481371Z remote: Counting objects: 80% (1360/1699) 2024-08-20T22:08:55.7481878Z remote: Counting objects: 81% (1377/1699) 2024-08-20T22:08:55.7482375Z remote: Counting objects: 82% (1394/1699) 2024-08-20T22:08:55.7482882Z remote: Counting objects: 83% (1411/1699) 2024-08-20T22:08:55.7483396Z remote: Counting objects: 84% (1428/1699) 2024-08-20T22:08:55.7483903Z remote: Counting objects: 85% (1445/1699) 2024-08-20T22:08:55.7484403Z remote: Counting objects: 86% (1462/1699) 2024-08-20T22:08:55.7484910Z remote: Counting objects: 87% (1479/1699) 2024-08-20T22:08:55.7485415Z remote: Counting objects: 88% (1496/1699) 2024-08-20T22:08:55.7485920Z remote: Counting objects: 89% (1513/1699) 2024-08-20T22:08:55.7486422Z remote: Counting objects: 90% (1530/1699) 2024-08-20T22:08:55.7487047Z remote: Counting objects: 91% (1547/1699) 2024-08-20T22:08:55.7487562Z remote: Counting objects: 92% (1564/1699) 2024-08-20T22:08:55.7488069Z remote: Counting objects: 93% (1581/1699) 2024-08-20T22:08:55.7488571Z remote: Counting objects: 94% (1598/1699) 2024-08-20T22:08:55.7489076Z remote: Counting objects: 95% (1615/1699) 2024-08-20T22:08:55.7489581Z remote: Counting objects: 96% (1632/1699) 2024-08-20T22:08:55.7490085Z remote: Counting objects: 97% (1649/1699) 2024-08-20T22:08:55.7490592Z remote: Counting objects: 98% (1666/1699) 2024-08-20T22:08:55.7491102Z remote: Counting objects: 99% (1683/1699) 2024-08-20T22:08:55.7491608Z remote: Counting objects: 100% (1699/1699) 2024-08-20T22:08:55.7492144Z remote: Counting objects: 100% (1699/1699), done. 2024-08-20T22:08:55.7614155Z remote: Compressing objects: 0% (1/826) 2024-08-20T22:08:55.7869286Z remote: Compressing objects: 1% (9/826) 2024-08-20T22:08:55.8156771Z remote: Compressing objects: 2% (17/826) 2024-08-20T22:08:55.8633762Z remote: Compressing objects: 3% (25/826) 2024-08-20T22:08:55.9434258Z remote: Compressing objects: 4% (34/826) 2024-08-20T22:08:55.9854383Z remote: Compressing objects: 5% (42/826) 2024-08-20T22:08:56.0483335Z remote: Compressing objects: 6% (50/826) 2024-08-20T22:08:56.0910140Z remote: Compressing objects: 7% (58/826) 2024-08-20T22:08:56.1202605Z remote: Compressing objects: 8% (67/826) 2024-08-20T22:08:56.1432363Z remote: Compressing objects: 9% (75/826) 2024-08-20T22:08:56.1676230Z remote: Compressing objects: 10% (83/826) 2024-08-20T22:08:56.1887110Z remote: Compressing objects: 11% (91/826) 2024-08-20T22:08:56.2079664Z remote: Compressing objects: 12% (100/826) 2024-08-20T22:08:56.2225398Z remote: Compressing objects: 13% (108/826) 2024-08-20T22:08:56.2344899Z remote: Compressing objects: 14% (116/826) 2024-08-20T22:08:56.2434029Z remote: Compressing objects: 15% (124/826) 2024-08-20T22:08:56.2520162Z remote: Compressing objects: 16% (133/826) 2024-08-20T22:08:56.2587481Z remote: Compressing objects: 17% (141/826) 2024-08-20T22:08:56.2620271Z remote: Compressing objects: 18% (149/826) 2024-08-20T22:08:56.2668102Z remote: Compressing objects: 19% (157/826) 2024-08-20T22:08:56.2699329Z remote: Compressing objects: 20% (166/826) 2024-08-20T22:08:56.2718608Z remote: Compressing objects: 21% (174/826) 2024-08-20T22:08:56.2723570Z remote: Compressing objects: 22% (182/826) 2024-08-20T22:08:56.2728214Z remote: Compressing objects: 23% (190/826) 2024-08-20T22:08:56.2737179Z remote: Compressing objects: 24% (199/826) 2024-08-20T22:08:56.2740855Z remote: Compressing objects: 25% (207/826) 2024-08-20T22:08:56.2750860Z remote: Compressing objects: 26% (215/826) 2024-08-20T22:08:56.2759492Z remote: Compressing objects: 27% (224/826) 2024-08-20T22:08:56.2785852Z remote: Compressing objects: 28% (232/826) 2024-08-20T22:08:56.2791859Z remote: Compressing objects: 29% (240/826) 2024-08-20T22:08:56.2804825Z remote: Compressing objects: 30% (248/826) 2024-08-20T22:08:56.2814120Z remote: Compressing objects: 31% (257/826) 2024-08-20T22:08:56.2819261Z remote: Compressing objects: 32% (265/826) 2024-08-20T22:08:56.2828234Z remote: Compressing objects: 33% (273/826) 2024-08-20T22:08:56.2835066Z remote: Compressing objects: 34% (281/826) 2024-08-20T22:08:56.2843081Z remote: Compressing objects: 35% (290/826) 2024-08-20T22:08:56.2845915Z remote: Compressing objects: 36% (298/826) 2024-08-20T22:08:56.2856029Z remote: Compressing objects: 37% (306/826) 2024-08-20T22:08:56.2862688Z remote: Compressing objects: 38% (314/826) 2024-08-20T22:08:56.2870283Z remote: Compressing objects: 39% (323/826) 2024-08-20T22:08:56.2874956Z remote: Compressing objects: 40% (331/826) 2024-08-20T22:08:56.2879905Z remote: Compressing objects: 41% (339/826) 2024-08-20T22:08:56.2890205Z remote: Compressing objects: 42% (347/826) 2024-08-20T22:08:56.2893982Z remote: Compressing objects: 43% (356/826) 2024-08-20T22:08:56.2900596Z remote: Compressing objects: 44% (364/826) 2024-08-20T22:08:56.2905343Z remote: Compressing objects: 45% (372/826) 2024-08-20T22:08:56.2910538Z remote: Compressing objects: 46% (380/826) 2024-08-20T22:08:56.2913718Z remote: Compressing objects: 47% (389/826) 2024-08-20T22:08:56.2917969Z remote: Compressing objects: 48% (397/826) 2024-08-20T22:08:56.2919681Z remote: Compressing objects: 49% (405/826) 2024-08-20T22:08:56.2922760Z remote: Compressing objects: 50% (413/826) 2024-08-20T22:08:56.2926253Z remote: Compressing objects: 51% (422/826) 2024-08-20T22:08:56.2928346Z remote: Compressing objects: 52% (430/826) 2024-08-20T22:08:56.2930788Z remote: Compressing objects: 53% (438/826) 2024-08-20T22:08:56.2932537Z remote: Compressing objects: 54% (447/826) 2024-08-20T22:08:56.2933799Z remote: Compressing objects: 55% (455/826) 2024-08-20T22:08:56.2935505Z remote: Compressing objects: 56% (463/826) 2024-08-20T22:08:56.2936597Z remote: Compressing objects: 57% (471/826) 2024-08-20T22:08:56.2937498Z remote: Compressing objects: 58% (480/826) 2024-08-20T22:08:56.2938392Z remote: Compressing objects: 59% (488/826) 2024-08-20T22:08:56.2939022Z remote: Compressing objects: 60% (496/826) 2024-08-20T22:08:56.2939551Z remote: Compressing objects: 61% (504/826) 2024-08-20T22:08:56.2940086Z remote: Compressing objects: 62% (513/826) 2024-08-20T22:08:56.2940814Z remote: Compressing objects: 63% (521/826) 2024-08-20T22:08:56.2942020Z remote: Compressing objects: 64% (529/826) 2024-08-20T22:08:56.2949573Z remote: Compressing objects: 65% (537/826) 2024-08-20T22:08:56.2954522Z remote: Compressing objects: 66% (546/826) 2024-08-20T22:08:56.2958886Z remote: Compressing objects: 67% (554/826) 2024-08-20T22:08:56.2962891Z remote: Compressing objects: 68% (562/826) 2024-08-20T22:08:56.2967368Z remote: Compressing objects: 69% (570/826) 2024-08-20T22:08:56.2971598Z remote: Compressing objects: 70% (579/826) 2024-08-20T22:08:56.2974550Z remote: Compressing objects: 71% (587/826) 2024-08-20T22:08:56.2977256Z remote: Compressing objects: 72% (595/826) 2024-08-20T22:08:56.2981335Z remote: Compressing objects: 73% (603/826) 2024-08-20T22:08:56.2983996Z remote: Compressing objects: 74% (612/826) 2024-08-20T22:08:56.2987005Z remote: Compressing objects: 75% (620/826) 2024-08-20T22:08:56.2994959Z remote: Compressing objects: 76% (628/826) 2024-08-20T22:08:56.2995606Z remote: Compressing objects: 77% (637/826) 2024-08-20T22:08:56.2996127Z remote: Compressing objects: 78% (645/826) 2024-08-20T22:08:56.2997085Z remote: Compressing objects: 79% (653/826) 2024-08-20T22:08:56.2997920Z remote: Compressing objects: 80% (661/826) 2024-08-20T22:08:56.3000645Z remote: Compressing objects: 81% (670/826) 2024-08-20T22:08:56.3002314Z remote: Compressing objects: 82% (678/826) 2024-08-20T22:08:56.3003824Z remote: Compressing objects: 83% (686/826) 2024-08-20T22:08:56.3005946Z remote: Compressing objects: 84% (694/826) 2024-08-20T22:08:56.3008003Z remote: Compressing objects: 85% (703/826) 2024-08-20T22:08:56.3009218Z remote: Compressing objects: 86% (711/826) 2024-08-20T22:08:56.3010457Z remote: Compressing objects: 87% (719/826) 2024-08-20T22:08:56.3012959Z remote: Compressing objects: 88% (727/826) 2024-08-20T22:08:56.3014987Z remote: Compressing objects: 89% (736/826) 2024-08-20T22:08:56.3016343Z remote: Compressing objects: 90% (744/826) 2024-08-20T22:08:56.3017365Z remote: Compressing objects: 91% (752/826) 2024-08-20T22:08:56.3017929Z remote: Compressing objects: 92% (760/826) 2024-08-20T22:08:56.3018739Z remote: Compressing objects: 93% (769/826) 2024-08-20T22:08:56.3019989Z remote: Compressing objects: 94% (777/826) 2024-08-20T22:08:56.3021839Z remote: Compressing objects: 95% (785/826) 2024-08-20T22:08:56.3023116Z remote: Compressing objects: 96% (793/826) 2024-08-20T22:08:56.3023792Z remote: Compressing objects: 97% (802/826) 2024-08-20T22:08:56.3024936Z remote: Compressing objects: 98% (810/826) 2024-08-20T22:08:56.3025994Z remote: Compressing objects: 99% (818/826) 2024-08-20T22:08:56.3027727Z remote: Compressing objects: 100% (826/826) 2024-08-20T22:08:56.3028298Z remote: Compressing objects: 100% (826/826), done. 2024-08-20T22:09:19.0353827Z remote: Total 1010208 (delta 1106), reused 1335 (delta 870), pack-reused 1008509 (from 1) 2024-08-20T22:09:40.1504581Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-20T22:09:40.1538433Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:09:40.1543670Z ##[endgroup] 2024-08-20T22:09:40.1545721Z ##[group]Determining the checkout info 2024-08-20T22:09:40.1546393Z ##[endgroup] 2024-08-20T22:09:40.1546917Z ##[group]Checking out the ref 2024-08-20T22:09:40.1549703Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:09:41.8788975Z ##[endgroup] 2024-08-20T22:09:41.8789577Z ##[group]Setting up auth for fetching submodules 2024-08-20T22:09:41.8794542Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:09:41.8852364Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-20T22:09:41.8892735Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-20T22:09:41.8932738Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-20T22:09:41.8969269Z ##[endgroup] 2024-08-20T22:09:41.8969796Z ##[group]Fetching submodules 2024-08-20T22:09:41.8972998Z [command]/usr/bin/git submodule sync --recursive 2024-08-20T22:09:41.9346853Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-20T22:09:41.9709305Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-20T22:09:41.9711292Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-20T22:09:41.9715049Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-20T22:09:41.9718215Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-20T22:09:41.9722222Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-20T22:09:41.9724718Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-20T22:09:41.9728652Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-20T22:09:41.9732245Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-20T22:09:41.9735813Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-20T22:09:41.9739694Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-20T22:09:41.9743431Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-20T22:09:41.9747752Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-20T22:09:41.9751660Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-20T22:09:41.9755784Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-20T22:09:41.9759683Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-20T22:09:41.9765254Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:09:41.9772938Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-20T22:09:41.9777204Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-20T22:09:41.9781403Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-20T22:09:41.9785973Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-20T22:09:41.9790432Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-20T22:09:41.9795076Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-20T22:09:41.9799575Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-20T22:09:41.9804318Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-20T22:09:41.9809667Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-20T22:09:41.9816173Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-20T22:09:41.9820650Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-20T22:09:41.9825643Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-20T22:09:41.9830642Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-20T22:09:41.9835843Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-20T22:09:41.9840848Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-20T22:09:41.9846304Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-20T22:09:41.9851525Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-20T22:09:41.9860223Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-20T22:09:41.9893167Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-20T22:09:42.2793675Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-20T22:09:42.4554277Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-20T22:09:42.6254174Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-20T22:09:42.8784513Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-20T22:09:44.9866020Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-20T22:09:56.1315803Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-20T22:09:56.5440209Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-20T22:09:56.9485052Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-20T22:09:57.5789023Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-20T22:09:58.8462410Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-20T22:10:00.8893695Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-20T22:10:06.6793234Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-20T22:10:08.1189743Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-20T22:10:09.8273535Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-20T22:10:11.0441474Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-20T22:10:11.5355591Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-20T22:10:11.8548226Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-20T22:10:12.9941006Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-20T22:10:13.3460763Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-20T22:10:13.6073365Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-20T22:10:15.1984687Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-20T22:10:16.1013689Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-20T22:10:16.7689583Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-20T22:10:22.9598591Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-20T22:10:24.9149585Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-20T22:10:29.1500442Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-20T22:10:29.3685208Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-20T22:10:38.7385345Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-20T22:10:38.9053974Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-20T22:10:39.0981485Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-20T22:10:40.0259584Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-20T22:10:40.3330620Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-20T22:10:40.9632822Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-20T22:10:41.3726174Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-20T22:10:41.3874309Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-20T22:10:41.3990114Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-20T22:10:41.4299496Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-20T22:10:41.4733660Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-20T22:10:42.6492025Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-20T22:10:42.6770035Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-20T22:10:42.7298420Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-20T22:10:42.8419847Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-20T22:10:42.8826245Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-20T22:10:43.5041041Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-20T22:10:43.7915296Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-20T22:10:43.8855918Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-20T22:10:43.8877568Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:10:43.8881137Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:10:43.8884559Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:10:43.8888217Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:10:43.8892009Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:10:43.8926784Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-20T22:10:44.8048202Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-20T22:10:45.4042666Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-20T22:10:47.5239850Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-20T22:10:48.5757082Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-20T22:10:48.9404755Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-20T22:10:49.0508031Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-20T22:10:49.5614808Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-20T22:10:49.6311159Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-20T22:10:49.6459068Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-20T22:10:49.7919057Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-20T22:10:49.8375187Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-20T22:10:49.8836323Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-20T22:10:49.9151849Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-20T22:10:49.9671922Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:10:49.9833437Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-20T22:10:49.9852974Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-20T22:10:49.9882906Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-20T22:11:03.2843700Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-20T22:11:03.3064703Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-20T22:11:03.4063289Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-20T22:11:03.4087301Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:11:03.4090744Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:11:03.4094268Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:11:03.4124318Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-20T22:11:03.9328271Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-20T22:11:05.1045219Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-20T22:11:06.3049966Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-20T22:11:06.3068956Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:11:06.3072951Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:11:06.3076516Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:11:06.3079743Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:11:06.3084036Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:11:06.3088007Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:11:06.3091953Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:11:06.3095834Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:11:06.3129497Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-20T22:11:07.1379838Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-20T22:11:07.5089519Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-20T22:11:08.6988438Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-20T22:11:08.9653286Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-20T22:11:09.5208313Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-20T22:11:10.5507094Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-20T22:11:16.6238478Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-20T22:11:17.0241801Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-20T22:11:17.0475901Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-20T22:11:17.0913641Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-20T22:11:17.1079130Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-20T22:11:17.1098622Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:11:17.1129279Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-20T22:11:17.3882460Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-20T22:11:17.4118833Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-20T22:11:17.4602465Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-20T22:11:17.5820431Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-20T22:11:17.6020072Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-20T22:11:17.6463550Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-20T22:11:17.7124793Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-20T22:11:17.7575554Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-20T22:11:17.7878057Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-20T22:11:17.9129874Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-20T22:11:18.4240560Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-20T22:11:18.4278483Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-20T22:11:18.4281441Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-20T22:11:18.4314062Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-20T22:11:18.8691206Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-20T22:11:20.4510812Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-20T22:11:20.4917658Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-20T22:11:20.5810082Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-20T22:11:20.5835411Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:11:20.5838729Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:11:20.5842666Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:11:20.5846126Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:11:20.5850213Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:11:20.5853683Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:11:20.5857401Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:11:20.5861019Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:11:20.5893267Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-20T22:11:21.0548734Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-20T22:11:22.1166587Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-20T22:11:22.4273043Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-20T22:11:28.4701367Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-20T22:11:28.7373989Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-20T22:11:28.9438472Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-20T22:11:29.2720772Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-20T22:11:36.2688131Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-20T22:11:36.3152412Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-20T22:11:36.3341072Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-20T22:11:36.4573677Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-20T22:11:36.4735129Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-20T22:11:36.4916628Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-20T22:11:36.5114838Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-20T22:11:36.5133880Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:11:36.5137202Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:11:36.5168035Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-20T22:11:38.4157563Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-20T22:11:39.7606374Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-20T22:11:39.8131980Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:11:40.4891551Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-20T22:11:40.5035507Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-20T22:11:40.8208256Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-20T22:11:40.8233930Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:11:40.8237209Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-20T22:11:40.8270915Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-20T22:11:41.2660171Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-20T22:11:42.3143831Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-20T22:11:42.3959374Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-20T22:11:42.4074402Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-20T22:11:42.4223891Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-20T22:11:42.4660207Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-20T22:11:42.4997940Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-20T22:11:42.5487206Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-20T22:11:42.5830831Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-20T22:11:42.5852166Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:11:42.5855312Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:11:42.5859369Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:11:42.5862625Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:11:42.5893760Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-20T22:11:43.6748037Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-20T22:11:43.8902202Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-20T22:11:45.0533933Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-20T22:11:46.0207021Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-20T22:11:46.0402870Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-20T22:11:46.1168261Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-20T22:11:46.1505562Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-20T22:11:46.1525802Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:11:46.1557284Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-20T22:11:46.3652764Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-20T22:11:46.3702720Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-20T22:11:46.4066690Z Entering 'android/libs/fbjni' 2024-08-20T22:11:46.4119640Z Entering 'third_party/FP16' 2024-08-20T22:11:46.4171621Z Entering 'third_party/FXdiv' 2024-08-20T22:11:46.4225212Z Entering 'third_party/NNPACK' 2024-08-20T22:11:46.4277973Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:11:46.4330723Z Entering 'third_party/XNNPACK' 2024-08-20T22:11:46.4398093Z Entering 'third_party/benchmark' 2024-08-20T22:11:46.4450051Z Entering 'third_party/cpp-httplib' 2024-08-20T22:11:46.4501226Z Entering 'third_party/cpuinfo' 2024-08-20T22:11:46.4555081Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:11:46.4607397Z Entering 'third_party/cutlass' 2024-08-20T22:11:46.4667239Z Entering 'third_party/eigen' 2024-08-20T22:11:46.4725113Z Entering 'third_party/fbgemm' 2024-08-20T22:11:46.4780937Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:11:46.4832097Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:11:46.4882062Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:11:46.4939607Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:11:46.4988994Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:11:46.5043764Z Entering 'third_party/flatbuffers' 2024-08-20T22:11:46.5098951Z Entering 'third_party/fmt' 2024-08-20T22:11:46.5151985Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:11:46.5203748Z Entering 'third_party/gloo' 2024-08-20T22:11:46.5262230Z Entering 'third_party/googletest' 2024-08-20T22:11:46.5314539Z Entering 'third_party/ideep' 2024-08-20T22:11:46.5367980Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:11:46.5427239Z Entering 'third_party/ittapi' 2024-08-20T22:11:46.5478706Z Entering 'third_party/kineto' 2024-08-20T22:11:46.5531814Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:11:46.5582429Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:11:46.5634610Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:11:46.5686203Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:11:46.5742056Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:11:46.5790503Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:11:46.5845733Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:11:46.5896974Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:11:46.5949177Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:11:46.6001496Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:11:46.6062500Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:11:46.6114209Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:11:46.6165917Z Entering 'third_party/mimalloc' 2024-08-20T22:11:46.6222641Z Entering 'third_party/nccl/nccl' 2024-08-20T22:11:46.6276587Z Entering 'third_party/nlohmann' 2024-08-20T22:11:46.6332781Z Entering 'third_party/onnx' 2024-08-20T22:11:46.6399108Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:11:46.6450954Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:11:46.6506891Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:11:46.6560713Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:11:46.6611788Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:11:46.6659743Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:11:46.6708396Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:11:46.6758663Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:11:46.6806500Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:11:46.6857538Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:11:46.6906173Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:11:46.6964609Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:11:46.7019080Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:11:46.7092574Z Entering 'third_party/pocketfft' 2024-08-20T22:11:46.7146182Z Entering 'third_party/protobuf' 2024-08-20T22:11:46.7200129Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:11:46.7252704Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:11:46.7306583Z Entering 'third_party/psimd' 2024-08-20T22:11:46.7358965Z Entering 'third_party/pthreadpool' 2024-08-20T22:11:46.7416090Z Entering 'third_party/pybind11' 2024-08-20T22:11:46.7468575Z Entering 'third_party/python-peachpy' 2024-08-20T22:11:46.7519637Z Entering 'third_party/sleef' 2024-08-20T22:11:46.7572491Z Entering 'third_party/tensorpipe' 2024-08-20T22:11:46.7624010Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:11:46.7674012Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:11:46.7724118Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:11:46.7773828Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:11:46.7827191Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:11:46.7900193Z ##[endgroup] 2024-08-20T22:11:46.7902503Z ##[group]Persisting credentials for submodules 2024-08-20T22:11:46.7905106Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-20T22:11:46.8272894Z Entering 'android/libs/fbjni' 2024-08-20T22:11:46.8339847Z Entering 'third_party/FP16' 2024-08-20T22:11:46.8409726Z Entering 'third_party/FXdiv' 2024-08-20T22:11:46.8479612Z Entering 'third_party/NNPACK' 2024-08-20T22:11:46.8548575Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:11:46.8616474Z Entering 'third_party/XNNPACK' 2024-08-20T22:11:46.8698346Z Entering 'third_party/benchmark' 2024-08-20T22:11:46.8765457Z Entering 'third_party/cpp-httplib' 2024-08-20T22:11:46.8833120Z Entering 'third_party/cpuinfo' 2024-08-20T22:11:46.8900987Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:11:46.8968299Z Entering 'third_party/cutlass' 2024-08-20T22:11:46.9042655Z Entering 'third_party/eigen' 2024-08-20T22:11:46.9112330Z Entering 'third_party/fbgemm' 2024-08-20T22:11:46.9178555Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:11:46.9248748Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:11:46.9314987Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:11:46.9386860Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:11:46.9452816Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:11:46.9522377Z Entering 'third_party/flatbuffers' 2024-08-20T22:11:46.9594124Z Entering 'third_party/fmt' 2024-08-20T22:11:46.9661957Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:11:46.9731882Z Entering 'third_party/gloo' 2024-08-20T22:11:46.9799532Z Entering 'third_party/googletest' 2024-08-20T22:11:46.9869408Z Entering 'third_party/ideep' 2024-08-20T22:11:46.9935247Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:11:47.0009304Z Entering 'third_party/ittapi' 2024-08-20T22:11:47.0076617Z Entering 'third_party/kineto' 2024-08-20T22:11:47.0142969Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:11:47.0206762Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:11:47.0276537Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:11:47.0343992Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:11:47.0412852Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:11:47.0477400Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:11:47.0549195Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:11:47.0619251Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:11:47.0687541Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:11:47.0756493Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:11:47.0826727Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:11:47.0893228Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:11:47.0962845Z Entering 'third_party/mimalloc' 2024-08-20T22:11:47.1031132Z Entering 'third_party/nccl/nccl' 2024-08-20T22:11:47.1098546Z Entering 'third_party/nlohmann' 2024-08-20T22:11:47.1168166Z Entering 'third_party/onnx' 2024-08-20T22:11:47.1248264Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:11:47.1315836Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:11:47.1391471Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:11:47.1459466Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:11:47.1525538Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:11:47.1592861Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:11:47.1660198Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:11:47.1729662Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:11:47.1797201Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:11:47.1864350Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:11:47.1928970Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:11:47.1996590Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:11:47.2067838Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:11:47.2154425Z Entering 'third_party/pocketfft' 2024-08-20T22:11:47.2222835Z Entering 'third_party/protobuf' 2024-08-20T22:11:47.2292191Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:11:47.2361942Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:11:47.2432942Z Entering 'third_party/psimd' 2024-08-20T22:11:47.2500212Z Entering 'third_party/pthreadpool' 2024-08-20T22:11:47.2568059Z Entering 'third_party/pybind11' 2024-08-20T22:11:47.2637392Z Entering 'third_party/python-peachpy' 2024-08-20T22:11:47.2706651Z Entering 'third_party/sleef' 2024-08-20T22:11:47.2775672Z Entering 'third_party/tensorpipe' 2024-08-20T22:11:47.2841994Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:11:47.2907567Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:11:47.2972539Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:11:47.3040265Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:11:47.3102871Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:11:47.3195272Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-20T22:11:47.3556080Z Entering 'android/libs/fbjni' 2024-08-20T22:11:47.3623938Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-20T22:11:47.3645012Z Entering 'third_party/FP16' 2024-08-20T22:11:47.3708836Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-20T22:11:47.3731761Z Entering 'third_party/FXdiv' 2024-08-20T22:11:47.3793003Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-20T22:11:47.3813881Z Entering 'third_party/NNPACK' 2024-08-20T22:11:47.3876586Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-20T22:11:47.3899270Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:11:47.3961975Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-20T22:11:47.3983653Z Entering 'third_party/XNNPACK' 2024-08-20T22:11:47.4046298Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-20T22:11:47.4081741Z Entering 'third_party/benchmark' 2024-08-20T22:11:47.4144791Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:11:47.4165981Z Entering 'third_party/cpp-httplib' 2024-08-20T22:11:47.4233016Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-20T22:11:47.4254358Z Entering 'third_party/cpuinfo' 2024-08-20T22:11:47.4315793Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:11:47.4337657Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:11:47.4403636Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-20T22:11:47.4425320Z Entering 'third_party/cutlass' 2024-08-20T22:11:47.4486878Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:11:47.4516396Z Entering 'third_party/eigen' 2024-08-20T22:11:47.4578166Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-20T22:11:47.4601214Z Entering 'third_party/fbgemm' 2024-08-20T22:11:47.4664278Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-20T22:11:47.4683764Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:11:47.4746445Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-20T22:11:47.4766756Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:11:47.4829339Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:11:47.4850255Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:11:47.4912213Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:11:47.4942353Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:11:47.5002777Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:11:47.5023956Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:11:47.5085031Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-20T22:11:47.5109095Z Entering 'third_party/flatbuffers' 2024-08-20T22:11:47.5169990Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-20T22:11:47.5194032Z Entering 'third_party/fmt' 2024-08-20T22:11:47.5260140Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:11:47.5281670Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:11:47.5345981Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-20T22:11:47.5367530Z Entering 'third_party/gloo' 2024-08-20T22:11:47.5428011Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-20T22:11:47.5448922Z Entering 'third_party/googletest' 2024-08-20T22:11:47.5509942Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:11:47.5531384Z Entering 'third_party/ideep' 2024-08-20T22:11:47.5591319Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-20T22:11:47.5610620Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:11:47.5671781Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-20T22:11:47.5700777Z Entering 'third_party/ittapi' 2024-08-20T22:11:47.5761617Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-20T22:11:47.5782687Z Entering 'third_party/kineto' 2024-08-20T22:11:47.5847754Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-20T22:11:47.5867399Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:11:47.5931013Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-20T22:11:47.5950552Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:11:47.6014456Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-20T22:11:47.6036205Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:11:47.6097770Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-20T22:11:47.6120133Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:11:47.6181415Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:11:47.6201929Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:11:47.6264684Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-20T22:11:47.6282768Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:11:47.6346316Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-20T22:11:47.6369596Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:11:47.6432512Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-20T22:11:47.6453205Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:11:47.6516040Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:11:47.6536975Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:11:47.6599324Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-20T22:11:47.6620128Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:11:47.6682061Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-20T22:11:47.6705798Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:11:47.6767457Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-20T22:11:47.6789832Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:11:47.6851596Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-20T22:11:47.6874219Z Entering 'third_party/mimalloc' 2024-08-20T22:11:47.6934597Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-20T22:11:47.6956000Z Entering 'third_party/nccl/nccl' 2024-08-20T22:11:47.7028272Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-20T22:11:47.7049555Z Entering 'third_party/nlohmann' 2024-08-20T22:11:47.7110398Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-20T22:11:47.7132976Z Entering 'third_party/onnx' 2024-08-20T22:11:47.7197166Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-20T22:11:47.7236771Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:11:47.7298797Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:11:47.7320466Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:11:47.7383054Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:11:47.7409722Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:11:47.7472222Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-20T22:11:47.7493423Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:11:47.7555796Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:11:47.7576791Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:11:47.7638775Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:11:47.7659583Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:11:47.7720366Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-20T22:11:47.7740706Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:11:47.7802951Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-20T22:11:47.7824108Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:11:47.7884907Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-20T22:11:47.7905991Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:11:47.7968290Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-20T22:11:47.7988831Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:11:47.8053986Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-20T22:11:47.8072307Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:11:47.8134384Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-20T22:11:47.8156250Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:11:47.8219387Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-20T22:11:47.8242250Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:11:47.8303509Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-20T22:11:47.8345985Z Entering 'third_party/pocketfft' 2024-08-20T22:11:47.8406879Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-20T22:11:47.8427989Z Entering 'third_party/protobuf' 2024-08-20T22:11:47.8489902Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-20T22:11:47.8512779Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:11:47.8573608Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:11:47.8593645Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:11:47.8654759Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:11:47.8678791Z Entering 'third_party/psimd' 2024-08-20T22:11:47.8743017Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-20T22:11:47.8763776Z Entering 'third_party/pthreadpool' 2024-08-20T22:11:47.8825083Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-20T22:11:47.8845808Z Entering 'third_party/pybind11' 2024-08-20T22:11:47.8910389Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:11:47.8932084Z Entering 'third_party/python-peachpy' 2024-08-20T22:11:47.8992900Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-20T22:11:47.9014154Z Entering 'third_party/sleef' 2024-08-20T22:11:47.9074388Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-20T22:11:47.9095017Z Entering 'third_party/tensorpipe' 2024-08-20T22:11:47.9157999Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-20T22:11:47.9179537Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:11:47.9241993Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:11:47.9261878Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:11:47.9322483Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-20T22:11:47.9342406Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:11:47.9403511Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-20T22:11:47.9424622Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:11:47.9485254Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:11:47.9503519Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:11:47.9567191Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-20T22:11:48.0392844Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-20T22:11:48.0758660Z Entering 'android/libs/fbjni' 2024-08-20T22:11:48.0810638Z Entering 'third_party/FP16' 2024-08-20T22:11:48.0863531Z Entering 'third_party/FXdiv' 2024-08-20T22:11:48.0916009Z Entering 'third_party/NNPACK' 2024-08-20T22:11:48.0970365Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:11:48.1022928Z Entering 'third_party/XNNPACK' 2024-08-20T22:11:48.1091381Z Entering 'third_party/benchmark' 2024-08-20T22:11:48.1143759Z Entering 'third_party/cpp-httplib' 2024-08-20T22:11:48.1195563Z Entering 'third_party/cpuinfo' 2024-08-20T22:11:48.1249681Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:11:48.1300929Z Entering 'third_party/cutlass' 2024-08-20T22:11:48.1363351Z Entering 'third_party/eigen' 2024-08-20T22:11:48.1416715Z Entering 'third_party/fbgemm' 2024-08-20T22:11:48.1468310Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:11:48.1523261Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:11:48.1574326Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:11:48.1635714Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:11:48.1690945Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:11:48.1743464Z Entering 'third_party/flatbuffers' 2024-08-20T22:11:48.1797262Z Entering 'third_party/fmt' 2024-08-20T22:11:48.1849471Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:11:48.1899912Z Entering 'third_party/gloo' 2024-08-20T22:11:48.1954339Z Entering 'third_party/googletest' 2024-08-20T22:11:48.2004936Z Entering 'third_party/ideep' 2024-08-20T22:11:48.2056146Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:11:48.2113938Z Entering 'third_party/ittapi' 2024-08-20T22:11:48.2166650Z Entering 'third_party/kineto' 2024-08-20T22:11:48.2218692Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:11:48.2267726Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:11:48.2320428Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:11:48.2371131Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:11:48.2422108Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:11:48.2471390Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:11:48.2526009Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:11:48.2579702Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:11:48.2632901Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:11:48.2686368Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:11:48.2740118Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:11:48.2790225Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:11:48.2848085Z Entering 'third_party/mimalloc' 2024-08-20T22:11:48.2900725Z Entering 'third_party/nccl/nccl' 2024-08-20T22:11:48.2955560Z Entering 'third_party/nlohmann' 2024-08-20T22:11:48.3009507Z Entering 'third_party/onnx' 2024-08-20T22:11:48.3075310Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:11:48.3131087Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:11:48.3187070Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:11:48.3242623Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:11:48.3292225Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:11:48.3344292Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:11:48.3393883Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:11:48.3453647Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:11:48.3505026Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:11:48.3554607Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:11:48.3602912Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:11:48.3659090Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:11:48.3713129Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:11:48.3790701Z Entering 'third_party/pocketfft' 2024-08-20T22:11:48.3845358Z Entering 'third_party/protobuf' 2024-08-20T22:11:48.3899936Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:11:48.3950698Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:11:48.4004987Z Entering 'third_party/psimd' 2024-08-20T22:11:48.4057679Z Entering 'third_party/pthreadpool' 2024-08-20T22:11:48.4109492Z Entering 'third_party/pybind11' 2024-08-20T22:11:48.4160199Z Entering 'third_party/python-peachpy' 2024-08-20T22:11:48.4212075Z Entering 'third_party/sleef' 2024-08-20T22:11:48.4263032Z Entering 'third_party/tensorpipe' 2024-08-20T22:11:48.4313360Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:11:48.4364802Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:11:48.4415067Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:11:48.4465413Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:11:48.4514072Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:11:48.4590198Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-20T22:11:48.4957521Z Entering 'android/libs/fbjni' 2024-08-20T22:11:48.5010419Z Entering 'third_party/FP16' 2024-08-20T22:11:48.5061742Z Entering 'third_party/FXdiv' 2024-08-20T22:11:48.5113232Z Entering 'third_party/NNPACK' 2024-08-20T22:11:48.5164829Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:11:48.5217130Z Entering 'third_party/XNNPACK' 2024-08-20T22:11:48.5283874Z Entering 'third_party/benchmark' 2024-08-20T22:11:48.5336335Z Entering 'third_party/cpp-httplib' 2024-08-20T22:11:48.5387404Z Entering 'third_party/cpuinfo' 2024-08-20T22:11:48.5441035Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:11:48.5492242Z Entering 'third_party/cutlass' 2024-08-20T22:11:48.5551909Z Entering 'third_party/eigen' 2024-08-20T22:11:48.5605268Z Entering 'third_party/fbgemm' 2024-08-20T22:11:48.5677624Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:11:48.5734048Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:11:48.5784879Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:11:48.5844806Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:11:48.5895229Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:11:48.5949235Z Entering 'third_party/flatbuffers' 2024-08-20T22:11:48.6002982Z Entering 'third_party/fmt' 2024-08-20T22:11:48.6055405Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:11:48.6107830Z Entering 'third_party/gloo' 2024-08-20T22:11:48.6159345Z Entering 'third_party/googletest' 2024-08-20T22:11:48.6211809Z Entering 'third_party/ideep' 2024-08-20T22:11:48.6261314Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:11:48.6319706Z Entering 'third_party/ittapi' 2024-08-20T22:11:48.6369986Z Entering 'third_party/kineto' 2024-08-20T22:11:48.6419655Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:11:48.6470483Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:11:48.6521578Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:11:48.6572515Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:11:48.6629841Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:11:48.6678919Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:11:48.6734063Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:11:48.6784694Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:11:48.6838101Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:11:48.6890873Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:11:48.6945422Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:11:48.6995589Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:11:48.7054710Z Entering 'third_party/mimalloc' 2024-08-20T22:11:48.7106499Z Entering 'third_party/nccl/nccl' 2024-08-20T22:11:48.7159401Z Entering 'third_party/nlohmann' 2024-08-20T22:11:48.7212740Z Entering 'third_party/onnx' 2024-08-20T22:11:48.7277182Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:11:48.7333037Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:11:48.7388908Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:11:48.7442094Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:11:48.7491683Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:11:48.7543606Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:11:48.7593790Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:11:48.7650492Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:11:48.7700032Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:11:48.7750670Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:11:48.7799889Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:11:48.7851989Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:11:48.7906806Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:11:48.7984446Z Entering 'third_party/pocketfft' 2024-08-20T22:11:48.8036934Z Entering 'third_party/protobuf' 2024-08-20T22:11:48.8089980Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:11:48.8140727Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:11:48.8194424Z Entering 'third_party/psimd' 2024-08-20T22:11:48.8247314Z Entering 'third_party/pthreadpool' 2024-08-20T22:11:48.8298885Z Entering 'third_party/pybind11' 2024-08-20T22:11:48.8351129Z Entering 'third_party/python-peachpy' 2024-08-20T22:11:48.8402441Z Entering 'third_party/sleef' 2024-08-20T22:11:48.8455902Z Entering 'third_party/tensorpipe' 2024-08-20T22:11:48.8506358Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:11:48.8556348Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:11:48.8607161Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:11:48.8658579Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:11:48.8706792Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:11:48.8779760Z ##[endgroup] 2024-08-20T22:11:48.8829037Z [command]/usr/bin/git log -1 --format='%H' 2024-08-20T22:11:48.8866165Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-20T22:11:48.9086666Z Prepare all required actions 2024-08-20T22:11:48.9087208Z Getting action download info 2024-08-20T22:11:49.0838393Z ##[group]Run ./.github/actions/setup-linux 2024-08-20T22:11:49.0838808Z env: 2024-08-20T22:11:49.0839086Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:49.0839417Z ##[endgroup] 2024-08-20T22:11:49.0898243Z ##[group]Run set -euo pipefail 2024-08-20T22:11:49.0898682Z set -euo pipefail 2024-08-20T22:11:49.0899099Z function get_ec2_metadata() { 2024-08-20T22:11:49.0899634Z  # Pulled from instance metadata endpoint for EC2 2024-08-20T22:11:49.0900511Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-20T22:11:49.0901443Z  category=$1 2024-08-20T22:11:49.0901970Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-20T22:11:49.0902600Z  runner_name_str=i-0ab7c6e949d6545da 2024-08-20T22:11:49.0903088Z  if [[ -f /.inarc ]]; then 2024-08-20T22:11:49.0903578Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-20T22:11:49.0904139Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-20T22:11:49.0905169Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-20T22:11:49.0905771Z  else 2024-08-20T22:11:49.0906276Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-20T22:11:49.0906948Z  fi 2024-08-20T22:11:49.0907244Z } 2024-08-20T22:11:49.0907605Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-20T22:11:49.0908203Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-20T22:11:49.0908855Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-20T22:11:49.0909427Z echo "system info $(uname -a)" 2024-08-20T22:11:49.0921516Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:49.0922022Z env: 2024-08-20T22:11:49.0922313Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:49.0922660Z ##[endgroup] 2024-08-20T22:11:49.1031127Z ami-id: ami-06c68f701d8090592 2024-08-20T22:11:49.1094543Z instance-id: i-0ab7c6e949d6545da 2024-08-20T22:11:49.1155148Z instance-type: g5.4xlarge 2024-08-20T22:11:49.1169543Z system info Linux ip-10-0-37-113.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-20T22:11:49.1195543Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:11:49.1196452Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:11:49.1206133Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:49.1206650Z env: 2024-08-20T22:11:49.1206972Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:49.1207413Z ##[endgroup] 2024-08-20T22:11:49.1284651Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-20T22:11:49.1285259Z if systemctl is-active --quiet docker; then 2024-08-20T22:11:49.1285800Z  echo "Docker daemon is running..."; 2024-08-20T22:11:49.1286257Z else 2024-08-20T22:11:49.1286749Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-20T22:11:49.1287482Z fi 2024-08-20T22:11:49.1296324Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:49.1296815Z env: 2024-08-20T22:11:49.1297098Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:49.1297448Z ##[endgroup] 2024-08-20T22:11:49.1388124Z Docker daemon is running... 2024-08-20T22:11:49.1436843Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:11:49.1437448Z with: 2024-08-20T22:11:49.1437712Z shell: bash 2024-08-20T22:11:49.1438001Z timeout_minutes: 5 2024-08-20T22:11:49.1438319Z max_attempts: 3 2024-08-20T22:11:49.1438631Z retry_wait_seconds: 30 2024-08-20T22:11:49.1441695Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-20T22:11:49.1444623Z polling_interval_seconds: 1 2024-08-20T22:11:49.1445132Z warning_on_retry: true 2024-08-20T22:11:49.1445481Z continue_on_error: false 2024-08-20T22:11:49.1445815Z env: 2024-08-20T22:11:49.1446087Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:49.1446440Z AWS_RETRY_MODE: standard 2024-08-20T22:11:49.1446781Z AWS_MAX_ATTEMPTS: 5 2024-08-20T22:11:49.1447195Z AWS_DEFAULT_REGION: us-east-1 2024-08-20T22:11:49.1447574Z ##[endgroup] 2024-08-20T22:11:50.3031106Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:11:50.3031969Z Configure a credential helper to remove this warning. See 2024-08-20T22:11:50.3032822Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:11:50.3033341Z 2024-08-20T22:11:50.3034830Z Login Succeeded 2024-08-20T22:11:51.2008762Z Command completed after 1 attempt(s). 2024-08-20T22:11:51.2074815Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:11:51.2075575Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:11:51.2076275Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:11:51.2086559Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:51.2087057Z env: 2024-08-20T22:11:51.2087460Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:51.2087812Z ##[endgroup] 2024-08-20T22:11:51.2189851Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:11:51.2190608Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:11:51.2191196Z # shellcheck disable=SC2046 2024-08-20T22:11:51.2191655Z docker stop $(docker ps -q) || true 2024-08-20T22:11:51.2192128Z # Prune all of the docker images 2024-08-20T22:11:51.2192575Z docker system prune -af 2024-08-20T22:11:51.2201489Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:51.2201982Z env: 2024-08-20T22:11:51.2202251Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:51.2202602Z ##[endgroup] 2024-08-20T22:11:51.2522551Z "docker stop" requires at least 1 argument. 2024-08-20T22:11:51.2523339Z See 'docker stop --help'. 2024-08-20T22:11:51.2523650Z 2024-08-20T22:11:51.2523959Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-20T22:11:51.2524441Z 2024-08-20T22:11:51.2524652Z Stop one or more running containers 2024-08-20T22:11:51.2936949Z Total reclaimed space: 0B 2024-08-20T22:11:51.2987255Z ##[group]Run set +e 2024-08-20T22:11:51.2987612Z set +e 2024-08-20T22:11:51.2987915Z set -x 2024-08-20T22:11:51.2988209Z  2024-08-20T22:11:51.2988527Z PT_DOMAIN=download.pytorch.org 2024-08-20T22:11:51.2989300Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-20T22:11:51.2990362Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-20T22:11:51.2991099Z # one is returned at random 2024-08-20T22:11:51.2991655Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-20T22:11:51.2992158Z  2024-08-20T22:11:51.2992479Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:11:51.2993101Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-20T22:11:51.2994048Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-20T22:11:51.2994601Z  2024-08-20T22:11:51.2994930Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:11:51.2995476Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-20T22:11:51.2995982Z  exit 1 2024-08-20T22:11:51.2996287Z  fi 2024-08-20T22:11:51.2996574Z fi 2024-08-20T22:11:51.2996853Z  2024-08-20T22:11:51.2997212Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-20T22:11:51.2997723Z  # Clean up any old records first 2024-08-20T22:11:51.2998227Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-20T22:11:51.2998829Z fi 2024-08-20T22:11:51.2999100Z  2024-08-20T22:11:51.2999530Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-20T22:11:51.3000068Z cat /etc/hosts 2024-08-20T22:11:51.3009599Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:51.3010089Z env: 2024-08-20T22:11:51.3010376Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:51.3010719Z ##[endgroup] 2024-08-20T22:11:51.3040632Z + PT_DOMAIN=download.pytorch.org 2024-08-20T22:11:51.3047455Z ++ dig -4 +short download.pytorch.org 2024-08-20T22:11:51.3048296Z ++ tail -n1 2024-08-20T22:11:51.3349501Z + RESOLVED_IP=18.160.10.28 2024-08-20T22:11:51.3350133Z + '[' -z 18.160.10.28 ']' 2024-08-20T22:11:51.3350584Z + grep -r download.pytorch.org /etc/hosts 2024-08-20T22:11:51.3369278Z + echo '18.160.10.28 download.pytorch.org' 2024-08-20T22:11:51.3370370Z + sudo tee -a /etc/hosts 2024-08-20T22:11:51.6202964Z 18.160.10.28 download.pytorch.org 2024-08-20T22:11:51.6224898Z + cat /etc/hosts 2024-08-20T22:11:51.6235694Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-20T22:11:51.6248138Z ::1 localhost6 localhost6.localdomain6 2024-08-20T22:11:51.6248646Z 18.160.10.28 download.pytorch.org 2024-08-20T22:11:51.6400907Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-20T22:11:51.6401520Z with: 2024-08-20T22:11:51.6402605Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:51.6403810Z docker-build-dir: .ci/docker 2024-08-20T22:11:51.6404189Z working-directory: . 2024-08-20T22:11:51.6404902Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:51.6405433Z force-push: false 2024-08-20T22:11:51.6405738Z env: 2024-08-20T22:11:51.6406013Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:51.6406349Z ##[endgroup] 2024-08-20T22:11:51.6425185Z ##[group]Run set -ex 2024-08-20T22:11:51.6425546Z set -ex 2024-08-20T22:11:51.6425834Z  2024-08-20T22:11:51.6426385Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-20T22:11:51.6427343Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-20T22:11:51.6428127Z # job could then download the pre-built image as usual 2024-08-20T22:11:51.6428860Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-20T22:11:51.6429525Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:51.6430155Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:51.6430710Z  2024-08-20T22:11:51.6431213Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-20T22:11:51.6431834Z  exit 0 2024-08-20T22:11:51.6432142Z else 2024-08-20T22:11:51.6432512Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:51.6432965Z fi 2024-08-20T22:11:51.6433252Z  2024-08-20T22:11:51.6433722Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-20T22:11:51.6434552Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-20T22:11:51.6435319Z  # use it as it is, but first let's extract the tag 2024-08-20T22:11:51.6436013Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-20T22:11:51.6436733Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:51.6437424Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:51.6437972Z else 2024-08-20T22:11:51.6438407Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-20T22:11:51.6439066Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:51.6440112Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:51.6440840Z fi 2024-08-20T22:11:51.6454622Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:51.6455120Z env: 2024-08-20T22:11:51.6455400Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:51.6455765Z REPO_NAME: pytorch 2024-08-20T22:11:51.6456888Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:51.6458067Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:11:51.6458562Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:51.6459073Z ##[endgroup] 2024-08-20T22:11:51.6488654Z + [[ ! -d .ci/docker ]] 2024-08-20T22:11:51.6489170Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-20T22:11:51.6489554Z + echo skip=false 2024-08-20T22:11:51.6491566Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-20T22:11:51.6499024Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:51.6500229Z ++ awk -F '[:,]' '{print $2}' 2024-08-20T22:11:51.6525289Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:51.6526019Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:51.6527601Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:51.6560882Z ##[group]Run set +e 2024-08-20T22:11:51.6561369Z set +e 2024-08-20T22:11:51.6561846Z set -x 2024-08-20T22:11:51.6562198Z  2024-08-20T22:11:51.6562569Z login() { 2024-08-20T22:11:51.6563389Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:11:51.6564191Z } 2024-08-20T22:11:51.6564551Z  2024-08-20T22:11:51.6564972Z retry () { 2024-08-20T22:11:51.6565462Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:11:51.6565986Z } 2024-08-20T22:11:51.6566413Z  2024-08-20T22:11:51.6566819Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:11:51.6567513Z  2024-08-20T22:11:51.6568050Z # Check if image already exists, if it does then skip building it 2024-08-20T22:11:51.6568841Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-20T22:11:51.6569512Z  exit 0 2024-08-20T22:11:51.6569906Z fi 2024-08-20T22:11:51.6570265Z  2024-08-20T22:11:51.6570911Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-20T22:11:51.6571821Z # be empty. The default action would be to continue rebuild the image 2024-08-20T22:11:51.6572665Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-20T22:11:51.6573452Z  # if we're on the base branch then use the parent commit 2024-08-20T22:11:51.6574134Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-20T22:11:51.6574674Z else 2024-08-20T22:11:51.6575284Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-20T22:11:51.6576095Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-20T22:11:51.6576690Z fi 2024-08-20T22:11:51.6577078Z  2024-08-20T22:11:51.6577518Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-20T22:11:51.6578119Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:51.6578715Z  2024-08-20T22:11:51.6579651Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-20T22:11:51.6580494Z  exit 0 2024-08-20T22:11:51.6580967Z fi 2024-08-20T22:11:51.6581339Z  2024-08-20T22:11:51.6581836Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-20T22:11:51.6582958Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-20T22:11:51.6583841Z  exit 1 2024-08-20T22:11:51.6584304Z fi 2024-08-20T22:11:51.6584640Z  2024-08-20T22:11:51.6585214Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-20T22:11:51.6586306Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-20T22:11:51.6587260Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-20T22:11:51.6588316Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-20T22:11:51.6589583Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-20T22:11:51.6590467Z fi 2024-08-20T22:11:51.6590866Z  2024-08-20T22:11:51.6591365Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:11:51.6599844Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:51.6600497Z env: 2024-08-20T22:11:51.6600866Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:51.6601348Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:11:51.6601928Z BASE_REVISION: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:11:51.6603302Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:51.6604947Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:51.6605761Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:51.6606356Z ##[endgroup] 2024-08-20T22:11:51.6635736Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:51.6636490Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:51.6638703Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:11:51.6640200Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:52.2074435Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:11:52.2076455Z Configure a credential helper to remove this warning. See 2024-08-20T22:11:52.2078424Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:11:52.2079348Z 2024-08-20T22:11:52.2079498Z Login Succeeded 2024-08-20T22:11:52.2098356Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:52.4255125Z { 2024-08-20T22:11:52.4255675Z "schemaVersion": 2, 2024-08-20T22:11:52.4256730Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-20T22:11:52.4258530Z "config": { 2024-08-20T22:11:52.4259265Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-20T22:11:52.4260407Z "size": 49627, 2024-08-20T22:11:52.4261588Z "digest": "sha256:cee8d41a96d3e7eccb178c1529633c2dd768035bec694588cf53029b783ac292" 2024-08-20T22:11:52.4262962Z }, 2024-08-20T22:11:52.4263364Z "layers": [ 2024-08-20T22:11:52.4264653Z { 2024-08-20T22:11:52.4265457Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4266311Z "size": 28584317, 2024-08-20T22:11:52.4267359Z "digest": "sha256:63e9bbe323274e77e58d77c6ab6802d247458f784222fbb07a2556d6ec74ee05" 2024-08-20T22:11:52.4269138Z }, 2024-08-20T22:11:52.4269533Z { 2024-08-20T22:11:52.4270419Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4271109Z "size": 7944771, 2024-08-20T22:11:52.4272222Z "digest": "sha256:cfb3d849840ee60cee7b02bad68c1fc3c15928ebcd88f327754766b670578ed6" 2024-08-20T22:11:52.4273144Z }, 2024-08-20T22:11:52.4273680Z { 2024-08-20T22:11:52.4274426Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4275043Z "size": 57593718, 2024-08-20T22:11:52.4276541Z "digest": "sha256:968831e596a6288f0fed9b8612ee4ee8e75511037c4305058805492c5162e481" 2024-08-20T22:11:52.4277262Z }, 2024-08-20T22:11:52.4278968Z { 2024-08-20T22:11:52.4279933Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4280901Z "size": 187, 2024-08-20T22:11:52.4281807Z "digest": "sha256:ea310eb267ca1cab61b6b16f566cd28bfd59a741395a011f5e76716e15ba57c6" 2024-08-20T22:11:52.4282698Z }, 2024-08-20T22:11:52.4283005Z { 2024-08-20T22:11:52.4284320Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4285017Z "size": 6885, 2024-08-20T22:11:52.4285782Z "digest": "sha256:3af11d09e9cd1eb9c379f0a4071231e5a5642eb728b4b33bcb76be291f3c9488" 2024-08-20T22:11:52.4286782Z }, 2024-08-20T22:11:52.4287582Z { 2024-08-20T22:11:52.4288518Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4289274Z "size": 1361380219, 2024-08-20T22:11:52.4289963Z "digest": "sha256:ebfec18059b91e56882881ac34754f917861edb5f732c395d2a1a851bbd6db46" 2024-08-20T22:11:52.4290865Z + exit 0 2024-08-20T22:11:52.4291286Z }, 2024-08-20T22:11:52.4291590Z { 2024-08-20T22:11:52.4292130Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4292819Z "size": 62686, 2024-08-20T22:11:52.4293478Z "digest": "sha256:533b4aebf16914c763b7b0de3ce657590c6f979045e9fdf1f816adaf68d8a4d3" 2024-08-20T22:11:52.4294147Z }, 2024-08-20T22:11:52.4294544Z { 2024-08-20T22:11:52.4295093Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4295728Z "size": 1685, 2024-08-20T22:11:52.4296369Z "digest": "sha256:9dd75d06a0910f28cb1e484b8808724e5a6ee570ecb8fc04631368f546b39ed9" 2024-08-20T22:11:52.4297087Z }, 2024-08-20T22:11:52.4297433Z { 2024-08-20T22:11:52.4297967Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4298629Z "size": 1523, 2024-08-20T22:11:52.4299306Z "digest": "sha256:30bfca4dd3492d60ed8035b0eeb1229897041140db117f5663465a551e25851d" 2024-08-20T22:11:52.4300074Z }, 2024-08-20T22:11:52.4300380Z { 2024-08-20T22:11:52.4300899Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4301623Z "size": 2608186849, 2024-08-20T22:11:52.4302279Z "digest": "sha256:1b57ce94cad9a5097d99b8d6f7bcd51df5a162fc3c5f5686b689b76993724bc8" 2024-08-20T22:11:52.4302939Z }, 2024-08-20T22:11:52.4303344Z { 2024-08-20T22:11:52.4303859Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4304523Z "size": 86631, 2024-08-20T22:11:52.4305553Z "digest": "sha256:9ee6bdb31195dd42fd98147a75d540efe0c5708e0ecda866a0bca060084fddab" 2024-08-20T22:11:52.4306736Z }, 2024-08-20T22:11:52.4307270Z { 2024-08-20T22:11:52.4307837Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4308479Z "size": 1823, 2024-08-20T22:11:52.4309130Z "digest": "sha256:722a6699b4ebdf1fff415451da7fd768fe9423762ae79022b3ef197dd336aba1" 2024-08-20T22:11:52.4309885Z }, 2024-08-20T22:11:52.4310184Z { 2024-08-20T22:11:52.4310722Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4311414Z "size": 245783893, 2024-08-20T22:11:52.4312072Z "digest": "sha256:164218df90f4c2138aa8523cb457e541c046b1165679e431cd9de868b53f2f64" 2024-08-20T22:11:52.4312722Z }, 2024-08-20T22:11:52.4313112Z { 2024-08-20T22:11:52.4313649Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4314516Z "size": 545, 2024-08-20T22:11:52.4315102Z "digest": "sha256:f1d74e82c83914108a9f58ec156ba74104629fffbc1d62bfd5f64a5d6de00338" 2024-08-20T22:11:52.4315821Z }, 2024-08-20T22:11:52.4316208Z { 2024-08-20T22:11:52.4316694Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4317348Z "size": 1259, 2024-08-20T22:11:52.4318013Z "digest": "sha256:13b68a95b298f6875179a04030c62eb4a361b69cac013519115ae86fa102a98a" 2024-08-20T22:11:52.4318712Z }, 2024-08-20T22:11:52.4319011Z { 2024-08-20T22:11:52.4319574Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4320228Z "size": 484, 2024-08-20T22:11:52.4320848Z "digest": "sha256:c95fa6ffe9908fff217023af77b2359f99f9fb5c01519080d04e280191c354ab" 2024-08-20T22:11:52.4321558Z }, 2024-08-20T22:11:52.4321914Z { 2024-08-20T22:11:52.4322434Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4323080Z "size": 91712634, 2024-08-20T22:11:52.4323757Z "digest": "sha256:831d1d1ad5de706debd5baab20657e9694be7412d6d7f775460ad00c5c5bd203" 2024-08-20T22:11:52.4324462Z }, 2024-08-20T22:11:52.4324870Z { 2024-08-20T22:11:52.4325352Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4326123Z "size": 3392, 2024-08-20T22:11:52.4326835Z "digest": "sha256:3116a975d40f2c45dfc323466942aca5d68ba4d81049fb90fb4f9ca02475aca1" 2024-08-20T22:11:52.4327651Z }, 2024-08-20T22:11:52.4327970Z { 2024-08-20T22:11:52.4328546Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4329179Z "size": 1910, 2024-08-20T22:11:52.4329877Z "digest": "sha256:aed1c44ec0f677f20a8336422625eec3f66558ecbc61b9e49856085188b117e9" 2024-08-20T22:11:52.4330589Z }, 2024-08-20T22:11:52.4330924Z { 2024-08-20T22:11:52.4331462Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4332115Z "size": 700, 2024-08-20T22:11:52.4332747Z "digest": "sha256:657a0885dc72f558a3fdaffb85a7d85c2ab25985a395ec9fdb422bd2048a9de4" 2024-08-20T22:11:52.4333474Z }, 2024-08-20T22:11:52.4333860Z { 2024-08-20T22:11:52.4334368Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4335013Z "size": 2880110522, 2024-08-20T22:11:52.4335730Z "digest": "sha256:e834cd84960314101228fad9e84dc2ac3661d07b32945d943eff70e57fa73e67" 2024-08-20T22:11:52.4336439Z }, 2024-08-20T22:11:52.4336741Z { 2024-08-20T22:11:52.4337312Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4337972Z "size": 380, 2024-08-20T22:11:52.4338563Z "digest": "sha256:6d388069c2d7b7900bee880ba0a9bb683165f3978613fec36ed7e3edf04b180b" 2024-08-20T22:11:52.4339308Z }, 2024-08-20T22:11:52.4339666Z { 2024-08-20T22:11:52.4340229Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4340854Z "size": 12906, 2024-08-20T22:11:52.4341468Z "digest": "sha256:51d2d37470818e64147c1b99897175234adecf55386b5ea8da0e800d80bd24b8" 2024-08-20T22:11:52.4342214Z }, 2024-08-20T22:11:52.4342571Z { 2024-08-20T22:11:52.4343058Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4343748Z "size": 805, 2024-08-20T22:11:52.4344394Z "digest": "sha256:84d7c1aff0165ae94c936361c872a5d98a96f81286d4901bfa4b8f8a6e0ed869" 2024-08-20T22:11:52.4345088Z }, 2024-08-20T22:11:52.4345439Z { 2024-08-20T22:11:52.4345974Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4346610Z "size": 106, 2024-08-20T22:11:52.4347277Z "digest": "sha256:ce74fbfc871d05c302bc99afff486a44c401c4d7fc1f2856d593e4d7c8e61a5c" 2024-08-20T22:11:52.4347981Z }, 2024-08-20T22:11:52.4348317Z { 2024-08-20T22:11:52.4348918Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4349520Z "size": 504, 2024-08-20T22:11:52.4350125Z "digest": "sha256:0972844ea4ec494726981a800e4a687d65d178eda00ef010e5f4031042e7b052" 2024-08-20T22:11:52.4352085Z }, 2024-08-20T22:11:52.4352419Z { 2024-08-20T22:11:52.4352921Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4353609Z "size": 121477355, 2024-08-20T22:11:52.4354270Z "digest": "sha256:20952c6c7b153921a34a3deca88e1bd7373bbefca60bea51531ae313dd6ad858" 2024-08-20T22:11:52.4354993Z }, 2024-08-20T22:11:52.4355344Z { 2024-08-20T22:11:52.4355857Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4356509Z "size": 109, 2024-08-20T22:11:52.4357130Z "digest": "sha256:98f539161af00f8a9e6085712bde8384f11669fb341a1387672b3814c3ebe1ab" 2024-08-20T22:11:52.4357837Z }, 2024-08-20T22:11:52.4358171Z { 2024-08-20T22:11:52.4358734Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4359351Z "size": 489, 2024-08-20T22:11:52.4360013Z "digest": "sha256:0183570ab6f1b76f21e0bd1aebd83ac04068546eaf762334a3398f1726b954c2" 2024-08-20T22:11:52.4360749Z }, 2024-08-20T22:11:52.4361116Z { 2024-08-20T22:11:52.4361596Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4362283Z "size": 297, 2024-08-20T22:11:52.4362930Z "digest": "sha256:58e41754815ca62fced600e17f7a2dd2ac4e1c5f759d64c3b72e9d390afe94db" 2024-08-20T22:11:52.4363687Z }, 2024-08-20T22:11:52.4364109Z { 2024-08-20T22:11:52.4364628Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4365313Z "size": 103, 2024-08-20T22:11:52.4365919Z "digest": "sha256:f16e739cc132112eb4aee330a7096c38d3063a202f55ddc660b172da4c94c278" 2024-08-20T22:11:52.4366611Z }, 2024-08-20T22:11:52.4367020Z { 2024-08-20T22:11:52.4367637Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4368237Z "size": 1473, 2024-08-20T22:11:52.4368949Z "digest": "sha256:ddc70883fbfec3c44ac5b825395f40dc6a3f2201314f4242f197735abcfae4f5" 2024-08-20T22:11:52.4369647Z }, 2024-08-20T22:11:52.4369947Z { 2024-08-20T22:11:52.4370549Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4371196Z "size": 594731067, 2024-08-20T22:11:52.4371828Z "digest": "sha256:3c5150459092a973024c73ab1af7d259d27bc39e1f0f6d3b321d5b1421c58ef8" 2024-08-20T22:11:52.4372557Z }, 2024-08-20T22:11:52.4372900Z { 2024-08-20T22:11:52.4373448Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4374138Z "size": 164, 2024-08-20T22:11:52.4374725Z "digest": "sha256:697db697809d30a05a9d159e1ca0c4ff57743783a5dd677bd048c3b72c5c2e74" 2024-08-20T22:11:52.4375443Z }, 2024-08-20T22:11:52.4375825Z { 2024-08-20T22:11:52.4376345Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4376967Z "size": 563, 2024-08-20T22:11:52.4377635Z "digest": "sha256:01240c436d2fed24c59304ba81b8c20d47365178244a3eafd102c03ed2a8d640" 2024-08-20T22:11:52.4378334Z }, 2024-08-20T22:11:52.4378654Z { 2024-08-20T22:11:52.4379299Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4379967Z "size": 35865715, 2024-08-20T22:11:52.4380615Z "digest": "sha256:526c21e3e021736f8e11a7c72706caed8b29a6ef45b8acd1a8ce101bda1d35cb" 2024-08-20T22:11:52.4381331Z }, 2024-08-20T22:11:52.4381690Z { 2024-08-20T22:11:52.4382217Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4382937Z "size": 104, 2024-08-20T22:11:52.4383518Z "digest": "sha256:8a1507f1f6002a8b82b720956e1fe6a67455c651a95a90db071880112ea83dd8" 2024-08-20T22:11:52.4384206Z }, 2024-08-20T22:11:52.4384616Z { 2024-08-20T22:11:52.4385101Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4385743Z "size": 425, 2024-08-20T22:11:52.4386447Z "digest": "sha256:efb435cd2eaa405daeb98ce6116c948b9d3290850c23691461be7310bec69bdd" 2024-08-20T22:11:52.4387166Z }, 2024-08-20T22:11:52.4387467Z { 2024-08-20T22:11:52.4388058Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4388805Z "size": 20262177, 2024-08-20T22:11:52.4389535Z "digest": "sha256:6b78a8535b2ce76b1dd44cada752d9cb93b90df8b51ee855854febe7ab2cb7b9" 2024-08-20T22:11:52.4390212Z }, 2024-08-20T22:11:52.4390553Z { 2024-08-20T22:11:52.4391162Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4391798Z "size": 440, 2024-08-20T22:11:52.4392387Z "digest": "sha256:1a5959b2d611f7a5d21dbb0c5f25c9315be475b25545e77317037b366a55aeba" 2024-08-20T22:11:52.4393156Z }, 2024-08-20T22:11:52.4393493Z { 2024-08-20T22:11:52.4393976Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4394699Z "size": 700, 2024-08-20T22:11:52.4395334Z "digest": "sha256:657a0885dc72f558a3fdaffb85a7d85c2ab25985a395ec9fdb422bd2048a9de4" 2024-08-20T22:11:52.4396061Z }, 2024-08-20T22:11:52.4396426Z { 2024-08-20T22:11:52.4396948Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4397607Z "size": 142, 2024-08-20T22:11:52.4398298Z "digest": "sha256:ec22bfa269b65f52c12ea974dbbbe850632692df1636da93618fb1cd5ba2759d" 2024-08-20T22:11:52.4398973Z }, 2024-08-20T22:11:52.4399333Z { 2024-08-20T22:11:52.4399904Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4400676Z "size": 135, 2024-08-20T22:11:52.4401317Z "digest": "sha256:143fcaf620b2ae761c579c3687fa0aace809e7f758b89f9538fc40777e9be91c" 2024-08-20T22:11:52.4402075Z }, 2024-08-20T22:11:52.4402433Z { 2024-08-20T22:11:52.4402921Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4403609Z "size": 4838067845, 2024-08-20T22:11:52.4404283Z "digest": "sha256:d9ebe2872e744f11fc55059a7be35140cad3699967c198ec877ad046d4e873ef" 2024-08-20T22:11:52.4405263Z }, 2024-08-20T22:11:52.4405666Z { 2024-08-20T22:11:52.4406193Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4406829Z "size": 195, 2024-08-20T22:11:52.4407620Z "digest": "sha256:2dfbafd7cb9fa64cc49904c931bfcefd64fff2a49ef4912b00540f96f2d6d11e" 2024-08-20T22:11:52.4408309Z }, 2024-08-20T22:11:52.4408647Z { 2024-08-20T22:11:52.4409236Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4409875Z "size": 566, 2024-08-20T22:11:52.4410508Z "digest": "sha256:5010474278e0c7ee4080071b6a3619c6db59e42e10ec9c7528ef3cd983e4a537" 2024-08-20T22:11:52.4411266Z }, 2024-08-20T22:11:52.4411604Z { 2024-08-20T22:11:52.4412108Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4412803Z "size": 43163542, 2024-08-20T22:11:52.4413441Z "digest": "sha256:62e2be9188ac5963bd23a889747b73501ea9712d1d9e09584cecefc6b30f9c1a" 2024-08-20T22:11:52.4414206Z }, 2024-08-20T22:11:52.4414510Z { 2024-08-20T22:11:52.4415027Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4415737Z "size": 106, 2024-08-20T22:11:52.4416324Z "digest": "sha256:8738005d6660c2efca91d8b34cb3f269d342d6a9c37b2d9c004c61aad6993e3f" 2024-08-20T22:11:52.4417029Z }, 2024-08-20T22:11:52.4417458Z { 2024-08-20T22:11:52.4417976Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4418599Z "size": 1380, 2024-08-20T22:11:52.4419266Z "digest": "sha256:7397e0c0192133e7084b2cdb03325a59350b480527751a8384b95ee1b73ff40b" 2024-08-20T22:11:52.4419950Z }, 2024-08-20T22:11:52.4420308Z { 2024-08-20T22:11:52.4420838Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4421561Z "size": 700, 2024-08-20T22:11:52.4422182Z "digest": "sha256:657a0885dc72f558a3fdaffb85a7d85c2ab25985a395ec9fdb422bd2048a9de4" 2024-08-20T22:11:52.4422930Z }, 2024-08-20T22:11:52.4423255Z { 2024-08-20T22:11:52.4423777Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4424466Z "size": 140, 2024-08-20T22:11:52.4425086Z "digest": "sha256:d31b6d35a64ac472ba535e111b62433faaa9f4f19e2f2bbcf221088e1dcb1dc7" 2024-08-20T22:11:52.4425947Z }, 2024-08-20T22:11:52.4426360Z { 2024-08-20T22:11:52.4426882Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4427491Z "size": 120, 2024-08-20T22:11:52.4428199Z "digest": "sha256:6e06b9b118be0d2c3ab2c4ab2070763bd63142fe0cc29cfe4534493d3a2aa9d5" 2024-08-20T22:11:52.4428906Z }, 2024-08-20T22:11:52.4429301Z { 2024-08-20T22:11:52.4429812Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4430510Z "size": 4324622506, 2024-08-20T22:11:52.4431250Z "digest": "sha256:1ba96cf48ba3c58c7a414e250aecfab00a65acb0c0e4d57d26c0cbbafcc7a826" 2024-08-20T22:11:52.4431967Z }, 2024-08-20T22:11:52.4432270Z { 2024-08-20T22:11:52.4432863Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4433497Z "size": 176, 2024-08-20T22:11:52.4434093Z "digest": "sha256:8269d46e5fda811dab8d04ad8629add72ba91233a8d3394b46c27b9569a00e7d" 2024-08-20T22:11:52.4434870Z }, 2024-08-20T22:11:52.4435221Z { 2024-08-20T22:11:52.4435744Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4436424Z "size": 907, 2024-08-20T22:11:52.4437051Z "digest": "sha256:877a9eaa525ffcd309df81c8ad41a0ce7788becba7998484e9835823532c4103" 2024-08-20T22:11:52.4437911Z }, 2024-08-20T22:11:52.4438329Z { 2024-08-20T22:11:52.4438815Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4439482Z "size": 700, 2024-08-20T22:11:52.4440162Z "digest": "sha256:657a0885dc72f558a3fdaffb85a7d85c2ab25985a395ec9fdb422bd2048a9de4" 2024-08-20T22:11:52.4440893Z }, 2024-08-20T22:11:52.4441195Z { 2024-08-20T22:11:52.4441769Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4442429Z "size": 134, 2024-08-20T22:11:52.4443026Z "digest": "sha256:b4a44fd6dcb4500b86d306c24275b2066f74fc58a6d754701aeafd4ab6503237" 2024-08-20T22:11:52.4443772Z }, 2024-08-20T22:11:52.4444128Z { 2024-08-20T22:11:52.4444646Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4445304Z "size": 32, 2024-08-20T22:11:52.4445960Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:11:52.4446668Z }, 2024-08-20T22:11:52.4447151Z { 2024-08-20T22:11:52.4447642Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4448283Z "size": 157, 2024-08-20T22:11:52.4448994Z "digest": "sha256:3923dee95bc8123433f79dec72afdab61db4bb0b6cee2cf18697e0f670302da7" 2024-08-20T22:11:52.4449669Z }, 2024-08-20T22:11:52.4450010Z { 2024-08-20T22:11:52.4450599Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4451235Z "size": 1843, 2024-08-20T22:11:52.4451836Z "digest": "sha256:3385be080d47f0bf4f800da707b391ef2aff82d4bedb454fbf033f5ffc0ea1f0" 2024-08-20T22:11:52.4452608Z }, 2024-08-20T22:11:52.4452990Z { 2024-08-20T22:11:52.4453589Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4454204Z "size": 7529774, 2024-08-20T22:11:52.4454861Z "digest": "sha256:fb3b968bdf0d9633f908cd7e2980ca6eb34998708148d9d50bed730d3d00a311" 2024-08-20T22:11:52.4455648Z }, 2024-08-20T22:11:52.4455959Z { 2024-08-20T22:11:52.4456493Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4457221Z "size": 165, 2024-08-20T22:11:52.4457865Z "digest": "sha256:dca079b1da9ded43032fb32760349d22466d7f35d328acb504c3a0915a7d399c" 2024-08-20T22:11:52.4458541Z }, 2024-08-20T22:11:52.4458974Z { 2024-08-20T22:11:52.4459536Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4460207Z "size": 7943, 2024-08-20T22:11:52.4460858Z "digest": "sha256:5809b0e302890af8a867695be09f99403b949cc81b11c292b048b7cd063b4622" 2024-08-20T22:11:52.4461561Z }, 2024-08-20T22:11:52.4461932Z { 2024-08-20T22:11:52.4462514Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4463119Z "size": 8070, 2024-08-20T22:11:52.4463892Z "digest": "sha256:3226b5c3a909ead1adffdfc2b46c9b07b660fd28700f5aae1f7d32dcbbb4af72" 2024-08-20T22:11:52.4464662Z }, 2024-08-20T22:11:52.4464974Z { 2024-08-20T22:11:52.4465524Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4466214Z "size": 304, 2024-08-20T22:11:52.4466900Z "digest": "sha256:16d2c60c3be566081400461f3a4fb263a7efb7038c767ec32afa61ea3a43606f" 2024-08-20T22:11:52.4467564Z }, 2024-08-20T22:11:52.4467959Z { 2024-08-20T22:11:52.4468510Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4469159Z "size": 7630410, 2024-08-20T22:11:52.4469819Z "digest": "sha256:a3645dbfb35189d14514844cc6ede8a6c7b144ffa36a918eeb7da013432d75b9" 2024-08-20T22:11:52.4470546Z }, 2024-08-20T22:11:52.4470892Z { 2024-08-20T22:11:52.4471431Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4472098Z "size": 108, 2024-08-20T22:11:52.4472734Z "digest": "sha256:450f041fd9b29b129617d3242c9ab41f8f9d81b2ce961dab6cd264110e5579f7" 2024-08-20T22:11:52.4473536Z }, 2024-08-20T22:11:52.4473846Z { 2024-08-20T22:11:52.4474381Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4475203Z "size": 54145659, 2024-08-20T22:11:52.4475855Z "digest": "sha256:fa90062cf4e64751f618b4107a8c0157b0b2d8d37da6b413579a9d03f54c5ce6" 2024-08-20T22:11:52.4476542Z }, 2024-08-20T22:11:52.4476943Z { 2024-08-20T22:11:52.4477475Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4478208Z "size": 474, 2024-08-20T22:11:52.4478814Z "digest": "sha256:c3392f1762d82dc09ebc3f2846d9d7efcea9dc0dd40d433c3f5eee69362d7705" 2024-08-20T22:11:52.4479525Z }, 2024-08-20T22:11:52.4479976Z { 2024-08-20T22:11:52.4480681Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4481332Z "size": 1374858412, 2024-08-20T22:11:52.4482073Z "digest": "sha256:e52dd7f40be6e1973f87fba1296f2bba0461107e2cc03c1bbb6468151ae99554" 2024-08-20T22:11:52.4482786Z }, 2024-08-20T22:11:52.4483117Z { 2024-08-20T22:11:52.4483695Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4484339Z "size": 106, 2024-08-20T22:11:52.4485001Z "digest": "sha256:87b663a94d66bfc4467688beab83925679fbd74e70bc85b2ef97dffd8b40e004" 2024-08-20T22:11:52.4485715Z }, 2024-08-20T22:11:52.4486062Z { 2024-08-20T22:11:52.4486608Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4487368Z "size": 559, 2024-08-20T22:11:52.4488006Z "digest": "sha256:1df0f571a9950af44d446e82235480d3626daf9c3aea3ed59a7ac23b36fdf449" 2024-08-20T22:11:52.4488732Z }, 2024-08-20T22:11:52.4489161Z { 2024-08-20T22:11:52.4489689Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4490333Z "size": 46248351, 2024-08-20T22:11:52.4491041Z "digest": "sha256:9fb55636587d5dfac735bccabbe62dc6802da082dba88c9859adb8402ff72fb0" 2024-08-20T22:11:52.4491781Z }, 2024-08-20T22:11:52.4492089Z { 2024-08-20T22:11:52.4492680Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4493341Z "size": 111, 2024-08-20T22:11:52.4494012Z "digest": "sha256:6f5f644dad086485e9dab47c9e119d092d3db8ea0d70ba65e5683fd8ae2c4813" 2024-08-20T22:11:52.4494744Z }, 2024-08-20T22:11:52.4495113Z { 2024-08-20T22:11:52.4495635Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4496314Z "size": 32, 2024-08-20T22:11:52.4496942Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:11:52.4497654Z }, 2024-08-20T22:11:52.4498067Z { 2024-08-20T22:11:52.4498560Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4499241Z "size": 32, 2024-08-20T22:11:52.4499944Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:11:52.4500660Z }, 2024-08-20T22:11:52.4501122Z { 2024-08-20T22:11:52.4501698Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4502339Z "size": 32, 2024-08-20T22:11:52.4502959Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:11:52.4503723Z }, 2024-08-20T22:11:52.4504066Z { 2024-08-20T22:11:52.4504979Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:11:52.4505613Z "size": 32, 2024-08-20T22:11:52.4506278Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:11:52.4507052Z } 2024-08-20T22:11:52.4507412Z ] 2024-08-20T22:11:52.4507778Z } 2024-08-20T22:11:52.4818241Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:11:52.4818745Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:11:52.4819291Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-20T22:11:52.4828641Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:52.4829153Z env: 2024-08-20T22:11:52.4829435Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:52.4830580Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:52.4831735Z ##[endgroup] 2024-08-20T22:11:52.4863198Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:52.4918278Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-20T22:11:52.4918861Z with: 2024-08-20T22:11:52.4919980Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:52.4921281Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:52.4921796Z env: 2024-08-20T22:11:52.4922083Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:52.4922448Z ##[endgroup] 2024-08-20T22:11:52.4939849Z ##[group]Run set -x 2024-08-20T22:11:52.4940193Z set -x 2024-08-20T22:11:52.4940499Z set +e 2024-08-20T22:11:52.4940798Z  2024-08-20T22:11:52.4941075Z login() { 2024-08-20T22:11:52.4941729Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:11:52.4942434Z } 2024-08-20T22:11:52.4942715Z  2024-08-20T22:11:52.4943024Z retry () { 2024-08-20T22:11:52.4944873Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:11:52.4945318Z } 2024-08-20T22:11:52.4945600Z  2024-08-20T22:11:52.4945916Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:11:52.4946328Z  2024-08-20T22:11:52.4946609Z set -e 2024-08-20T22:11:52.4947082Z # ignore output since only exit code is used for conditional 2024-08-20T22:11:52.4947778Z # only pull docker image if it's not available locally 2024-08-20T22:11:52.4948538Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-20T22:11:52.4949222Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-20T22:11:52.4949666Z fi 2024-08-20T22:11:52.4958464Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:11:52.4958970Z env: 2024-08-20T22:11:52.4959267Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:11:52.4960416Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:52.4961712Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:52.4962222Z ##[endgroup] 2024-08-20T22:11:52.4991585Z + set +e 2024-08-20T22:11:52.4992124Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:52.4992768Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:52.4995497Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:11:52.4997002Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:11:53.0279068Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:11:53.0279958Z Configure a credential helper to remove this warning. See 2024-08-20T22:11:53.0280790Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:11:53.0281326Z 2024-08-20T22:11:53.0281460Z Login Succeeded 2024-08-20T22:11:53.0302341Z + set -e 2024-08-20T22:11:53.0303673Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.0464795Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.0467273Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:11:53.2752178Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks 2024-08-20T22:11:53.2753437Z 63e9bbe32327: Pulling fs layer 2024-08-20T22:11:53.2753955Z cfb3d849840e: Pulling fs layer 2024-08-20T22:11:53.2754740Z 968831e596a6: Pulling fs layer 2024-08-20T22:11:53.2755276Z ea310eb267ca: Pulling fs layer 2024-08-20T22:11:53.2755881Z 3af11d09e9cd: Pulling fs layer 2024-08-20T22:11:53.2756385Z ebfec18059b9: Pulling fs layer 2024-08-20T22:11:53.2756852Z 533b4aebf169: Pulling fs layer 2024-08-20T22:11:53.2757362Z 9dd75d06a091: Pulling fs layer 2024-08-20T22:11:53.2757875Z 30bfca4dd349: Pulling fs layer 2024-08-20T22:11:53.2758368Z ebfec18059b9: Waiting 2024-08-20T22:11:53.2758811Z 1b57ce94cad9: Pulling fs layer 2024-08-20T22:11:53.2759316Z 9ee6bdb31195: Pulling fs layer 2024-08-20T22:11:53.2759822Z ea310eb267ca: Waiting 2024-08-20T22:11:53.2760276Z 722a6699b4eb: Pulling fs layer 2024-08-20T22:11:53.2760774Z 164218df90f4: Pulling fs layer 2024-08-20T22:11:53.2761263Z f1d74e82c839: Pulling fs layer 2024-08-20T22:11:53.2761756Z 13b68a95b298: Pulling fs layer 2024-08-20T22:11:53.2762142Z c95fa6ffe990: Pulling fs layer 2024-08-20T22:11:53.2762526Z 831d1d1ad5de: Pulling fs layer 2024-08-20T22:11:53.2762913Z 3116a975d40f: Pulling fs layer 2024-08-20T22:11:53.2763293Z aed1c44ec0f6: Pulling fs layer 2024-08-20T22:11:53.2763667Z 657a0885dc72: Pulling fs layer 2024-08-20T22:11:53.2764044Z e834cd849603: Pulling fs layer 2024-08-20T22:11:53.2764978Z 6d388069c2d7: Pulling fs layer 2024-08-20T22:11:53.2765482Z 51d2d3747081: Pulling fs layer 2024-08-20T22:11:53.2766208Z 9ee6bdb31195: Waiting 2024-08-20T22:11:53.2766686Z 84d7c1aff016: Pulling fs layer 2024-08-20T22:11:53.2767235Z ce74fbfc871d: Pulling fs layer 2024-08-20T22:11:53.2767687Z 0972844ea4ec: Pulling fs layer 2024-08-20T22:11:53.2768072Z 20952c6c7b15: Pulling fs layer 2024-08-20T22:11:53.2768454Z 98f539161af0: Pulling fs layer 2024-08-20T22:11:53.2768819Z 9dd75d06a091: Waiting 2024-08-20T22:11:53.2769161Z 0183570ab6f1: Pulling fs layer 2024-08-20T22:11:53.2769523Z 30bfca4dd349: Waiting 2024-08-20T22:11:53.2769863Z 58e41754815c: Pulling fs layer 2024-08-20T22:11:53.2770244Z f16e739cc132: Pulling fs layer 2024-08-20T22:11:53.2770619Z ddc70883fbfe: Pulling fs layer 2024-08-20T22:11:53.2771131Z 3af11d09e9cd: Waiting 2024-08-20T22:11:53.2771462Z 1b57ce94cad9: Waiting 2024-08-20T22:11:53.2772198Z 3c5150459092: Pulling fs layer 2024-08-20T22:11:53.2772590Z 697db697809d: Pulling fs layer 2024-08-20T22:11:53.2773028Z 722a6699b4eb: Waiting 2024-08-20T22:11:53.2773450Z 01240c436d2f: Pulling fs layer 2024-08-20T22:11:53.2773900Z 526c21e3e021: Pulling fs layer 2024-08-20T22:11:53.2774271Z 8a1507f1f600: Pulling fs layer 2024-08-20T22:11:53.2774698Z efb435cd2eaa: Pulling fs layer 2024-08-20T22:11:53.2775274Z c95fa6ffe990: Waiting 2024-08-20T22:11:53.2775606Z 6b78a8535b2c: Pulling fs layer 2024-08-20T22:11:53.2775979Z 1a5959b2d611: Pulling fs layer 2024-08-20T22:11:53.2776354Z ec22bfa269b6: Pulling fs layer 2024-08-20T22:11:53.2776715Z 0972844ea4ec: Waiting 2024-08-20T22:11:53.2777254Z 143fcaf620b2: Pulling fs layer 2024-08-20T22:11:53.2777628Z d9ebe2872e74: Pulling fs layer 2024-08-20T22:11:53.2778005Z 2dfbafd7cb9f: Pulling fs layer 2024-08-20T22:11:53.2778365Z 6d388069c2d7: Waiting 2024-08-20T22:11:53.2778864Z 58e41754815c: Waiting 2024-08-20T22:11:53.2779190Z 5010474278e0: Pulling fs layer 2024-08-20T22:11:53.2779546Z 657a0885dc72: Waiting 2024-08-20T22:11:53.2779872Z 62e2be9188ac: Pulling fs layer 2024-08-20T22:11:53.2780232Z e834cd849603: Waiting 2024-08-20T22:11:53.2780747Z 8738005d6660: Pulling fs layer 2024-08-20T22:11:53.2781354Z 7397e0c01921: Pulling fs layer 2024-08-20T22:11:53.2781854Z 84d7c1aff016: Waiting 2024-08-20T22:11:53.2782405Z 3116a975d40f: Waiting 2024-08-20T22:11:53.2782952Z d31b6d35a64a: Pulling fs layer 2024-08-20T22:11:53.2783454Z 6e06b9b118be: Pulling fs layer 2024-08-20T22:11:53.2783949Z 3c5150459092: Waiting 2024-08-20T22:11:53.2784409Z 1ba96cf48ba3: Pulling fs layer 2024-08-20T22:11:53.2784915Z 697db697809d: Waiting 2024-08-20T22:11:53.2785367Z 8269d46e5fda: Pulling fs layer 2024-08-20T22:11:53.2785892Z 877a9eaa525f: Pulling fs layer 2024-08-20T22:11:53.2786389Z ddc70883fbfe: Waiting 2024-08-20T22:11:53.2786854Z b4a44fd6dcb4: Pulling fs layer 2024-08-20T22:11:53.2787550Z aed1c44ec0f6: Waiting 2024-08-20T22:11:53.2788032Z 4f4fb700ef54: Pulling fs layer 2024-08-20T22:11:53.2788811Z 3923dee95bc8: Pulling fs layer 2024-08-20T22:11:53.2789328Z 3385be080d47: Pulling fs layer 2024-08-20T22:11:53.2789855Z fb3b968bdf0d: Pulling fs layer 2024-08-20T22:11:53.2790384Z dca079b1da9d: Pulling fs layer 2024-08-20T22:11:53.2790896Z 5809b0e30289: Pulling fs layer 2024-08-20T22:11:53.2791403Z efb435cd2eaa: Waiting 2024-08-20T22:11:53.2791852Z 7397e0c01921: Waiting 2024-08-20T22:11:53.2792317Z 3226b5c3a909: Pulling fs layer 2024-08-20T22:11:53.2792829Z 16d2c60c3be5: Pulling fs layer 2024-08-20T22:11:53.2793357Z a3645dbfb351: Pulling fs layer 2024-08-20T22:11:53.2793867Z 831d1d1ad5de: Waiting 2024-08-20T22:11:53.2794322Z 450f041fd9b2: Pulling fs layer 2024-08-20T22:11:53.2794818Z 13b68a95b298: Waiting 2024-08-20T22:11:53.2795275Z fa90062cf4e6: Pulling fs layer 2024-08-20T22:11:53.2795799Z c3392f1762d8: Pulling fs layer 2024-08-20T22:11:53.2796339Z e52dd7f40be6: Pulling fs layer 2024-08-20T22:11:53.2796854Z 1ba96cf48ba3: Waiting 2024-08-20T22:11:53.2797318Z 87b663a94d66: Pulling fs layer 2024-08-20T22:11:53.2797841Z 1df0f571a995: Pulling fs layer 2024-08-20T22:11:53.2798345Z 8269d46e5fda: Waiting 2024-08-20T22:11:53.2798792Z 9fb55636587d: Pulling fs layer 2024-08-20T22:11:53.2799397Z 6f5f644dad08: Pulling fs layer 2024-08-20T22:11:53.2799885Z 0183570ab6f1: Waiting 2024-08-20T22:11:53.2800320Z 877a9eaa525f: Waiting 2024-08-20T22:11:53.2800766Z fb3b968bdf0d: Waiting 2024-08-20T22:11:53.2801214Z f1d74e82c839: Waiting 2024-08-20T22:11:53.2801654Z 01240c436d2f: Waiting 2024-08-20T22:11:53.2802072Z dca079b1da9d: Waiting 2024-08-20T22:11:53.2802586Z 16d2c60c3be5: Waiting 2024-08-20T22:11:53.2803023Z 5809b0e30289: Waiting 2024-08-20T22:11:53.2803493Z 3226b5c3a909: Waiting 2024-08-20T22:11:53.2803924Z 1a5959b2d611: Waiting 2024-08-20T22:11:53.2804492Z d9ebe2872e74: Waiting 2024-08-20T22:11:53.2805343Z 6b78a8535b2c: Waiting 2024-08-20T22:11:53.2805970Z 8a1507f1f600: Waiting 2024-08-20T22:11:53.2806516Z f16e739cc132: Waiting 2024-08-20T22:11:53.2806956Z 450f041fd9b2: Waiting 2024-08-20T22:11:53.2807517Z 62e2be9188ac: Waiting 2024-08-20T22:11:53.2807950Z 3923dee95bc8: Waiting 2024-08-20T22:11:53.2808378Z 8738005d6660: Waiting 2024-08-20T22:11:53.2808805Z 6f5f644dad08: Waiting 2024-08-20T22:11:53.2809233Z b4a44fd6dcb4: Waiting 2024-08-20T22:11:53.2809672Z 4f4fb700ef54: Waiting 2024-08-20T22:11:53.2810112Z c3392f1762d8: Waiting 2024-08-20T22:11:53.2810542Z 51d2d3747081: Waiting 2024-08-20T22:11:53.2811176Z 5010474278e0: Waiting 2024-08-20T22:11:53.2811603Z 98f539161af0: Waiting 2024-08-20T22:11:53.2812035Z e52dd7f40be6: Waiting 2024-08-20T22:11:53.2812458Z 9fb55636587d: Waiting 2024-08-20T22:11:53.2812890Z 6e06b9b118be: Waiting 2024-08-20T22:11:53.2813330Z ec22bfa269b6: Waiting 2024-08-20T22:11:53.2813770Z 1df0f571a995: Waiting 2024-08-20T22:11:53.2814199Z 3385be080d47: Waiting 2024-08-20T22:11:53.2814633Z 143fcaf620b2: Waiting 2024-08-20T22:11:53.2815070Z 87b663a94d66: Waiting 2024-08-20T22:11:53.2815520Z a3645dbfb351: Waiting 2024-08-20T22:11:53.5181647Z cfb3d849840e: Verifying Checksum 2024-08-20T22:11:53.5182215Z cfb3d849840e: Download complete 2024-08-20T22:11:53.6117644Z ea310eb267ca: Verifying Checksum 2024-08-20T22:11:53.6118248Z ea310eb267ca: Download complete 2024-08-20T22:11:53.6608557Z 63e9bbe32327: Verifying Checksum 2024-08-20T22:11:53.6609024Z 63e9bbe32327: Download complete 2024-08-20T22:11:53.6847261Z 3af11d09e9cd: Verifying Checksum 2024-08-20T22:11:53.6847767Z 3af11d09e9cd: Download complete 2024-08-20T22:11:53.7626957Z 533b4aebf169: Download complete 2024-08-20T22:11:53.8390880Z 9dd75d06a091: Download complete 2024-08-20T22:11:53.9209044Z 30bfca4dd349: Verifying Checksum 2024-08-20T22:11:53.9209591Z 30bfca4dd349: Download complete 2024-08-20T22:11:54.0565163Z 968831e596a6: Verifying Checksum 2024-08-20T22:11:54.0565651Z 968831e596a6: Download complete 2024-08-20T22:11:54.3640576Z 722a6699b4eb: Verifying Checksum 2024-08-20T22:11:54.3641070Z 722a6699b4eb: Download complete 2024-08-20T22:11:54.8150587Z 63e9bbe32327: Pull complete 2024-08-20T22:11:55.1224982Z cfb3d849840e: Pull complete 2024-08-20T22:11:56.0308560Z 968831e596a6: Pull complete 2024-08-20T22:11:56.0512707Z ea310eb267ca: Pull complete 2024-08-20T22:11:56.0713929Z 3af11d09e9cd: Pull complete 2024-08-20T22:12:01.3751890Z 164218df90f4: Verifying Checksum 2024-08-20T22:12:01.3752364Z 164218df90f4: Download complete 2024-08-20T22:12:01.5040595Z f1d74e82c839: Download complete 2024-08-20T22:12:01.5890188Z 13b68a95b298: Verifying Checksum 2024-08-20T22:12:01.5890635Z 13b68a95b298: Download complete 2024-08-20T22:12:01.7334027Z c95fa6ffe990: Verifying Checksum 2024-08-20T22:12:01.7334464Z c95fa6ffe990: Download complete 2024-08-20T22:12:05.0236499Z 831d1d1ad5de: Verifying Checksum 2024-08-20T22:12:05.0237470Z 831d1d1ad5de: Download complete 2024-08-20T22:12:05.1011126Z 3116a975d40f: Download complete 2024-08-20T22:12:05.1838348Z aed1c44ec0f6: Verifying Checksum 2024-08-20T22:12:05.1838772Z aed1c44ec0f6: Download complete 2024-08-20T22:12:05.2957636Z 657a0885dc72: Verifying Checksum 2024-08-20T22:12:05.2958066Z 657a0885dc72: Download complete 2024-08-20T22:12:15.5199032Z ebfec18059b9: Verifying Checksum 2024-08-20T22:12:15.5199513Z ebfec18059b9: Download complete 2024-08-20T22:12:15.6073838Z 6d388069c2d7: Download complete 2024-08-20T22:12:15.7218159Z 51d2d3747081: Download complete 2024-08-20T22:12:15.7926149Z 84d7c1aff016: Verifying Checksum 2024-08-20T22:12:15.7926725Z 84d7c1aff016: Download complete 2024-08-20T22:12:15.8851957Z ce74fbfc871d: Verifying Checksum 2024-08-20T22:12:15.8852439Z ce74fbfc871d: Download complete 2024-08-20T22:12:15.9670472Z 0972844ea4ec: Download complete 2024-08-20T22:12:19.7357319Z 20952c6c7b15: Verifying Checksum 2024-08-20T22:12:19.7357849Z 20952c6c7b15: Download complete 2024-08-20T22:12:19.8517848Z 98f539161af0: Verifying Checksum 2024-08-20T22:12:19.8518437Z 98f539161af0: Download complete 2024-08-20T22:12:19.9397983Z 0183570ab6f1: Verifying Checksum 2024-08-20T22:12:19.9398563Z 0183570ab6f1: Download complete 2024-08-20T22:12:20.0098383Z 58e41754815c: Verifying Checksum 2024-08-20T22:12:20.0098832Z 58e41754815c: Download complete 2024-08-20T22:12:20.0928199Z f16e739cc132: Download complete 2024-08-20T22:12:20.1786726Z ddc70883fbfe: Verifying Checksum 2024-08-20T22:12:20.1787298Z ddc70883fbfe: Download complete 2024-08-20T22:12:28.2996189Z ebfec18059b9: Pull complete 2024-08-20T22:12:28.3151021Z 533b4aebf169: Pull complete 2024-08-20T22:12:28.3294384Z 9dd75d06a091: Pull complete 2024-08-20T22:12:28.3433791Z 30bfca4dd349: Pull complete 2024-08-20T22:12:35.3230582Z 1b57ce94cad9: Verifying Checksum 2024-08-20T22:12:35.3231038Z 1b57ce94cad9: Download complete 2024-08-20T22:12:35.4435576Z 697db697809d: Download complete 2024-08-20T22:12:35.5257461Z 01240c436d2f: Verifying Checksum 2024-08-20T22:12:35.5258036Z 01240c436d2f: Download complete 2024-08-20T22:12:36.4484994Z 526c21e3e021: Verifying Checksum 2024-08-20T22:12:36.4485609Z 526c21e3e021: Download complete 2024-08-20T22:12:36.5353949Z 8a1507f1f600: Verifying Checksum 2024-08-20T22:12:36.5354525Z 8a1507f1f600: Download complete 2024-08-20T22:12:36.6261413Z efb435cd2eaa: Download complete 2024-08-20T22:12:37.2391477Z 3c5150459092: Verifying Checksum 2024-08-20T22:12:37.2392130Z 3c5150459092: Download complete 2024-08-20T22:12:37.3336179Z 1a5959b2d611: Verifying Checksum 2024-08-20T22:12:37.3336649Z 1a5959b2d611: Download complete 2024-08-20T22:12:37.4155874Z ec22bfa269b6: Verifying Checksum 2024-08-20T22:12:37.4156375Z ec22bfa269b6: Download complete 2024-08-20T22:12:37.4745744Z 6b78a8535b2c: Verifying Checksum 2024-08-20T22:12:37.4746248Z 6b78a8535b2c: Download complete 2024-08-20T22:12:37.5099185Z 143fcaf620b2: Download complete 2024-08-20T22:12:37.5866956Z 2dfbafd7cb9f: Verifying Checksum 2024-08-20T22:12:37.5867402Z 2dfbafd7cb9f: Download complete 2024-08-20T22:12:37.6785626Z 5010474278e0: Verifying Checksum 2024-08-20T22:12:37.6786120Z 5010474278e0: Download complete 2024-08-20T22:12:39.1370072Z 62e2be9188ac: Verifying Checksum 2024-08-20T22:12:39.1370939Z 62e2be9188ac: Download complete 2024-08-20T22:12:39.2228230Z 8738005d6660: Verifying Checksum 2024-08-20T22:12:39.2228790Z 8738005d6660: Download complete 2024-08-20T22:12:39.2941412Z 7397e0c01921: Verifying Checksum 2024-08-20T22:12:39.2941851Z 7397e0c01921: Download complete 2024-08-20T22:12:39.3834181Z d31b6d35a64a: Download complete 2024-08-20T22:12:39.4829827Z 6e06b9b118be: Download complete 2024-08-20T22:13:06.6543002Z 1b57ce94cad9: Pull complete 2024-08-20T22:13:06.8895000Z 9ee6bdb31195: Pull complete 2024-08-20T22:13:07.1106468Z 722a6699b4eb: Pull complete 2024-08-20T22:13:15.8496975Z 164218df90f4: Pull complete 2024-08-20T22:13:16.0061669Z f1d74e82c839: Pull complete 2024-08-20T22:13:16.2202505Z 13b68a95b298: Pull complete 2024-08-20T22:13:16.4374311Z c95fa6ffe990: Pull complete 2024-08-20T22:13:19.1137754Z 831d1d1ad5de: Pull complete 2024-08-20T22:13:19.3419125Z 3116a975d40f: Pull complete 2024-08-20T22:13:19.5898550Z aed1c44ec0f6: Pull complete 2024-08-20T22:13:19.8089684Z 657a0885dc72: Pull complete 2024-08-20T22:13:22.2867983Z e834cd849603: Verifying Checksum 2024-08-20T22:13:22.2868454Z e834cd849603: Download complete 2024-08-20T22:13:22.3541657Z 8269d46e5fda: Download complete 2024-08-20T22:13:22.4299294Z 877a9eaa525f: Verifying Checksum 2024-08-20T22:13:22.4299736Z 877a9eaa525f: Download complete 2024-08-20T22:13:22.5235418Z b4a44fd6dcb4: Verifying Checksum 2024-08-20T22:13:22.5236005Z b4a44fd6dcb4: Download complete 2024-08-20T22:13:22.5325927Z 4f4fb700ef54: Download complete 2024-08-20T22:13:22.6194918Z 3923dee95bc8: Verifying Checksum 2024-08-20T22:13:22.6195511Z 3923dee95bc8: Download complete 2024-08-20T22:13:22.6872137Z 3385be080d47: Verifying Checksum 2024-08-20T22:13:22.6873149Z 3385be080d47: Download complete 2024-08-20T22:13:23.0472310Z fb3b968bdf0d: Verifying Checksum 2024-08-20T22:13:23.0472947Z fb3b968bdf0d: Download complete 2024-08-20T22:13:23.1214061Z dca079b1da9d: Verifying Checksum 2024-08-20T22:13:23.1214679Z dca079b1da9d: Download complete 2024-08-20T22:13:23.2028422Z 5809b0e30289: Download complete 2024-08-20T22:13:23.2871833Z 3226b5c3a909: Verifying Checksum 2024-08-20T22:13:23.2872273Z 3226b5c3a909: Download complete 2024-08-20T22:13:23.3649650Z 16d2c60c3be5: Verifying Checksum 2024-08-20T22:13:23.3650077Z 16d2c60c3be5: Download complete 2024-08-20T22:13:23.5902996Z a3645dbfb351: Verifying Checksum 2024-08-20T22:13:23.5903443Z a3645dbfb351: Download complete 2024-08-20T22:13:23.6689041Z 450f041fd9b2: Download complete 2024-08-20T22:13:25.0981258Z fa90062cf4e6: Verifying Checksum 2024-08-20T22:13:25.0982213Z fa90062cf4e6: Download complete 2024-08-20T22:13:25.2098835Z c3392f1762d8: Download complete 2024-08-20T22:14:04.9991972Z e52dd7f40be6: Verifying Checksum 2024-08-20T22:14:04.9994699Z e52dd7f40be6: Download complete 2024-08-20T22:14:05.0914728Z 87b663a94d66: Verifying Checksum 2024-08-20T22:14:05.0915299Z 87b663a94d66: Download complete 2024-08-20T22:14:05.1834232Z 1df0f571a995: Verifying Checksum 2024-08-20T22:14:05.1834653Z 1df0f571a995: Download complete 2024-08-20T22:14:06.6937059Z 9fb55636587d: Verifying Checksum 2024-08-20T22:14:06.6937626Z 9fb55636587d: Download complete 2024-08-20T22:14:06.7720692Z 6f5f644dad08: Download complete 2024-08-20T22:14:42.9905682Z 1ba96cf48ba3: Download complete 2024-08-20T22:14:50.3612431Z d9ebe2872e74: Verifying Checksum 2024-08-20T22:14:50.3613072Z d9ebe2872e74: Download complete 2024-08-20T22:15:02.4320339Z e834cd849603: Pull complete 2024-08-20T22:15:02.6614971Z 6d388069c2d7: Pull complete 2024-08-20T22:15:02.9019190Z 51d2d3747081: Pull complete 2024-08-20T22:15:03.0205308Z 84d7c1aff016: Pull complete 2024-08-20T22:15:03.1510719Z ce74fbfc871d: Pull complete 2024-08-20T22:15:03.3595961Z 0972844ea4ec: Pull complete 2024-08-20T22:15:06.8949569Z 20952c6c7b15: Pull complete 2024-08-20T22:15:07.1167494Z 98f539161af0: Pull complete 2024-08-20T22:15:07.3461604Z 0183570ab6f1: Pull complete 2024-08-20T22:15:07.5791881Z 58e41754815c: Pull complete 2024-08-20T22:15:07.8070332Z f16e739cc132: Pull complete 2024-08-20T22:15:08.0334223Z ddc70883fbfe: Pull complete 2024-08-20T22:15:18.7297175Z 3c5150459092: Pull complete 2024-08-20T22:15:18.8711601Z 697db697809d: Pull complete 2024-08-20T22:15:19.0149110Z 01240c436d2f: Pull complete 2024-08-20T22:15:19.9152999Z 526c21e3e021: Pull complete 2024-08-20T22:15:20.0232610Z 8a1507f1f600: Pull complete 2024-08-20T22:15:20.1777087Z efb435cd2eaa: Pull complete 2024-08-20T22:15:20.6743854Z 6b78a8535b2c: Pull complete 2024-08-20T22:15:20.8846500Z 1a5959b2d611: Pull complete 2024-08-20T22:15:21.2564083Z ec22bfa269b6: Pull complete 2024-08-20T22:15:21.4406833Z 143fcaf620b2: Pull complete 2024-08-20T22:15:59.8772569Z d9ebe2872e74: Pull complete 2024-08-20T22:16:00.1100755Z 2dfbafd7cb9f: Pull complete 2024-08-20T22:16:00.3398387Z 5010474278e0: Pull complete 2024-08-20T22:16:02.7783248Z 62e2be9188ac: Pull complete 2024-08-20T22:16:02.7954160Z 8738005d6660: Pull complete 2024-08-20T22:16:02.8125942Z 7397e0c01921: Pull complete 2024-08-20T22:16:02.8498479Z d31b6d35a64a: Pull complete 2024-08-20T22:16:02.8662108Z 6e06b9b118be: Pull complete 2024-08-20T22:17:24.6593040Z 1ba96cf48ba3: Pull complete 2024-08-20T22:17:24.8787807Z 8269d46e5fda: Pull complete 2024-08-20T22:17:25.1097797Z 877a9eaa525f: Pull complete 2024-08-20T22:17:25.5067733Z b4a44fd6dcb4: Pull complete 2024-08-20T22:17:25.7204491Z 4f4fb700ef54: Pull complete 2024-08-20T22:17:25.9569526Z 3923dee95bc8: Pull complete 2024-08-20T22:17:26.1829683Z 3385be080d47: Pull complete 2024-08-20T22:17:26.5693294Z fb3b968bdf0d: Pull complete 2024-08-20T22:17:26.8059656Z dca079b1da9d: Pull complete 2024-08-20T22:17:27.0332206Z 5809b0e30289: Pull complete 2024-08-20T22:17:27.2542937Z 3226b5c3a909: Pull complete 2024-08-20T22:17:27.4787866Z 16d2c60c3be5: Pull complete 2024-08-20T22:17:28.8649593Z a3645dbfb351: Pull complete 2024-08-20T22:17:29.0890815Z 450f041fd9b2: Pull complete 2024-08-20T22:17:31.3140413Z fa90062cf4e6: Pull complete 2024-08-20T22:17:31.5492560Z c3392f1762d8: Pull complete 2024-08-20T22:17:44.9138130Z e52dd7f40be6: Pull complete 2024-08-20T22:17:45.1465918Z 87b663a94d66: Pull complete 2024-08-20T22:17:45.3717241Z 1df0f571a995: Pull complete 2024-08-20T22:17:46.2562216Z 9fb55636587d: Pull complete 2024-08-20T22:17:46.4933714Z 6f5f644dad08: Pull complete 2024-08-20T22:17:47.4980344Z Digest: sha256:4753b1ce6a27c0af5c669dd74e9e16618d0a1a015eb474ef14f6fd51409fc5f7 2024-08-20T22:17:47.5460580Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:17:47.5681365Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:17:47.5769408Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:17:47.5770327Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:17:47.5780620Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:17:47.5781127Z env: 2024-08-20T22:17:47.5781416Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:17:47.5781770Z ##[endgroup] 2024-08-20T22:17:47.5985484Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-20T22:17:47.5986029Z with: 2024-08-20T22:17:47.5986309Z driver-version: 550.54.15 2024-08-20T22:17:47.5986647Z env: 2024-08-20T22:17:47.5986912Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:17:47.5987257Z ##[endgroup] 2024-08-20T22:17:47.6073564Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:17:47.6074125Z with: 2024-08-20T22:17:47.6074403Z timeout_minutes: 10 2024-08-20T22:17:47.6074725Z max_attempts: 3 2024-08-20T22:17:47.6107316Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-20T22:17:47.6139911Z retry_wait_seconds: 10 2024-08-20T22:17:47.6140280Z polling_interval_seconds: 1 2024-08-20T22:17:47.6140649Z warning_on_retry: true 2024-08-20T22:17:47.6141006Z continue_on_error: false 2024-08-20T22:17:47.6141344Z env: 2024-08-20T22:17:47.6141618Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:17:47.6141967Z DRIVER_VERSION: 550.54.15 2024-08-20T22:17:47.6142322Z ##[endgroup] 2024-08-20T22:17:47.6951617Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-20T22:17:47.6952622Z + pre_install_nvidia_driver_amzn2 2024-08-20T22:17:47.6953824Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-20T22:17:48.1584260Z No match for argument: nvidia-driver-latest-dkms 2024-08-20T22:17:48.1584802Z No packages marked for removal. 2024-08-20T22:17:48.1647165Z Dependencies resolved. 2024-08-20T22:17:48.1656868Z Nothing to do. 2024-08-20T22:17:48.1657710Z Complete! 2024-08-20T22:17:48.2531051Z + install_nvidia_driver_common 2024-08-20T22:17:48.2534473Z + echo 'Before installing NVIDIA driver' 2024-08-20T22:17:48.2534935Z Before installing NVIDIA driver 2024-08-20T22:17:48.2535842Z + lspci 2024-08-20T22:17:48.3626217Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:17:48.3627188Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:17:48.3628335Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:17:48.3629411Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:17:48.3630237Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:17:48.3631032Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:17:48.3631771Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:17:48.3632584Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:17:48.3633179Z + lsmod 2024-08-20T22:17:48.3672720Z Module Size Used by 2024-08-20T22:17:48.3673330Z xt_conntrack 16384 1 2024-08-20T22:17:48.3673831Z nft_chain_nat 16384 3 2024-08-20T22:17:48.3674348Z xt_MASQUERADE 20480 1 2024-08-20T22:17:48.3674846Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-20T22:17:48.3675338Z nf_conntrack_netlink 57344 0 2024-08-20T22:17:48.3675951Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:17:48.3676823Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:17:48.3677372Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:17:48.3677939Z xfrm_user 57344 1 2024-08-20T22:17:48.3678441Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:17:48.3678868Z xt_addrtype 16384 2 2024-08-20T22:17:48.3679251Z nft_compat 20480 4 2024-08-20T22:17:48.3679708Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:17:48.3680338Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:17:48.3681043Z br_netfilter 36864 0 2024-08-20T22:17:48.3681601Z bridge 307200 1 br_netfilter 2024-08-20T22:17:48.3682046Z stp 16384 1 bridge 2024-08-20T22:17:48.3682470Z llc 16384 2 bridge,stp 2024-08-20T22:17:48.3682893Z overlay 167936 0 2024-08-20T22:17:48.3683284Z tls 114688 0 2024-08-20T22:17:48.3683659Z nls_ascii 16384 1 2024-08-20T22:17:48.3684037Z nls_cp437 20480 1 2024-08-20T22:17:48.3684407Z vfat 24576 1 2024-08-20T22:17:48.3684775Z fat 86016 1 vfat 2024-08-20T22:17:48.3685172Z sunrpc 692224 1 2024-08-20T22:17:48.3685564Z ena 167936 0 2024-08-20T22:17:48.3685944Z ghash_clmulni_intel 16384 0 2024-08-20T22:17:48.3686334Z ptp 36864 1 ena 2024-08-20T22:17:48.3686741Z pps_core 24576 1 ptp 2024-08-20T22:17:48.3687262Z aesni_intel 393216 0 2024-08-20T22:17:48.3687638Z i8042 45056 0 2024-08-20T22:17:48.3688012Z serio 28672 3 i8042 2024-08-20T22:17:48.3688443Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:17:48.3688972Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:17:48.3689473Z button 24576 0 2024-08-20T22:17:48.3689854Z sch_fq_codel 20480 17 2024-08-20T22:17:48.3690238Z dm_mod 188416 0 2024-08-20T22:17:48.3690609Z fuse 163840 1 2024-08-20T22:17:48.3691003Z configfs 57344 1 2024-08-20T22:17:48.3691418Z dax 45056 1 dm_mod 2024-08-20T22:17:48.3691819Z loop 36864 0 2024-08-20T22:17:48.3692194Z dmi_sysfs 20480 0 2024-08-20T22:17:48.3692569Z crc32_pclmul 16384 0 2024-08-20T22:17:48.3692960Z crc32c_intel 24576 0 2024-08-20T22:17:48.3693336Z efivarfs 24576 1 2024-08-20T22:17:48.3693812Z + modinfo nvidia 2024-08-20T22:17:48.3694708Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:17:48.3695634Z alias: char-major-195-* 2024-08-20T22:17:48.3696150Z version: 550.54.15 2024-08-20T22:17:48.3696628Z supported: external 2024-08-20T22:17:48.3697070Z license: NVIDIA 2024-08-20T22:17:48.3697617Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:17:48.3698105Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:17:48.3698575Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:17:48.3699040Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:17:48.3699546Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:17:48.3700040Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:17:48.3700592Z depends: i2c-core,drm 2024-08-20T22:17:48.3700968Z retpoline: Y 2024-08-20T22:17:48.3701283Z name: nvidia 2024-08-20T22:17:48.3701994Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:17:48.3702698Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:17:48.3703352Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:17:48.3703950Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:17:48.3704399Z parm: NVreg_RmLogonRC:int 2024-08-20T22:17:48.3705223Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:17:48.3705690Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:17:48.3706134Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:17:48.3706580Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:17:48.3707156Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:17:48.3707728Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:17:48.3708218Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:17:48.3708657Z parm: NVreg_EnableMSI:int 2024-08-20T22:17:48.3709080Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:17:48.3709543Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:17:48.3710079Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:17:48.3710668Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:17:48.3711229Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:17:48.3711848Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:17:48.3712448Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:17:48.3713059Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:17:48.3713647Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:17:48.3714143Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:17:48.3714682Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:17:48.3715238Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:17:48.3715736Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:17:48.3716211Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:17:48.3716704Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:17:48.3717184Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:17:48.3717630Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:17:48.3718141Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:17:48.3718678Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:17:48.3719157Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:17:48.3719651Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:17:48.3720158Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:17:48.3720649Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:17:48.3721146Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:17:48.3721630Z parm: NVreg_RmMsg:charp 2024-08-20T22:17:48.3722050Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:17:48.3722523Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:17:48.3722997Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:17:48.3723461Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:17:48.3723949Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:17:48.3724433Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:17:48.3724890Z parm: rm_firmware_active:charp 2024-08-20T22:17:48.3725471Z + HAS_NVIDIA_DRIVER=0 2024-08-20T22:17:48.3725876Z ++ command -v nvidia-smi 2024-08-20T22:17:48.3726287Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-20T22:17:48.3726655Z + set +e 2024-08-20T22:17:48.3727314Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-20T22:17:50.6703594Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-20T22:17:50.6704243Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:17:50.6705142Z + '[' 0 -ne 0 ']' 2024-08-20T22:17:50.6705538Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-20T22:17:50.6705926Z + HAS_NVIDIA_DRIVER=1 2024-08-20T22:17:50.6707096Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-20T22:17:50.6707966Z + set -e 2024-08-20T22:17:50.6708279Z + '[' 1 -eq 0 ']' 2024-08-20T22:17:50.6708854Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-20T22:17:50.6709539Z + post_install_nvidia_driver_common 2024-08-20T22:17:50.6710632Z + sudo modprobe nvidia 2024-08-20T22:17:50.7805836Z + echo 'After installing NVIDIA driver' 2024-08-20T22:17:50.7806421Z + lspci 2024-08-20T22:17:50.7806736Z After installing NVIDIA driver 2024-08-20T22:17:50.7913287Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:17:50.7914258Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:17:50.7915207Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:17:50.7916001Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:17:50.7916841Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:17:50.7917635Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:17:50.7918359Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:17:50.7919164Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:17:50.7919757Z + lsmod 2024-08-20T22:17:50.7954171Z Module Size Used by 2024-08-20T22:17:50.7954594Z nvidia_uvm 4706304 0 2024-08-20T22:17:50.7955004Z nvidia 54071296 1 nvidia_uvm 2024-08-20T22:17:50.7955442Z drm 602112 1 nvidia 2024-08-20T22:17:50.7955891Z drm_panel_orientation_quirks 28672 1 drm 2024-08-20T22:17:50.7956339Z backlight 24576 1 drm 2024-08-20T22:17:50.7956760Z i2c_core 106496 2 nvidia,drm 2024-08-20T22:17:50.7957217Z xt_conntrack 16384 1 2024-08-20T22:17:50.7957609Z nft_chain_nat 16384 3 2024-08-20T22:17:50.7957984Z xt_MASQUERADE 20480 1 2024-08-20T22:17:50.7958453Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-20T22:17:50.7958947Z nf_conntrack_netlink 57344 0 2024-08-20T22:17:50.7959530Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:17:50.7960165Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:17:50.7960634Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:17:50.7961083Z xfrm_user 57344 1 2024-08-20T22:17:50.7961482Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:17:50.7961900Z xt_addrtype 16384 2 2024-08-20T22:17:50.7962333Z nft_compat 20480 4 2024-08-20T22:17:50.7962793Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:17:50.7963411Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:17:50.7963960Z br_netfilter 36864 0 2024-08-20T22:17:50.7964373Z bridge 307200 1 br_netfilter 2024-08-20T22:17:50.7964825Z stp 16384 1 bridge 2024-08-20T22:17:50.7965248Z llc 16384 2 bridge,stp 2024-08-20T22:17:50.7965674Z overlay 167936 0 2024-08-20T22:17:50.7966046Z tls 114688 0 2024-08-20T22:17:50.7966420Z nls_ascii 16384 1 2024-08-20T22:17:50.7966788Z nls_cp437 20480 1 2024-08-20T22:17:50.7967585Z vfat 24576 1 2024-08-20T22:17:50.7967961Z fat 86016 1 vfat 2024-08-20T22:17:50.7968359Z sunrpc 692224 1 2024-08-20T22:17:50.7968724Z ena 167936 0 2024-08-20T22:17:50.7969097Z ghash_clmulni_intel 16384 0 2024-08-20T22:17:50.7969484Z ptp 36864 1 ena 2024-08-20T22:17:50.7969898Z pps_core 24576 1 ptp 2024-08-20T22:17:50.7970291Z aesni_intel 393216 0 2024-08-20T22:17:50.7970661Z i8042 45056 0 2024-08-20T22:17:50.7971040Z serio 28672 3 i8042 2024-08-20T22:17:50.7971619Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:17:50.7972143Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:17:50.7972646Z button 24576 0 2024-08-20T22:17:50.7973023Z sch_fq_codel 20480 17 2024-08-20T22:17:50.7973398Z dm_mod 188416 0 2024-08-20T22:17:50.7973770Z fuse 163840 1 2024-08-20T22:17:50.7974150Z configfs 57344 1 2024-08-20T22:17:50.7974531Z dax 45056 1 dm_mod 2024-08-20T22:17:50.7974926Z loop 36864 0 2024-08-20T22:17:50.7975314Z dmi_sysfs 20480 0 2024-08-20T22:17:50.7975803Z crc32_pclmul 16384 0 2024-08-20T22:17:50.7976280Z crc32c_intel 24576 0 2024-08-20T22:17:50.7976778Z efivarfs 24576 1 2024-08-20T22:17:50.7977252Z + modinfo nvidia 2024-08-20T22:17:50.7978136Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:17:50.7978863Z alias: char-major-195-* 2024-08-20T22:17:50.7979242Z version: 550.54.15 2024-08-20T22:17:50.7979592Z supported: external 2024-08-20T22:17:50.7979936Z license: NVIDIA 2024-08-20T22:17:50.7980310Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:17:50.7980798Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:17:50.7981259Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:17:50.7981730Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:17:50.7982268Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:17:50.7982763Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:17:50.7983267Z depends: i2c-core,drm 2024-08-20T22:17:50.7983629Z retpoline: Y 2024-08-20T22:17:50.7983933Z name: nvidia 2024-08-20T22:17:50.7984547Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:17:50.7985250Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:17:50.7985901Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:17:50.7986494Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:17:50.7986942Z parm: NVreg_RmLogonRC:int 2024-08-20T22:17:50.7987377Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:17:50.7987840Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:17:50.7988271Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:17:50.7988718Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:17:50.7989253Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:17:50.7989816Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:17:50.7990288Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:17:50.7990723Z parm: NVreg_EnableMSI:int 2024-08-20T22:17:50.7991148Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:17:50.7991604Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:17:50.7992138Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:17:50.7992724Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:17:50.7993283Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:17:50.7993885Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:17:50.7994483Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:17:50.7995102Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:17:50.7995839Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:17:50.7996327Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:17:50.7996863Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:17:50.7997409Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:17:50.7997905Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:17:50.7998366Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:17:50.7998852Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:17:50.7999328Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:17:50.7999867Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:17:50.8000367Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:17:50.8000894Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:17:50.8001376Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:17:50.8001864Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:17:50.8002414Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:17:50.8002902Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:17:50.8003403Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:17:50.8003877Z parm: NVreg_RmMsg:charp 2024-08-20T22:17:50.8004292Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:17:50.8005164Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:17:50.8005638Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:17:50.8006092Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:17:50.8006587Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:17:50.8007166Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:17:50.8007612Z parm: rm_firmware_active:charp 2024-08-20T22:17:50.8008015Z + set +e 2024-08-20T22:17:50.8008337Z + nvidia-smi 2024-08-20T22:17:52.3718144Z Tue Aug 20 22:17:52 2024 2024-08-20T22:17:52.3719051Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:17:52.3719913Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:17:52.3720717Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:17:52.3721533Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:17:52.3722423Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:17:52.3723122Z | | | MIG M. | 2024-08-20T22:17:52.3723685Z |=========================================+========================+======================| 2024-08-20T22:17:52.3812554Z | 0 NVIDIA A10G Off | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:17:52.3813290Z | 0% 29C P0 59W / 300W | 0MiB / 23028MiB | 6% Default | 2024-08-20T22:17:52.3813937Z | | | N/A | 2024-08-20T22:17:52.3814636Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:17:52.3815225Z 2024-08-20T22:17:52.3815884Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:17:52.3816505Z | Processes: | 2024-08-20T22:17:52.3817208Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:17:52.3817901Z | ID ID Usage | 2024-08-20T22:17:52.3818464Z |=========================================================================================| 2024-08-20T22:17:52.3819105Z | No running processes found | 2024-08-20T22:17:52.3820150Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:17:52.9903088Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-20T22:17:54.5624475Z NVIDIA A10G 2024-08-20T22:17:55.0072536Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:17:55.0073172Z + '[' 0 -eq 0 ']' 2024-08-20T22:17:55.0073601Z + echo 'INFO: Ignoring allowed status 0' 2024-08-20T22:17:55.0074042Z + set -e 2024-08-20T22:17:55.0074336Z INFO: Ignoring allowed status 0 2024-08-20T22:17:55.0083310Z == Installing nvidia container toolkit for amzn2023 == 2024-08-20T22:17:55.0089405Z + sudo yum install -y yum-utils 2024-08-20T22:17:55.4299692Z Last metadata expiration check: 0:09:34 ago on Tue Aug 20 22:08:21 2024. 2024-08-20T22:17:55.4520460Z Package dnf-utils-4.3.0-13.amzn2023.0.4.noarch is already installed. 2024-08-20T22:17:55.4822435Z Dependencies resolved. 2024-08-20T22:17:55.4947702Z Nothing to do. 2024-08-20T22:17:55.4948058Z Complete! 2024-08-20T22:17:55.5875216Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2024-08-20T22:17:55.5876387Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:17:55.5877892Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:17:55.8347235Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:17:55.9077333Z + sudo yum install -y nvidia-docker2 2024-08-20T22:17:56.3617278Z nvidia-container-toolkit 11 kB/s | 833 B 00:00 2024-08-20T22:17:56.3844485Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2024-08-20T22:17:56.4142294Z Dependencies resolved. 2024-08-20T22:17:56.4265075Z Nothing to do. 2024-08-20T22:17:56.4265388Z Complete! 2024-08-20T22:17:56.5253640Z + sudo systemctl restart docker 2024-08-20T22:18:50.2405984Z Tue Aug 20 22:18:50 2024 2024-08-20T22:18:50.2407010Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:18:50.2408128Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:18:50.2409064Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:18:50.2410006Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:18:50.2414437Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:18:50.2415180Z | | | MIG M. | 2024-08-20T22:18:50.2415737Z |=========================================+========================+======================| 2024-08-20T22:18:50.2529146Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:18:50.2529967Z | 0% 30C P0 59W / 300W | 0MiB / 23028MiB | 5% Default | 2024-08-20T22:18:50.2530627Z | | | N/A | 2024-08-20T22:18:50.2531368Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:18:50.2531970Z 2024-08-20T22:18:50.2532727Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:18:50.2533379Z | Processes: | 2024-08-20T22:18:50.2534097Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:18:50.2534797Z | ID ID Usage | 2024-08-20T22:18:50.2535373Z |=========================================================================================| 2024-08-20T22:18:50.2536392Z | No running processes found | 2024-08-20T22:18:50.2537165Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:18:50.7319276Z Command completed after 1 attempt(s). 2024-08-20T22:18:50.7400645Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:18:50.7401415Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:18:50.7402102Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-20T22:18:50.7402746Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:18:50.7417167Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:18:50.7417674Z env: 2024-08-20T22:18:50.7417960Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:18:50.7418425Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:18:50.7418903Z ##[endgroup] 2024-08-20T22:18:51.0329879Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:18:51.4459839Z Collecting psutil==5.9.1 2024-08-20T22:18:51.4803628Z Downloading psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB) 2024-08-20T22:18:51.5301940Z Collecting nvidia-ml-py==11.525.84 2024-08-20T22:18:51.5377069Z Downloading nvidia_ml_py-11.525.84-py3-none-any.whl (34 kB) 2024-08-20T22:18:51.6199482Z Installing collected packages: psutil, nvidia-ml-py 2024-08-20T22:18:51.7741926Z Successfully installed nvidia-ml-py-11.525.84 psutil-5.9.1 2024-08-20T22:18:51.9738249Z Prepare all required actions 2024-08-20T22:18:51.9738786Z Getting action download info 2024-08-20T22:18:52.0925562Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-20T22:18:52.3331320Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-20T22:18:52.4661830Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-20T22:18:52.4662300Z with: 2024-08-20T22:18:52.4662634Z name: linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-08-20T22:18:52.4663087Z s3-bucket: gha-artifacts 2024-08-20T22:18:52.4663422Z env: 2024-08-20T22:18:52.4663688Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:18:52.4664139Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:18:52.4664614Z ##[endgroup] 2024-08-20T22:18:52.4698365Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:18:52.4698806Z with: 2024-08-20T22:18:52.4699180Z name: linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-08-20T22:18:52.4699638Z s3-bucket: gha-artifacts 2024-08-20T22:18:52.4699999Z region: us-east-1 2024-08-20T22:18:52.4700305Z env: 2024-08-20T22:18:52.4700592Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:18:52.4701040Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:18:52.4701534Z ##[endgroup] 2024-08-20T22:18:52.9585612Z (node:52441) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:18:52.9586489Z 2024-08-20T22:18:52.9586807Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:18:52.9587637Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:18:52.9588729Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:18:53.0244163Z Found 1 objects with prefix pytorch/pytorch/10479310725/linux-focal-cuda12.4-py3.10-gcc9-sm86/ 2024-08-20T22:18:53.0246319Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:19:20.5978050Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:19:20.5987547Z Artifact download has finished successfully 2024-08-20T22:19:20.6314202Z ##[group]Run unzip -o artifacts.zip 2024-08-20T22:19:20.6314641Z unzip -o artifacts.zip 2024-08-20T22:19:20.6323641Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:20.6324326Z env: 2024-08-20T22:19:20.6324605Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:20.6325069Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:20.6325556Z ##[endgroup] 2024-08-20T22:19:20.6400885Z Archive: artifacts.zip 2024-08-20T22:19:20.6402633Z creating: dist/ 2024-08-20T22:19:22.7768130Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:19:22.7768942Z creating: build/custom_test_artifacts/ 2024-08-20T22:19:22.7769630Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-20T22:19:22.7770377Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-20T22:19:22.7771207Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:19:22.7777766Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:19:22.7778881Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-20T22:19:22.7779975Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:19:22.7780987Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:19:22.7782112Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:19:22.7783241Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:19:22.7784899Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:19:22.7785986Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:19:22.7787046Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:19:22.7788326Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:19:22.7789677Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:19:22.7791357Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:19:22.7792680Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:19:22.7794179Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:19:22.7795689Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:19:22.7796870Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:19:22.7797884Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:19:22.7839345Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:19:22.7880797Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:19:22.7882264Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:19:22.7931042Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:19:22.7932792Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:19:22.7934485Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:19:22.7936173Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:19:22.7938039Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:19:22.7939659Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:19:22.7941208Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:19:22.7942848Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:19:22.7944360Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:19:22.7945786Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:19:22.7947031Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:19:22.7948255Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:19:22.7949485Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:19:22.7950684Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:19:22.7951891Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:19:22.8020646Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:19:22.8094389Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:19:22.8095738Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:19:22.8096925Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:19:22.8098012Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:19:22.8099096Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-20T22:19:22.8100187Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-20T22:19:22.8101377Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-20T22:19:22.8102742Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-20T22:19:22.8104041Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-20T22:19:22.8105678Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-20T22:19:22.8106759Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-20T22:19:22.8107847Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-20T22:19:22.8108924Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-20T22:19:22.8109990Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-20T22:19:22.8111052Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-20T22:19:22.8126809Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-20T22:19:22.8273358Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-20T22:19:22.8275198Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-20T22:19:22.8276463Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-20T22:19:22.8278113Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-20T22:19:22.8279476Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-20T22:19:22.8280652Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-20T22:19:22.8281783Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-20T22:19:22.8282917Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-20T22:19:22.8284051Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-20T22:19:22.8285174Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-20T22:19:22.8286290Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-20T22:19:22.8301609Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-20T22:19:22.8385283Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-20T22:19:22.8386696Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:19:22.8387919Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:19:22.8389209Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-20T22:19:22.8390366Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-20T22:19:22.8391306Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-20T22:19:22.8392215Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-20T22:19:22.8393440Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-20T22:19:22.8394541Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-20T22:19:22.8395736Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-20T22:19:22.8516664Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-20T22:19:22.8579975Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-20T22:19:22.8580818Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-20T22:19:22.8581644Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-20T22:19:22.8582467Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:19:22.8589525Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:19:22.8590600Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-20T22:19:22.8591611Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:19:22.8592594Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:19:22.8593712Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:19:22.8594866Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:19:22.8596176Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:19:22.8597245Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:19:22.8598290Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:19:22.8599579Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:19:22.8601120Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:19:22.8602799Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:19:22.8604134Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:19:22.8605921Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:19:22.8607517Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:19:22.8608674Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:19:22.8609691Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:19:22.8650917Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:19:22.8692467Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:19:22.8693919Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:19:22.8740595Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:19:22.8742269Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:19:22.8744186Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:19:22.8745914Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:19:22.8747486Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:19:22.8749140Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:19:22.8750679Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:19:22.8752267Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:19:22.8753881Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:19:22.8755273Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:19:22.8756586Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:19:22.8757797Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:19:22.8759017Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:19:22.8760210Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:19:22.8761423Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:19:22.8830188Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:19:22.8904538Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:19:22.8906156Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:19:22.8907351Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:19:22.8908404Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:19:22.8909757Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-20T22:19:22.8910858Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-20T22:19:22.8912126Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-20T22:19:22.8913571Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-20T22:19:22.8914873Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-20T22:19:22.8916133Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-20T22:19:22.8917254Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-20T22:19:22.8918372Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-20T22:19:22.8919488Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-20T22:19:22.8920590Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-20T22:19:22.8921686Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-20T22:19:22.8938207Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-20T22:19:22.9004097Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-20T22:19:22.9005914Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:19:22.9007319Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:19:22.9008399Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-20T22:19:22.9009321Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-20T22:19:22.9010351Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-20T22:19:22.9011357Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-20T22:19:22.9012379Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-20T22:19:22.9013191Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-20T22:19:22.9014019Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-20T22:19:22.9062735Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-20T22:19:22.9063635Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-20T22:19:22.9064410Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-20T22:19:22.9065314Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:19:22.9071693Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:19:22.9072874Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-20T22:19:22.9073991Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:19:22.9075100Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:19:22.9076305Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:19:22.9077575Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:19:22.9078899Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:19:22.9079971Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:19:22.9081395Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:19:22.9082671Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:19:22.9084051Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:19:22.9085685Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:19:22.9087136Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:19:22.9088609Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:19:22.9090152Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:19:22.9091352Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:19:22.9092420Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:19:22.9134223Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:19:22.9175896Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:19:22.9177619Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:19:22.9224480Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:19:22.9226163Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:19:22.9227960Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:19:22.9229730Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:19:22.9231330Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:19:22.9232947Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:19:22.9234663Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:19:22.9236335Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:19:22.9237906Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:19:22.9239397Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:19:22.9240710Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:19:22.9242001Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:19:22.9243293Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:19:22.9244565Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:19:22.9245885Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:19:22.9314877Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:19:22.9389019Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:19:22.9390468Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:19:22.9391753Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:19:22.9392843Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:19:22.9394016Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-20T22:19:22.9395245Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-20T22:19:22.9396538Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-20T22:19:22.9398045Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-20T22:19:22.9399458Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-20T22:19:22.9400623Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-20T22:19:22.9401806Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-20T22:19:22.9403157Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-20T22:19:22.9404559Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-20T22:19:22.9405989Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-20T22:19:22.9407262Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-20T22:19:22.9408524Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-20T22:19:22.9531177Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-20T22:19:22.9532605Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-20T22:19:22.9534011Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-20T22:19:22.9535521Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-20T22:19:22.9537082Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-20T22:19:22.9538512Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-20T22:19:22.9539833Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-20T22:19:22.9541078Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-20T22:19:22.9542320Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-20T22:19:22.9543549Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-20T22:19:22.9544788Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-20T22:19:22.9560722Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-20T22:19:22.9616700Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-20T22:19:22.9618530Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:19:22.9619824Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:19:22.9620986Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-20T22:19:22.9622127Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-20T22:19:22.9623129Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-20T22:19:22.9624113Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-20T22:19:22.9625227Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-20T22:19:22.9626245Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-20T22:19:22.9627209Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-20T22:19:22.9730968Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-20T22:19:22.9773906Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-20T22:19:22.9774539Z creating: build/lib/ 2024-08-20T22:19:22.9862990Z inflating: build/lib/libprotobuf-lite.a 2024-08-20T22:19:23.0322135Z inflating: build/lib/libprotobuf.a 2024-08-20T22:19:23.0332243Z inflating: build/lib/libpthreadpool.a 2024-08-20T22:19:23.0340531Z inflating: build/lib/libcpuinfo.a 2024-08-20T22:19:23.0348755Z inflating: build/lib/libcpuinfo_internals.a 2024-08-20T22:19:23.0349573Z inflating: build/lib/libclog.a 2024-08-20T22:19:23.0367697Z inflating: build/lib/libnnpack.a 2024-08-20T22:19:23.0370469Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-20T22:19:23.0436113Z inflating: build/lib/libgtest.a 2024-08-20T22:19:23.0511312Z inflating: build/lib/libbenchmark.a 2024-08-20T22:19:23.0575136Z inflating: build/lib/libasmjit.a 2024-08-20T22:19:23.0582829Z inflating: build/lib/libittnotify.a 2024-08-20T22:19:23.0611327Z inflating: build/lib/libtensorpipe_uv.a 2024-08-20T22:19:23.0739435Z inflating: build/lib/libgloo.a 2024-08-20T22:19:23.0760303Z inflating: build/lib/libfmt.a 2024-08-20T22:19:23.0855605Z inflating: build/lib/libc10.so 2024-08-20T22:19:23.0857465Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-20T22:19:23.0859025Z inflating: build/lib/libtorch_global_deps.so 2024-08-20T22:19:23.1363795Z inflating: build/lib/libprotoc.a 2024-08-20T22:19:23.1382034Z inflating: build/lib/libgmock.a 2024-08-20T22:19:23.1382766Z inflating: build/lib/libgtest_main.a 2024-08-20T22:19:23.1383374Z inflating: build/lib/libbenchmark_main.a 2024-08-20T22:19:23.1403068Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-20T22:19:23.1795073Z inflating: build/lib/libgloo_cuda.a 2024-08-20T22:19:24.2000341Z inflating: build/lib/libdnnl.a 2024-08-20T22:19:24.2577417Z inflating: build/lib/libtensorpipe.a 2024-08-20T22:19:24.2634380Z inflating: build/lib/libc10_cuda.so 2024-08-20T22:19:24.2635056Z inflating: build/lib/libgmock_main.a 2024-08-20T22:19:24.3898796Z inflating: build/lib/libfbgemm.a 2024-08-20T22:19:24.4155595Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-20T22:19:24.4661892Z inflating: build/lib/libkineto.a 2024-08-20T22:19:24.4846034Z inflating: build/lib/libXNNPACK.a 2024-08-20T22:19:24.4889029Z inflating: build/lib/libonnx_proto.a 2024-08-20T22:19:24.5583314Z inflating: build/lib/libonnx.a 2024-08-20T22:19:27.0151868Z inflating: build/lib/libtorch_cpu.so 2024-08-20T22:19:27.0156772Z inflating: build/lib/libunbox_lib.a 2024-08-20T22:19:27.0160436Z inflating: build/lib/libshm.so 2024-08-20T22:19:29.0863810Z inflating: build/lib/libtorch_cuda.so 2024-08-20T22:19:29.0865331Z inflating: build/lib/libtorch.so 2024-08-20T22:19:29.0868045Z inflating: build/lib/libc10d_cuda_test.so 2024-08-20T22:19:29.9689942Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-20T22:19:30.1694582Z inflating: build/lib/libtorch_python.so 2024-08-20T22:19:30.1766973Z inflating: build/lib/libtorchbind_test.so 2024-08-20T22:19:30.1788031Z inflating: build/lib/libjitbackend_test.so 2024-08-20T22:19:30.1815052Z inflating: build/lib/libbackend_with_compiler.so 2024-08-20T22:19:30.1841577Z inflating: build/lib/libaoti_custom_ops.so 2024-08-20T22:19:30.1875870Z inflating: build/lib/libnnapi_backend.so 2024-08-20T22:19:30.1876461Z creating: build/bin/ 2024-08-20T22:19:30.1927309Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-20T22:19:30.1980200Z inflating: build/bin/c10_DeviceGuard_test 2024-08-20T22:19:30.2031607Z inflating: build/bin/c10_Device_test 2024-08-20T22:19:30.2091276Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-20T22:19:30.2145431Z inflating: build/bin/c10_Scalar_test 2024-08-20T22:19:30.2194773Z inflating: build/bin/c10_StreamGuard_test 2024-08-20T22:19:30.2246532Z inflating: build/bin/c10_SymInt_test 2024-08-20T22:19:30.2301467Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-20T22:19:30.2357788Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-20T22:19:30.2415014Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-20T22:19:30.2486901Z inflating: build/bin/c10_cow_test 2024-08-20T22:19:30.2540017Z inflating: build/bin/c10_Bitset_test 2024-08-20T22:19:30.2589161Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-20T22:19:30.2640056Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-20T22:19:30.2690874Z inflating: build/bin/c10_Half_test 2024-08-20T22:19:30.2747681Z inflating: build/bin/c10_LeftRight_test 2024-08-20T22:19:30.2804018Z inflating: build/bin/c10_Metaprogramming_test 2024-08-20T22:19:30.2854756Z inflating: build/bin/c10_Synchronized_test 2024-08-20T22:19:30.2912343Z inflating: build/bin/c10_ThreadLocal_test 2024-08-20T22:19:30.2964953Z inflating: build/bin/c10_TypeIndex_test 2024-08-20T22:19:30.3016054Z inflating: build/bin/c10_TypeList_test 2024-08-20T22:19:30.3065364Z inflating: build/bin/c10_TypeTraits_test 2024-08-20T22:19:30.3117833Z inflating: build/bin/c10_accumulate_test 2024-08-20T22:19:30.3173876Z inflating: build/bin/c10_bfloat16_test 2024-08-20T22:19:30.3224737Z inflating: build/bin/c10_bit_cast_test 2024-08-20T22:19:30.3281627Z inflating: build/bin/c10_complex_math_test 2024-08-20T22:19:30.3338307Z inflating: build/bin/c10_complex_test 2024-08-20T22:19:30.3391178Z inflating: build/bin/c10_exception_test 2024-08-20T22:19:30.3443770Z inflating: build/bin/c10_flags_test 2024-08-20T22:19:30.3493430Z inflating: build/bin/c10_generic_math_test 2024-08-20T22:19:30.3659297Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-20T22:19:30.3711141Z inflating: build/bin/c10_irange_test 2024-08-20T22:19:30.3765743Z inflating: build/bin/c10_lazy_test 2024-08-20T22:19:30.3823859Z inflating: build/bin/c10_logging_test 2024-08-20T22:19:30.3900714Z inflating: build/bin/c10_optional_test 2024-08-20T22:19:30.3963587Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-20T22:19:30.4019330Z inflating: build/bin/c10_registry_test 2024-08-20T22:19:30.4171465Z inflating: build/bin/c10_small_vector_test 2024-08-20T22:19:30.4223752Z inflating: build/bin/c10_ssize_test 2024-08-20T22:19:30.4276487Z inflating: build/bin/c10_string_util_test 2024-08-20T22:19:30.4328111Z inflating: build/bin/c10_tempfile_test 2024-08-20T22:19:30.4387410Z inflating: build/bin/c10_string_view_test 2024-08-20T22:19:30.4437024Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-20T22:19:30.4494625Z inflating: build/bin/c10_typeid_test 2024-08-20T22:19:30.4945079Z inflating: build/bin/protoc-3.13.0.0 2024-08-20T22:19:30.5397647Z inflating: build/bin/protoc 2024-08-20T22:19:30.5451045Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-20T22:19:30.5504233Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-20T22:19:30.5557675Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-20T22:19:30.5610318Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-20T22:19:30.5663759Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-20T22:19:30.5717067Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-20T22:19:30.5766457Z inflating: build/bin/c10_cuda_CUDATest 2024-08-20T22:19:30.5819933Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-20T22:19:30.6155596Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-20T22:19:30.6501514Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-20T22:19:30.6863187Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-20T22:19:30.6916164Z inflating: build/bin/BackoffTest 2024-08-20T22:19:30.6969736Z inflating: build/bin/FileStoreTest 2024-08-20T22:19:30.7026291Z inflating: build/bin/TCPStoreTest 2024-08-20T22:19:30.7080377Z inflating: build/bin/HashStoreTest 2024-08-20T22:19:30.7094394Z inflating: build/bin/ProcessGroupMPITest 2024-08-20T22:19:30.7150302Z inflating: build/bin/test_edge_op_registration 2024-08-20T22:19:30.7156305Z inflating: build/bin/torch_shm_manager 2024-08-20T22:19:30.7158936Z inflating: build/bin/example_allreduce 2024-08-20T22:19:30.7215195Z inflating: build/bin/test_dist_autograd 2024-08-20T22:19:30.7284834Z inflating: build/bin/test_cpp_rpc 2024-08-20T22:19:30.7287447Z inflating: build/bin/parallel_benchmark 2024-08-20T22:19:30.7355222Z inflating: build/bin/test_mobile_nnc 2024-08-20T22:19:30.7364478Z inflating: build/bin/aot_model_compiler_test 2024-08-20T22:19:30.7712980Z inflating: build/bin/test_lazy 2024-08-20T22:19:30.8904527Z inflating: build/bin/test_api 2024-08-20T22:19:30.8978684Z inflating: build/bin/Dict_test 2024-08-20T22:19:30.9031381Z inflating: build/bin/Dimname_test 2024-08-20T22:19:30.9096466Z inflating: build/bin/MaybeOwned_test 2024-08-20T22:19:30.9154600Z inflating: build/bin/NamedTensor_test 2024-08-20T22:19:30.9216396Z inflating: build/bin/apply_utils_test 2024-08-20T22:19:30.9276456Z inflating: build/bin/atest 2024-08-20T22:19:30.9339786Z inflating: build/bin/basic 2024-08-20T22:19:30.9395100Z inflating: build/bin/broadcast_test 2024-08-20T22:19:30.9448222Z inflating: build/bin/cpu_allocator_test 2024-08-20T22:19:30.9508077Z inflating: build/bin/cpu_generator_test 2024-08-20T22:19:30.9561562Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-20T22:19:30.9655660Z inflating: build/bin/cpu_rng_test 2024-08-20T22:19:30.9706192Z inflating: build/bin/dispatch_key_set_test 2024-08-20T22:19:30.9756750Z inflating: build/bin/dlconvertor_test 2024-08-20T22:19:30.9817417Z inflating: build/bin/extension_backend_test 2024-08-20T22:19:30.9872900Z inflating: build/bin/half_test 2024-08-20T22:19:30.9973945Z inflating: build/bin/ivalue_test 2024-08-20T22:19:31.0024287Z inflating: build/bin/lazy_tensor_test 2024-08-20T22:19:31.0079348Z inflating: build/bin/math_kernel_test 2024-08-20T22:19:31.0134519Z inflating: build/bin/memory_format_test 2024-08-20T22:19:31.0188190Z inflating: build/bin/memory_overlapping_test 2024-08-20T22:19:31.0242020Z inflating: build/bin/mobile_memory_cleanup 2024-08-20T22:19:31.0298260Z inflating: build/bin/native_test 2024-08-20T22:19:31.0349581Z inflating: build/bin/operator_name_test 2024-08-20T22:19:31.0401773Z inflating: build/bin/operators_test 2024-08-20T22:19:31.0454196Z inflating: build/bin/packedtensoraccessor_test 2024-08-20T22:19:31.0523889Z inflating: build/bin/pow_test 2024-08-20T22:19:31.0581286Z inflating: build/bin/quantized_test 2024-08-20T22:19:31.0632042Z inflating: build/bin/reduce_ops_test 2024-08-20T22:19:31.0684379Z inflating: build/bin/reportMemoryUsage_test 2024-08-20T22:19:31.0741744Z inflating: build/bin/scalar_tensor_test 2024-08-20T22:19:31.0800741Z inflating: build/bin/scalar_test 2024-08-20T22:19:31.0853853Z inflating: build/bin/StorageUtils_test 2024-08-20T22:19:31.0906982Z inflating: build/bin/stride_properties_test 2024-08-20T22:19:31.0986914Z inflating: build/bin/tensor_iterator_test 2024-08-20T22:19:31.1042034Z inflating: build/bin/test_parallel 2024-08-20T22:19:31.1045091Z inflating: build/bin/thread_init_test 2024-08-20T22:19:31.1101263Z inflating: build/bin/type_ptr_test 2024-08-20T22:19:31.1162231Z inflating: build/bin/type_test 2024-08-20T22:19:31.1215234Z inflating: build/bin/undefined_tensor_test 2024-08-20T22:19:31.1216617Z inflating: build/bin/verify_api_visibility 2024-08-20T22:19:31.1286923Z inflating: build/bin/legacy_vmap_test 2024-08-20T22:19:31.1339297Z inflating: build/bin/weakref_test 2024-08-20T22:19:31.1391039Z inflating: build/bin/wrapdim_test 2024-08-20T22:19:31.1452098Z inflating: build/bin/IListRef_test 2024-08-20T22:19:31.1504512Z inflating: build/bin/xla_tensor_test 2024-08-20T22:19:31.1613725Z inflating: build/bin/List_test 2024-08-20T22:19:31.1681318Z inflating: build/bin/KernelFunction_test 2024-08-20T22:19:31.1804173Z inflating: build/bin/kernel_function_legacy_test 2024-08-20T22:19:31.1901992Z inflating: build/bin/kernel_function_test 2024-08-20T22:19:31.2032796Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-20T22:19:31.2137654Z inflating: build/bin/kernel_lambda_test 2024-08-20T22:19:31.2199612Z inflating: build/bin/kernel_stackbased_test 2024-08-20T22:19:31.2296753Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-20T22:19:31.2348334Z inflating: build/bin/CppSignature_test 2024-08-20T22:19:31.2404923Z inflating: build/bin/backend_fallback_test 2024-08-20T22:19:31.2454227Z inflating: build/bin/op_allowlist_test 2024-08-20T22:19:31.2761499Z inflating: build/bin/op_registration_test 2024-08-20T22:19:31.2825913Z inflating: build/bin/inline_container_test 2024-08-20T22:19:31.2880031Z inflating: build/bin/cuda_apply_test 2024-08-20T22:19:31.2932192Z inflating: build/bin/cuda_allocator_test 2024-08-20T22:19:31.2987171Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-20T22:19:31.3046801Z inflating: build/bin/cuda_atomic_ops_test 2024-08-20T22:19:31.3117437Z inflating: build/bin/cuda_complex_math_test 2024-08-20T22:19:31.3177248Z inflating: build/bin/cuda_complex_test 2024-08-20T22:19:31.3227241Z inflating: build/bin/cuda_device_test 2024-08-20T22:19:31.3286859Z inflating: build/bin/cuda_cub_test 2024-08-20T22:19:31.3337464Z inflating: build/bin/cuda_dlconvertor_test 2024-08-20T22:19:31.3402675Z inflating: build/bin/cuda_distributions_test 2024-08-20T22:19:31.3461148Z inflating: build/bin/cuda_generator_test 2024-08-20T22:19:31.3511672Z inflating: build/bin/cuda_half_test 2024-08-20T22:19:31.3563330Z inflating: build/bin/cuda_integer_divider_test 2024-08-20T22:19:31.3613414Z inflating: build/bin/cuda_optional_test 2024-08-20T22:19:31.3665457Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-20T22:19:31.3718537Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-20T22:19:31.3770502Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-20T22:19:31.3832728Z inflating: build/bin/cuda_stream_test 2024-08-20T22:19:31.3882482Z inflating: build/bin/cuda_cudnn_test 2024-08-20T22:19:31.3935780Z inflating: build/bin/cuda_vectorized_test 2024-08-20T22:19:31.3950310Z inflating: build/bin/tutorial_tensorexpr 2024-08-20T22:19:31.4017191Z inflating: build/bin/ProcessGroupGlooTest 2024-08-20T22:19:31.4075554Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-20T22:19:31.4140134Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-20T22:19:31.4205349Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-20T22:19:31.5041535Z inflating: build/bin/test_tensorexpr 2024-08-20T22:19:31.5626364Z inflating: build/bin/test_jit 2024-08-20T22:19:31.5627491Z creating: .additional_ci_files/ 2024-08-20T22:19:31.5688084Z inflating: .additional_ci_files/test-times.json 2024-08-20T22:19:31.5931722Z inflating: .additional_ci_files/test-class-times.json 2024-08-20T22:19:31.5993049Z ##[group]Run rm artifacts.zip 2024-08-20T22:19:31.5993446Z rm artifacts.zip 2024-08-20T22:19:31.6002447Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:31.6002953Z env: 2024-08-20T22:19:31.6003237Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:31.6003696Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:31.6004178Z ##[endgroup] 2024-08-20T22:19:31.9577248Z ##[group]Run df -H 2024-08-20T22:19:31.9577580Z df -H 2024-08-20T22:19:31.9586413Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:31.9586930Z env: 2024-08-20T22:19:31.9587216Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:31.9587679Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:31.9588168Z ##[endgroup] 2024-08-20T22:19:31.9637557Z Filesystem Size Used Avail Use% Mounted on 2024-08-20T22:19:31.9638203Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-20T22:19:31.9638823Z tmpfs 34G 0 34G 0% /dev/shm 2024-08-20T22:19:31.9639378Z tmpfs 14G 553k 14G 1% /run 2024-08-20T22:19:31.9639830Z /dev/nvme0n1p1 161G 53G 109G 33% / 2024-08-20T22:19:31.9640534Z tmpfs 34G 4.1k 34G 1% /tmp 2024-08-20T22:19:31.9642435Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-20T22:19:31.9643130Z tmpfs 6.7G 0 6.7G 0% /run/user/0 2024-08-20T22:19:31.9682658Z Prepare all required actions 2024-08-20T22:19:31.9683105Z Getting action download info 2024-08-20T22:19:32.0871390Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-20T22:19:32.0871859Z with: 2024-08-20T22:19:32.0872117Z env: 2024-08-20T22:19:32.0872394Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:32.0872842Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:32.0873322Z ##[endgroup] 2024-08-20T22:19:32.1014789Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:19:32.1015240Z with: 2024-08-20T22:19:32.1015509Z name: td_results 2024-08-20T22:19:32.1015835Z s3-bucket: gha-artifacts 2024-08-20T22:19:32.1016190Z region: us-east-1 2024-08-20T22:19:32.1016495Z env: 2024-08-20T22:19:32.1016772Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:32.1017231Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:32.1017719Z ##[endgroup] 2024-08-20T22:19:32.5927749Z (node:52465) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:19:32.5928519Z 2024-08-20T22:19:32.5928790Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:19:32.5929515Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:19:32.5930372Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:19:32.6565747Z Found 0 objects with prefix pytorch/pytorch/10479310725/td_results/ 2024-08-20T22:19:32.6572719Z Artifact download has finished successfully 2024-08-20T22:19:32.6895268Z ##[group]Run mkdir -p .additional_ci_files 2024-08-20T22:19:32.6895755Z mkdir -p .additional_ci_files 2024-08-20T22:19:32.6896314Z mv td_results.json .additional_ci_files/td_results.json 2024-08-20T22:19:32.6905934Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:32.6906437Z env: 2024-08-20T22:19:32.6906734Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:32.6907212Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:32.6907705Z ##[endgroup] 2024-08-20T22:19:32.6964541Z mv: cannot stat 'td_results.json': No such file or directory 2024-08-20T22:19:32.6977296Z ##[error]Process completed with exit code 1. 2024-08-20T22:19:32.7019816Z ##[group]Run .github/scripts/parse_ref.py 2024-08-20T22:19:32.7020299Z .github/scripts/parse_ref.py 2024-08-20T22:19:32.7029110Z shell: /usr/bin/bash -e {0} 2024-08-20T22:19:32.7029466Z env: 2024-08-20T22:19:32.7029748Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:32.7030206Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:32.7030686Z ##[endgroup] 2024-08-20T22:19:32.7313629Z Prepare all required actions 2024-08-20T22:19:32.7364410Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-20T22:19:32.7364873Z with: 2024-08-20T22:19:32.7365412Z github-token: *** 2024-08-20T22:19:32.7365726Z env: 2024-08-20T22:19:32.7366008Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:32.7366471Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:32.7367056Z ##[endgroup] 2024-08-20T22:19:32.7498926Z ##[group]Run set -eux 2024-08-20T22:19:32.7499266Z set -eux 2024-08-20T22:19:32.7499852Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-20T22:19:32.7508829Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:32.7509320Z env: 2024-08-20T22:19:32.7509598Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:32.7510052Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:32.7510688Z GITHUB_TOKEN: *** 2024-08-20T22:19:32.7510996Z ##[endgroup] 2024-08-20T22:19:32.7539741Z + python3 .github/scripts/get_workflow_job_id.py 10479310725 i-0ab7c6e949d6545da 2024-08-20T22:19:34.5312510Z setting job-id=29026426126 2024-08-20T22:19:34.5313607Z setting job-name=cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:19:34.5536334Z Prepare all required actions 2024-08-20T22:19:34.5536785Z Getting action download info 2024-08-20T22:19:34.6563946Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-20T22:19:34.6564421Z with: 2024-08-20T22:19:34.6564903Z github-token: *** 2024-08-20T22:19:34.6566059Z test-matrix: {"include": [{"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:19:34.6567699Z job-name: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:19:34.6568426Z env: 2024-08-20T22:19:34.6568721Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:34.6569183Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:34.6569665Z ##[endgroup] 2024-08-20T22:19:34.6613041Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:19:34.6613624Z with: 2024-08-20T22:19:34.6613891Z shell: bash 2024-08-20T22:19:34.6614193Z timeout_minutes: 10 2024-08-20T22:19:34.6614519Z max_attempts: 5 2024-08-20T22:19:34.6614835Z retry_wait_seconds: 30 2024-08-20T22:19:34.6615926Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:19:34.6617076Z polling_interval_seconds: 1 2024-08-20T22:19:34.6617452Z warning_on_retry: true 2024-08-20T22:19:34.6617806Z continue_on_error: false 2024-08-20T22:19:34.6618141Z env: 2024-08-20T22:19:34.6618423Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:34.6618880Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:34.6619525Z GITHUB_TOKEN: *** 2024-08-20T22:19:34.6619848Z ##[endgroup] 2024-08-20T22:19:34.7364466Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:19:34.9769396Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:19:35.1219132Z Collecting requests==2.27.1 2024-08-20T22:19:35.1565610Z Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB) 2024-08-20T22:19:35.3487392Z Collecting pyyaml==6.0.1 2024-08-20T22:19:35.3531531Z Downloading PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (738 kB) 2024-08-20T22:19:35.3778263Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-20T22:19:35.4229385Z Collecting certifi>=2017.4.17 2024-08-20T22:19:35.4278330Z Downloading certifi-2024.7.4-py3-none-any.whl (162 kB) 2024-08-20T22:19:35.7298240Z Collecting charset-normalizer~=2.0.0 2024-08-20T22:19:35.7336487Z Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 2024-08-20T22:19:35.7383625Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-20T22:19:35.8148523Z Installing collected packages: charset-normalizer, certifi, requests, pyyaml 2024-08-20T22:19:35.9322901Z Successfully installed certifi-2024.7.4 charset-normalizer-2.0.12 pyyaml-6.0.1 requests-2.27.1 2024-08-20T22:19:36.7169687Z Command completed after 1 attempt(s). 2024-08-20T22:19:36.7228389Z ##[group]Run set -x 2024-08-20T22:19:36.7228730Z set -x 2024-08-20T22:19:36.7229029Z  2024-08-20T22:19:36.7229573Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:19:36.7230240Z # in runner workspace 2024-08-20T22:19:36.7230766Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-20T22:19:36.7239760Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:36.7240265Z env: 2024-08-20T22:19:36.7240548Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:36.7241007Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:36.7241496Z ##[endgroup] 2024-08-20T22:19:36.7271818Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-20T22:19:36.7508600Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:19:36.7509146Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:19:36.7509626Z echo "Job name: ${JOB_NAME}" 2024-08-20T22:19:36.7510032Z  2024-08-20T22:19:36.7510587Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:19:36.7511266Z # in runner workspace 2024-08-20T22:19:36.7511840Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-20T22:19:36.7512482Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-20T22:19:36.7512950Z  --job-name "${JOB_NAME}" \ 2024-08-20T22:19:36.7514293Z  --test-matrix "{"include": [{"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-08-20T22:19:36.7515581Z  --selected-test-configs "" \ 2024-08-20T22:19:36.7516039Z  --pr-number "${PR_NUMBER}" \ 2024-08-20T22:19:36.7516469Z  --tag "${TAG}" \ 2024-08-20T22:19:36.7516877Z  --event-name "${EVENT_NAME}" \ 2024-08-20T22:19:36.7517330Z  --schedule "${SCHEDULE}" \ 2024-08-20T22:19:36.7517760Z  --branch "${HEAD_BRANCH}" 2024-08-20T22:19:36.7526667Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:36.7527173Z env: 2024-08-20T22:19:36.7527463Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:36.7527916Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:36.7528591Z GITHUB_TOKEN: *** 2024-08-20T22:19:36.7529249Z JOB_NAME: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:19:36.7529971Z PR_NUMBER: 2024-08-20T22:19:36.7530290Z TAG: ciflow/inductor/133712 2024-08-20T22:19:36.7530670Z EVENT_NAME: push 2024-08-20T22:19:36.7530984Z SCHEDULE: 2024-08-20T22:19:36.7531282Z HEAD_BRANCH: 2024-08-20T22:19:36.7531577Z ##[endgroup] 2024-08-20T22:19:36.7564010Z Workflow: inductor 2024-08-20T22:19:36.7564987Z Job name: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:19:36.9880638Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-20T22:19:37.1473667Z ##[group]Run echo "Filtered matrix:" 2024-08-20T22:19:37.1474119Z echo "Filtered matrix:" 2024-08-20T22:19:37.1475326Z echo "{"include": [{"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]}" 2024-08-20T22:19:37.1476524Z  2024-08-20T22:19:37.1476801Z echo 2024-08-20T22:19:37.1477197Z echo "Is the current job unstable? False" 2024-08-20T22:19:37.1477649Z  2024-08-20T22:19:37.1477922Z echo 2024-08-20T22:19:37.1478286Z echo "Is keep-going label set? False" 2024-08-20T22:19:37.1478728Z  2024-08-20T22:19:37.1478993Z echo 2024-08-20T22:19:37.1479319Z echo "Renabled issues? " 2024-08-20T22:19:37.1488267Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:37.1488773Z env: 2024-08-20T22:19:37.1489053Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:37.1489518Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:37.1490003Z ##[endgroup] 2024-08-20T22:19:37.1518762Z Filtered matrix: 2024-08-20T22:19:37.1519961Z {include: [{config: inductor_timm, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}]} 2024-08-20T22:19:37.1521403Z 2024-08-20T22:19:37.1521608Z Is the current job unstable? False 2024-08-20T22:19:37.1522242Z 2024-08-20T22:19:37.1522669Z Is keep-going label set? False 2024-08-20T22:19:37.1522939Z 2024-08-20T22:19:37.1523063Z Renabled issues? 2024-08-20T22:19:37.1565532Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:19:37.1566260Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:19:37.1574698Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:37.1575209Z env: 2024-08-20T22:19:37.1575505Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:37.1575977Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:37.1576476Z JOB_TIMEOUT: 240 2024-08-20T22:19:37.1576792Z ##[endgroup] 2024-08-20T22:19:37.1664670Z ##[group]Run set -x 2024-08-20T22:19:37.1665257Z set -x 2024-08-20T22:19:37.1665661Z  2024-08-20T22:19:37.1666102Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-20T22:19:37.1666745Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-20T22:19:37.1667417Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-20T22:19:37.1668009Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-20T22:19:37.1668574Z else 2024-08-20T22:19:37.1668967Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:19:37.1669478Z fi 2024-08-20T22:19:37.1669912Z  2024-08-20T22:19:37.1670458Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-20T22:19:37.1671253Z # TODO: Stop building test binaries as part of the build phase 2024-08-20T22:19:37.1672081Z # Used for GPU_FLAG since that doesn't play nice 2024-08-20T22:19:37.1699458Z # shellcheck disable=SC2086,SC2090 2024-08-20T22:19:37.1699995Z container_name=$(docker run \ 2024-08-20T22:19:37.1700415Z  ${GPU_FLAG:-} \ 2024-08-20T22:19:37.1700795Z  -e BUILD_ENVIRONMENT \ 2024-08-20T22:19:37.1701199Z  -e PR_NUMBER \ 2024-08-20T22:19:37.1701564Z  -e GITHUB_ACTIONS \ 2024-08-20T22:19:37.1701949Z  -e GITHUB_REPOSITORY \ 2024-08-20T22:19:37.1702357Z  -e GITHUB_WORKFLOW \ 2024-08-20T22:19:37.1702744Z  -e GITHUB_JOB \ 2024-08-20T22:19:37.1703105Z  -e GITHUB_RUN_ID \ 2024-08-20T22:19:37.1703703Z  -e GITHUB_RUN_NUMBER \ 2024-08-20T22:19:37.1704115Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-20T22:19:37.1704509Z  -e JOB_ID \ 2024-08-20T22:19:37.1705192Z  -e JOB_NAME \ 2024-08-20T22:19:37.1705546Z  -e BASE_SHA \ 2024-08-20T22:19:37.1705889Z  -e BRANCH \ 2024-08-20T22:19:37.1706223Z  -e SHA1 \ 2024-08-20T22:19:37.1706562Z  -e AWS_DEFAULT_REGION \ 2024-08-20T22:19:37.1706964Z  -e IN_WHEEL_TEST \ 2024-08-20T22:19:37.1707340Z  -e SHARD_NUMBER \ 2024-08-20T22:19:37.1707708Z  -e TEST_CONFIG \ 2024-08-20T22:19:37.1708068Z  -e NUM_TEST_SHARDS \ 2024-08-20T22:19:37.1708464Z  -e REENABLED_ISSUES \ 2024-08-20T22:19:37.1708876Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-20T22:19:37.1709299Z  -e VERBOSE_TEST_LOGS \ 2024-08-20T22:19:37.1709696Z  -e TEST_SHOWLOCALS \ 2024-08-20T22:19:37.1710085Z  -e NO_TEST_TIMEOUT \ 2024-08-20T22:19:37.1710467Z  -e NO_TD \ 2024-08-20T22:19:37.1710802Z  -e TD_DISTRIBUTED \ 2024-08-20T22:19:37.1711183Z  -e PR_LABELS \ 2024-08-20T22:19:37.1711587Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-20T22:19:37.1712040Z  -e SCCACHE_BUCKET \ 2024-08-20T22:19:37.1712425Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-20T22:19:37.1712828Z  -e XLA_CUDA \ 2024-08-20T22:19:37.1713232Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-20T22:19:37.1713747Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-20T22:19:37.1714281Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-20T22:19:37.1714815Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-20T22:19:37.1715279Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-20T22:19:37.1715732Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-20T22:19:37.1716165Z  -e DASHBOARD_TAG \ 2024-08-20T22:19:37.1716645Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-20T22:19:37.1717197Z  --security-opt seccomp=unconfined \ 2024-08-20T22:19:37.1717661Z  --cap-add=SYS_PTRACE \ 2024-08-20T22:19:37.1718047Z  --ipc=host \ 2024-08-20T22:19:37.1718406Z  --shm-size="${SHM_SIZE}" \ 2024-08-20T22:19:37.1718802Z  --tty \ 2024-08-20T22:19:37.1719115Z  --detach \ 2024-08-20T22:19:37.1719692Z  --name="${container_name}" \ 2024-08-20T22:19:37.1720118Z  --user jenkins \ 2024-08-20T22:19:37.1720602Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-20T22:19:37.1721155Z  -w /var/lib/jenkins/workspace \ 2024-08-20T22:19:37.1721587Z  "${DOCKER_IMAGE}" 2024-08-20T22:19:37.1721935Z ) 2024-08-20T22:19:37.1722341Z # Propagate download.pytorch.org IP to container 2024-08-20T22:19:37.1723251Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-20T22:19:37.1724204Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-20T22:19:37.1725106Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-20T22:19:37.1734301Z shell: /usr/bin/bash -e {0} 2024-08-20T22:19:37.1734658Z env: 2024-08-20T22:19:37.1734936Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:37.1735404Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:37.1736013Z BUILD_ENVIRONMENT: linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-08-20T22:19:37.1736519Z PR_NUMBER: 2024-08-20T22:19:37.1736838Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-20T22:19:37.1737256Z GITHUB_WORKFLOW: inductor 2024-08-20T22:19:37.1737614Z GITHUB_JOB: test 2024-08-20T22:19:37.1737934Z GITHUB_RUN_ID: 10479310725 2024-08-20T22:19:37.1738290Z GITHUB_RUN_NUMBER: 88082 2024-08-20T22:19:37.1738646Z GITHUB_RUN_ATTEMPT: 1 2024-08-20T22:19:37.1738983Z JOB_ID: 29026426126 2024-08-20T22:19:37.1739784Z JOB_NAME: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:19:37.1740491Z BRANCH: 2024-08-20T22:19:37.1740833Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:19:37.1741363Z BASE_SHA: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:19:37.1741841Z TEST_CONFIG: inductor_timm 2024-08-20T22:19:37.1742191Z SHARD_NUMBER: 1 2024-08-20T22:19:37.1742505Z NUM_TEST_SHARDS: 2 2024-08-20T22:19:37.1742832Z REENABLED_ISSUES: 2024-08-20T22:19:37.1743166Z CONTINUE_THROUGH_ERROR: False 2024-08-20T22:19:37.1743558Z VERBOSE_TEST_LOGS: False 2024-08-20T22:19:37.1743912Z TEST_SHOWLOCALS: False 2024-08-20T22:19:37.1744262Z NO_TEST_TIMEOUT: False 2024-08-20T22:19:37.1744588Z NO_TD: False 2024-08-20T22:19:37.1744892Z TD_DISTRIBUTED: False 2024-08-20T22:19:37.1745316Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-20T22:19:37.1745808Z SCCACHE_S3_KEY_PREFIX: inductor 2024-08-20T22:19:37.1746186Z SHM_SIZE: 2g 2024-08-20T22:19:37.1747263Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:19:37.1748417Z XLA_CUDA: 2024-08-20T22:19:37.1748892Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:19:37.1749490Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-20T22:19:37.1749937Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-20T22:19:37.1750344Z DASHBOARD_TAG: 2024-08-20T22:19:37.1750860Z HUGGING_FACE_HUB_TOKEN: *** 2024-08-20T22:19:37.1751240Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-20T22:19:37.1751612Z ##[endgroup] 2024-08-20T22:19:37.1779992Z + [[ inductor_timm == \m\u\l\t\i\g\p\u ]] 2024-08-20T22:19:37.1780962Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *onnx* ]] 2024-08-20T22:19:37.1781524Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:19:37.1788770Z +++ nproc --ignore=2 2024-08-20T22:19:37.1819633Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479310725 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:19:44.0511306Z + container_name=b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T22:19:44.0514178Z + grep download.pytorch.org /etc/hosts 2024-08-20T22:19:44.0515613Z + docker exec -i b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-20T22:19:44.1761404Z + echo DOCKER_CONTAINER_ID=b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T22:19:44.1766652Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:19:44.1769692Z + docker exec -t b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-20T22:19:44.5723441Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-20T22:19:44.8977771Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-20T22:19:44.8980105Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-20T22:19:44.8983799Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-20T22:19:44.8987324Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-20T22:19:44.8990358Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-20T22:19:44.8995787Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.13.1) 2024-08-20T22:19:44.9013088Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-20T22:19:44.9026309Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-20T22:19:44.9043942Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.22.4) 2024-08-20T22:19:44.9427131Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-20T22:19:45.5207562Z Installing collected packages: torch 2024-08-20T22:19:55.3428415Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T22:19:55.3429801Z timm 1.0.9.dev0 requires torchvision, which is not installed. 2024-08-20T22:19:55.3430930Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-20T22:19:55.4253805Z + export TERM=vt100 2024-08-20T22:19:55.4254141Z + TERM=vt100 2024-08-20T22:19:55.4257123Z ++ dirname .ci/pytorch/test.sh 2024-08-20T22:19:55.4266332Z + source .ci/pytorch/common.sh 2024-08-20T22:19:55.4269899Z +++ dirname .ci/pytorch/common.sh 2024-08-20T22:19:55.4279110Z ++ source .ci/pytorch/common_utils.sh 2024-08-20T22:19:55.4280391Z +++ declare -f -t trap_add 2024-08-20T22:19:55.4287557Z ++ set -ex 2024-08-20T22:19:55.4288160Z ++ [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:19:55.4288695Z ++ BUILD_TEST_LIBTORCH=0 2024-08-20T22:19:55.4289204Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *rocm* ]] 2024-08-20T22:19:55.4291997Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-20T22:19:55.4312229Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-20T22:19:55.4312666Z + trap_add cleanup_workspace EXIT 2024-08-20T22:19:55.4313074Z + trap_add_cmd=cleanup_workspace 2024-08-20T22:19:55.4313446Z + shift 2024-08-20T22:19:55.4316503Z + for trap_add_name in "$@" 2024-08-20T22:19:55.4321890Z +++ trap -p EXIT 2024-08-20T22:19:55.4324490Z ++ eval 'extract_trap_cmd ' 2024-08-20T22:19:55.4324884Z +++ extract_trap_cmd 2024-08-20T22:19:55.4325257Z +++ printf '%s\n' '' 2024-08-20T22:19:55.4325662Z ++ printf '%s\n' cleanup_workspace 2024-08-20T22:19:55.4328473Z + trap -- ' 2024-08-20T22:19:55.4328880Z cleanup_workspace' EXIT 2024-08-20T22:19:55.4329363Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-20T22:19:56.1445815Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-20T22:19:56.1468188Z + echo 'Environment variables:' 2024-08-20T22:19:56.1468610Z Environment variables: 2024-08-20T22:19:56.1468953Z + env 2024-08-20T22:19:56.1479488Z INSTALLED_DB=yes 2024-08-20T22:19:56.1480013Z NV_LIBCUBLAS_VERSION=12.4.2.65-1 2024-08-20T22:19:56.1480538Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:19:56.1481080Z NV_NVML_DEV_VERSION=12.4.99-1 2024-08-20T22:19:56.1481748Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:19:56.1482321Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:19:56.1482838Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.20.5-1+cuda12.4 2024-08-20T22:19:56.1483354Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.20.5-1 2024-08-20T22:19:56.1483941Z BUILD_ENVIRONMENT=linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-08-20T22:19:56.1484444Z HOSTNAME=b4c2664d5dd5 2024-08-20T22:19:56.1485307Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_7527f097-eaf3-4d6f-84d6-d3052678327a 2024-08-20T22:19:56.1486102Z GITHUB_ACTION=__self 2024-08-20T22:19:56.1486486Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:19:56.1492022Z NVIDIA_REQUIRE_CUDA=cuda>=12.4 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 brand=tesla,driver>=535,driver<536 brand=unknown,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=geforce,driver>=535,driver<536 brand=geforcertx,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=titan,driver>=535,driver<536 brand=titanrtx,driver>=535,driver<536 2024-08-20T22:19:56.1497727Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-4=12.4.2.65-1 2024-08-20T22:19:56.1498238Z NV_NVTX_VERSION=12.4.99-1 2024-08-20T22:19:56.1498591Z GITHUB_RUN_NUMBER=88082 2024-08-20T22:19:56.1499144Z TEST_CONFIG=inductor_timm 2024-08-20T22:19:56.1499593Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:19:56.1500081Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:19:56.1500538Z NV_CUDA_CUDART_DEV_VERSION=12.4.99-1 2024-08-20T22:19:56.1500975Z NV_LIBCUSPARSE_VERSION=12.3.0.142-1 2024-08-20T22:19:56.1501381Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:19:56.1501778Z NV_LIBNPP_VERSION=12.2.5.2-1 2024-08-20T22:19:56.1502222Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:19:56.1502724Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:19:56.1503179Z GITHUB_REF_TYPE=tag 2024-08-20T22:19:56.1503511Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:19:56.1503892Z NCCL_VERSION=2.20.5-1 2024-08-20T22:19:56.1504280Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:19:56.1505080Z XLA_CUDA= 2024-08-20T22:19:56.1505592Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:19:56.1509459Z *** 2024-08-20T22:19:56.1509763Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:19:56.1510167Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:19:56.1510538Z GITHUB_ACTIONS=true 2024-08-20T22:19:56.1510877Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:56.1511407Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-4=12.4.99-1 2024-08-20T22:19:56.1511938Z NV_LIBNPP_PACKAGE=libnpp-12-4=12.2.5.2-1 2024-08-20T22:19:56.1512411Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:19:56.1512928Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:19:56.1513405Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:19:56.1514200Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:19:56.1515086Z UCC_HOME=/usr 2024-08-20T22:19:56.1515431Z NV_LIBCUBLAS_DEV_VERSION=12.4.2.65-1 2024-08-20T22:19:56.1515830Z VERBOSE_TEST_LOGS=False 2024-08-20T22:19:56.1516172Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:19:56.1516637Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-4 2024-08-20T22:19:56.1517133Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:19:56.1517599Z NV_CUDA_CUDART_VERSION=12.4.99-1 2024-08-20T22:19:56.1517972Z SHARD_NUMBER=1 2024-08-20T22:19:56.1518281Z GITHUB_REF_PROTECTED=false 2024-08-20T22:19:56.1518635Z HOME=/var/lib/jenkins 2024-08-20T22:19:56.1518994Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:19:56.1519432Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:19:56.1519911Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:19:56.1520384Z SCCACHE_S3_KEY_PREFIX=inductor 2024-08-20T22:19:56.1520751Z CUDA_VERSION=12.4.0 2024-08-20T22:19:56.1521180Z NV_LIBCUBLAS_PACKAGE=libcublas-12-4=12.4.2.65-1 2024-08-20T22:19:56.1521630Z NUM_TEST_SHARDS=2 2024-08-20T22:19:56.1521925Z UCX_HOME=/usr 2024-08-20T22:19:56.1522443Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-4=12.4.0-1 2024-08-20T22:19:56.1523524Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_7527f097-eaf3-4d6f-84d6-d3052678327a 2024-08-20T22:19:56.1524751Z JOB_NAME=cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:19:56.1525961Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_7527f097-eaf3-4d6f-84d6-d3052678327a 2024-08-20T22:19:56.1527178Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:19:56.1527835Z GITHUB_EVENT_NAME=push 2024-08-20T22:19:56.1528166Z DASHBOARD_TAG= 2024-08-20T22:19:56.1528470Z GITHUB_RUN_ID=10479310725 2024-08-20T22:19:56.1528937Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-4=12.2.5.2-1 2024-08-20T22:19:56.1529473Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-4 2024-08-20T22:19:56.1530479Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_7527f097-eaf3-4d6f-84d6-d3052678327a 2024-08-20T22:19:56.1531373Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:19:56.1531784Z NV_LIBNPP_DEV_VERSION=12.2.5.2-1 2024-08-20T22:19:56.1532160Z PR_NUMBER= 2024-08-20T22:19:56.1532447Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:19:56.1532783Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:19:56.1533366Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:19:56.1533836Z TERM=vt100 2024-08-20T22:19:56.1534171Z NV_LIBCUSPARSE_DEV_VERSION=12.3.0.142-1 2024-08-20T22:19:56.1534591Z INSTALLED_VISION=yes 2024-08-20T22:19:56.1534907Z BRANCH= 2024-08-20T22:19:56.1535193Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:19:56.1535586Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:19:56.1536011Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:19:56.1536765Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:19:56.1537561Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:19:56.1538043Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:19:56.1538515Z REENABLED_ISSUES= 2024-08-20T22:19:56.1538814Z SHLVL=1 2024-08-20T22:19:56.1539072Z MAX_JOBS=14 2024-08-20T22:19:56.1539389Z NV_CUDA_LIB_VERSION=12.4.0-1 2024-08-20T22:19:56.1539745Z NVARCH=x86_64 2024-08-20T22:19:56.1540048Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:19:56.1540503Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:19:56.1541039Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:19:56.1541534Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4 2024-08-20T22:19:56.1542203Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:19:56.1542767Z GITHUB_JOB=test 2024-08-20T22:19:56.1543174Z NV_LIBNCCL_PACKAGE=libnccl2=2.20.5-1+cuda12.4 2024-08-20T22:19:56.1543729Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:19:56.1544237Z NO_TEST_TIMEOUT=False 2024-08-20T22:19:56.1544658Z TD_DISTRIBUTED=False 2024-08-20T22:19:56.1545039Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.4.0-1 2024-08-20T22:19:56.1545478Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:19:56.1545890Z NV_NVPROF_VERSION=12.4.99-1 2024-08-20T22:19:56.1546254Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:19:56.1546603Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:19:56.1546956Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:19:56.1547984Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:19:56.1549046Z GITHUB_BASE_REF= 2024-08-20T22:19:56.1549376Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:19:56.1549750Z CI=true 2024-08-20T22:19:56.1550067Z NV_LIBNCCL_PACKAGE_VERSION=2.20.5-1 2024-08-20T22:19:56.1550483Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:19:56.1550862Z JOB_ID=29026426126 2024-08-20T22:19:56.1551172Z INSTALLED_PROTOBUF=yes 2024-08-20T22:19:56.1551513Z GITHUB_HEAD_REF= 2024-08-20T22:19:56.1551828Z GITHUB_ACTION_REF= 2024-08-20T22:19:56.1552269Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:19:56.1552730Z TEST_SHOWLOCALS=False 2024-08-20T22:19:56.1553069Z GITHUB_WORKFLOW=inductor 2024-08-20T22:19:56.1553430Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:19:56.1554418Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_7527f097-eaf3-4d6f-84d6-d3052678327a 2024-08-20T22:19:56.1555228Z NO_TD=False 2024-08-20T22:19:56.1555534Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:19:56.1555899Z _=/usr/bin/env 2024-08-20T22:19:56.1556390Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-20T22:19:56.1736997Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-20T22:19:56.1737927Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-20T22:19:56.1738794Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-20T22:19:56.1739674Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-20T22:19:56.1740284Z + BUILD_DIR=build 2024-08-20T22:19:56.1740612Z + BUILD_RENAMED_DIR=build_renamed 2024-08-20T22:19:56.1741007Z + BUILD_BIN_DIR=build/bin 2024-08-20T22:19:56.1741351Z + SHARD_NUMBER=1 2024-08-20T22:19:56.1741734Z + NUM_TEST_SHARDS=2 2024-08-20T22:19:56.1742062Z + export VALGRIND=ON 2024-08-20T22:19:56.1742383Z + VALGRIND=ON 2024-08-20T22:19:56.1743077Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *clang9* ]] 2024-08-20T22:19:56.1743579Z + [[ 0 == \1 ]] 2024-08-20T22:19:56.1743875Z + [[ False == \1 ]] 2024-08-20T22:19:56.1744357Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *bazel* ]] 2024-08-20T22:19:56.1744888Z ++ realpath build/custom_test_artifacts 2024-08-20T22:19:56.1755296Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-20T22:19:56.1755999Z + [[ -n '' ]] 2024-08-20T22:19:56.1756358Z + echo 'Environment variables' 2024-08-20T22:19:56.1756733Z Environment variables 2024-08-20T22:19:56.1757067Z + env 2024-08-20T22:19:56.1765985Z INSTALLED_DB=yes 2024-08-20T22:19:56.1766502Z NV_LIBCUBLAS_VERSION=12.4.2.65-1 2024-08-20T22:19:56.1767128Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:19:56.1767659Z NV_NVML_DEV_VERSION=12.4.99-1 2024-08-20T22:19:56.1768421Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:19:56.1769140Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:19:56.1769813Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.20.5-1+cuda12.4 2024-08-20T22:19:56.1770503Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.20.5-1 2024-08-20T22:19:56.1771145Z BUILD_ENVIRONMENT=linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-08-20T22:19:56.1771638Z HOSTNAME=b4c2664d5dd5 2024-08-20T22:19:56.1772531Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_7527f097-eaf3-4d6f-84d6-d3052678327a 2024-08-20T22:19:56.1773396Z GITHUB_ACTION=__self 2024-08-20T22:19:56.1773749Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:19:56.1779530Z NVIDIA_REQUIRE_CUDA=cuda>=12.4 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 brand=tesla,driver>=535,driver<536 brand=unknown,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=geforce,driver>=535,driver<536 brand=geforcertx,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=titan,driver>=535,driver<536 brand=titanrtx,driver>=535,driver<536 2024-08-20T22:19:56.1786664Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-4=12.4.2.65-1 2024-08-20T22:19:56.1787201Z NV_NVTX_VERSION=12.4.99-1 2024-08-20T22:19:56.1787580Z GITHUB_RUN_NUMBER=88082 2024-08-20T22:19:56.1787925Z TEST_CONFIG=inductor_timm 2024-08-20T22:19:56.1788290Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:19:56.1788776Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:19:56.1789234Z NV_CUDA_CUDART_DEV_VERSION=12.4.99-1 2024-08-20T22:19:56.1789675Z NV_LIBCUSPARSE_VERSION=12.3.0.142-1 2024-08-20T22:19:56.1790085Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:19:56.1790474Z NV_LIBNPP_VERSION=12.2.5.2-1 2024-08-20T22:19:56.1790914Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:19:56.1791407Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:19:56.1791935Z GITHUB_REF_TYPE=tag 2024-08-20T22:19:56.1792261Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:19:56.1792636Z NCCL_VERSION=2.20.5-1 2024-08-20T22:19:56.1793042Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:19:56.1793485Z XLA_CUDA= 2024-08-20T22:19:56.1793971Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:19:56.1794364Z *** 2024-08-20T22:19:56.1794640Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:19:56.1795038Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:19:56.1795399Z GITHUB_ACTIONS=true 2024-08-20T22:19:56.1795888Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:19:56.1796396Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-4=12.4.99-1 2024-08-20T22:19:56.1797171Z NV_LIBNPP_PACKAGE=libnpp-12-4=12.2.5.2-1 2024-08-20T22:19:56.1797642Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:19:56.1798160Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:19:56.1798650Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:19:56.1799439Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:19:56.1800145Z UCC_HOME=/usr 2024-08-20T22:19:56.1800483Z NV_LIBCUBLAS_DEV_VERSION=12.4.2.65-1 2024-08-20T22:19:56.1800888Z VERBOSE_TEST_LOGS=False 2024-08-20T22:19:56.1801230Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:19:56.1801692Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-4 2024-08-20T22:19:56.1802181Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:19:56.1802648Z NV_CUDA_CUDART_VERSION=12.4.99-1 2024-08-20T22:19:56.1803028Z SHARD_NUMBER=1 2024-08-20T22:19:56.1803336Z GITHUB_REF_PROTECTED=false 2024-08-20T22:19:56.1803686Z HOME=/var/lib/jenkins 2024-08-20T22:19:56.1804057Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:19:56.1804845Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:19:56.1805378Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:19:56.1805857Z SCCACHE_S3_KEY_PREFIX=inductor 2024-08-20T22:19:56.1806223Z CUDA_VERSION=12.4.0 2024-08-20T22:19:56.1806675Z NV_LIBCUBLAS_PACKAGE=libcublas-12-4=12.4.2.65-1 2024-08-20T22:19:56.1807233Z NUM_TEST_SHARDS=2 2024-08-20T22:19:56.1807537Z UCX_HOME=/usr 2024-08-20T22:19:56.1808357Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-4=12.4.0-1 2024-08-20T22:19:56.1809442Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_7527f097-eaf3-4d6f-84d6-d3052678327a 2024-08-20T22:19:56.1810660Z JOB_NAME=cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:19:56.1811887Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_7527f097-eaf3-4d6f-84d6-d3052678327a 2024-08-20T22:19:56.1812999Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:19:56.1813647Z GITHUB_EVENT_NAME=push 2024-08-20T22:19:56.1813975Z DASHBOARD_TAG= 2024-08-20T22:19:56.1814283Z GITHUB_RUN_ID=10479310725 2024-08-20T22:19:56.1814748Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-4=12.2.5.2-1 2024-08-20T22:19:56.1815287Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-4 2024-08-20T22:19:56.1816276Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_7527f097-eaf3-4d6f-84d6-d3052678327a 2024-08-20T22:19:56.1817215Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:19:56.1817649Z NV_LIBNPP_DEV_VERSION=12.2.5.2-1 2024-08-20T22:19:56.1818019Z PR_NUMBER= 2024-08-20T22:19:56.1818295Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:19:56.1818614Z VALGRIND=ON 2024-08-20T22:19:56.1818913Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:19:56.1819365Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:19:56.1819820Z TERM=vt100 2024-08-20T22:19:56.1820157Z NV_LIBCUSPARSE_DEV_VERSION=12.3.0.142-1 2024-08-20T22:19:56.1820573Z INSTALLED_VISION=yes 2024-08-20T22:19:56.1820885Z BRANCH= 2024-08-20T22:19:56.1821176Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:19:56.1821581Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:19:56.1821997Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:19:56.1822752Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:19:56.1823500Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:19:56.1823997Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:19:56.1824462Z REENABLED_ISSUES= 2024-08-20T22:19:56.1824764Z SHLVL=1 2024-08-20T22:19:56.1825033Z MAX_JOBS=14 2024-08-20T22:19:56.1825355Z NV_CUDA_LIB_VERSION=12.4.0-1 2024-08-20T22:19:56.1825703Z NVARCH=x86_64 2024-08-20T22:19:56.1826003Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:19:56.1826615Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:19:56.1827166Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:19:56.1827712Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4 2024-08-20T22:19:56.1828388Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:19:56.1829017Z GITHUB_JOB=test 2024-08-20T22:19:56.1829437Z NV_LIBNCCL_PACKAGE=libnccl2=2.20.5-1+cuda12.4 2024-08-20T22:19:56.1830041Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:19:56.1830598Z NO_TEST_TIMEOUT=False 2024-08-20T22:19:56.1830945Z TD_DISTRIBUTED=False 2024-08-20T22:19:56.1831352Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.4.0-1 2024-08-20T22:19:56.1831824Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:19:56.1832273Z NV_NVPROF_VERSION=12.4.99-1 2024-08-20T22:19:56.1832655Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:19:56.1833019Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:19:56.1833395Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:19:56.1834565Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:19:56.1835875Z GITHUB_BASE_REF= 2024-08-20T22:19:56.1836291Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:19:56.1836790Z CI=true 2024-08-20T22:19:56.1837200Z NV_LIBNCCL_PACKAGE_VERSION=2.20.5-1 2024-08-20T22:19:56.1837640Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:19:56.1838015Z JOB_ID=29026426126 2024-08-20T22:19:56.1838330Z INSTALLED_PROTOBUF=yes 2024-08-20T22:19:56.1838660Z GITHUB_HEAD_REF= 2024-08-20T22:19:56.1838959Z GITHUB_ACTION_REF= 2024-08-20T22:19:56.1839400Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:19:56.1839971Z TEST_SHOWLOCALS=False 2024-08-20T22:19:56.1840305Z GITHUB_WORKFLOW=inductor 2024-08-20T22:19:56.1840660Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:19:56.1841565Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_7527f097-eaf3-4d6f-84d6-d3052678327a 2024-08-20T22:19:56.1842362Z NO_TD=False 2024-08-20T22:19:56.1842664Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:19:56.1843029Z _=/usr/bin/env 2024-08-20T22:19:56.1843373Z + echo 'Testing pytorch' 2024-08-20T22:19:56.1843716Z Testing pytorch 2024-08-20T22:19:56.1844036Z + export LANG=C.UTF-8 2024-08-20T22:19:56.1844377Z + LANG=C.UTF-8 2024-08-20T22:19:56.1844669Z + PR_NUMBER= 2024-08-20T22:19:56.1844993Z + [[ inductor_timm == \d\e\f\a\u\l\t ]] 2024-08-20T22:19:56.1845450Z + [[ inductor_timm == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-20T22:19:56.1845905Z + [[ inductor_timm == \s\l\o\w ]] 2024-08-20T22:19:56.1846511Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-08-20T22:19:56.1847296Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *cuda* ]] 2024-08-20T22:19:56.1847843Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:19:56.1848319Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:19:56.1848753Z + [[ inductor_timm == *crossref* ]] 2024-08-20T22:19:56.1849298Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:19:56.1849941Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *xpu* ]] 2024-08-20T22:19:56.1850600Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:19:56.1851169Z + pip_install --user ninja==1.10.2 2024-08-20T22:19:56.1851703Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-20T22:19:57.3953723Z Collecting ninja==1.10.2 2024-08-20T22:19:57.4144677Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-20T22:19:57.4752153Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-20T22:19:58.0747074Z Installing collected packages: ninja 2024-08-20T22:19:58.0825270Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-20T22:19:58.0826537Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-20T22:19:58.1301827Z Successfully installed ninja-1.10.2 2024-08-20T22:19:58.2085213Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:19:58.2087550Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:19:58.2089068Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:19:58.2089585Z + install_tlparse 2024-08-20T22:19:58.2089986Z + pip_install --user tlparse==0.3.25 2024-08-20T22:19:58.2090562Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-20T22:19:58.6441369Z Collecting tlparse==0.3.25 2024-08-20T22:19:58.6629395Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-20T22:19:58.7212815Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-20T22:19:59.3818936Z Installing collected packages: tlparse 2024-08-20T22:19:59.4203798Z Successfully installed tlparse-0.3.25 2024-08-20T22:19:59.4980966Z ++ python -m site --user-base 2024-08-20T22:19:59.5224917Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:19:59.5226600Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *asan* ]] 2024-08-20T22:19:59.5227266Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *-debug* ]] 2024-08-20T22:19:59.5228274Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:19:59.5229222Z + echo 'We are not in debug mode: linux-focal-cuda12.4-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-08-20T22:19:59.5230418Z We are not in debug mode: linux-focal-cuda12.4-py3.10-gcc9-sm86. Expect the assertion to pass 2024-08-20T22:19:59.5231372Z + cd test 2024-08-20T22:19:59.5232019Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-20T22:20:01.1601065Z + [[ inductor_timm == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-20T22:20:01.1601877Z + [[ inductor_timm == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-20T22:20:01.1605416Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-20T22:20:01.1606095Z + [[ inductor_timm == *pr_time_benchmarks* ]] 2024-08-20T22:20:01.1606660Z + [[ inductor_timm == *dynamo_eager* ]] 2024-08-20T22:20:01.1607323Z + [[ inductor_timm == *aot_eager* ]] 2024-08-20T22:20:01.1607832Z + [[ inductor_timm == *aot_inductor* ]] 2024-08-20T22:20:01.1608347Z + [[ inductor_timm == *inductor* ]] 2024-08-20T22:20:01.1608809Z + [[ inductor_timm != *perf* ]] 2024-08-20T22:20:01.1609528Z + DYNAMO_BENCHMARK_FLAGS+=(--inductor) 2024-08-20T22:20:01.1610018Z + [[ inductor_timm == *dynamic* ]] 2024-08-20T22:20:01.1614600Z + [[ inductor_timm == *cpu* ]] 2024-08-20T22:20:01.1615282Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-20T22:20:01.1644028Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:20:01.1644734Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-08-20T22:20:01.1650519Z + cd test 2024-08-20T22:20:01.1651022Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-20T22:20:02.6562652Z PyTorch built with: 2024-08-20T22:20:02.6563410Z - GCC 9.4 2024-08-20T22:20:02.6563885Z - C++ Version: 201703 2024-08-20T22:20:02.6564892Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:20:02.6565972Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:20:02.6566676Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:20:02.6567309Z - LAPACK is enabled (usually provided by MKL) 2024-08-20T22:20:02.6567795Z - NNPACK is enabled 2024-08-20T22:20:02.6568187Z - CPU capability usage: AVX2 2024-08-20T22:20:02.6568601Z - CUDA Runtime 12.4 2024-08-20T22:20:02.6569478Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-08-20T22:20:02.6570041Z - CuDNN 90.1 2024-08-20T22:20:02.6570372Z - Magma 2.6.1 2024-08-20T22:20:02.6577061Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.4, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-20T22:20:02.6583225Z 2024-08-20T22:20:02.9824720Z + cd test 2024-08-20T22:20:02.9825512Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-20T22:20:04.3320419Z ATen/Parallel: 2024-08-20T22:20:04.3321392Z at::get_num_threads() : 8 2024-08-20T22:20:04.3321880Z at::get_num_interop_threads() : 16 2024-08-20T22:20:04.3322392Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:20:04.3322827Z omp_get_max_threads() : 8 2024-08-20T22:20:04.3323908Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:20:04.3324726Z mkl_get_max_threads() : 8 2024-08-20T22:20:04.3325358Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:20:04.3326000Z std::thread::hardware_concurrency() : 16 2024-08-20T22:20:04.3326434Z Environment variables: 2024-08-20T22:20:04.3326792Z OMP_NUM_THREADS : [not set] 2024-08-20T22:20:04.3327259Z MKL_NUM_THREADS : [not set] 2024-08-20T22:20:04.3327641Z ATen parallel backend: OpenMP 2024-08-20T22:20:04.3327896Z 2024-08-20T22:20:04.6160707Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:20:04.6161472Z + [[ inductor_timm == *backward* ]] 2024-08-20T22:20:04.6162027Z + [[ inductor_timm == *xla* ]] 2024-08-20T22:20:04.6162435Z + [[ inductor_timm == *executorch* ]] 2024-08-20T22:20:04.6162896Z + [[ inductor_timm == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-20T22:20:04.6163531Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:20:04.6164105Z + [[ inductor_timm == distributed ]] 2024-08-20T22:20:04.6164698Z + [[ inductor_timm == *inductor_distributed* ]] 2024-08-20T22:20:04.6165277Z + [[ inductor_timm == *inductor-halide* ]] 2024-08-20T22:20:04.6165871Z + [[ inductor_timm == *inductor-micro-benchmark* ]] 2024-08-20T22:20:04.6166515Z + [[ inductor_timm == *huggingface* ]] 2024-08-20T22:20:04.6167198Z + [[ inductor_timm == *timm* ]] 2024-08-20T22:20:04.6167624Z + install_torchvision 2024-08-20T22:20:04.6167955Z + local orig_preload 2024-08-20T22:20:04.6168282Z + local commit 2024-08-20T22:20:04.6168606Z ++ get_pinned_commit vision 2024-08-20T22:20:04.6169000Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-20T22:20:04.6184561Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:04.6185173Z + orig_preload= 2024-08-20T22:20:04.6185639Z + '[' -n '' ']' 2024-08-20T22:20:04.6186544Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:04.6188194Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:04.9519425Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:04.9524952Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-s0ucaiks 2024-08-20T22:20:04.9557484Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-s0ucaiks 2024-08-20T22:20:06.4336560Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-20T22:20:06.4363078Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:07.7526229Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:08.0517891Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:20:10.5645788Z Preparing metadata (setup.py) ... [?25l- \ done 2024-08-20T22:20:10.5678325Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.22.4) 2024-08-20T22:20:10.5682146Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git40ec5f6) 2024-08-20T22:20:10.5687050Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-20T22:20:10.5765567Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-20T22:20:10.5770668Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-20T22:20:10.5772934Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-20T22:20:10.5775300Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-20T22:20:10.5778552Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-20T22:20:10.5783489Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.1) 2024-08-20T22:20:10.5797020Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-20T22:20:10.6275601Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-20T22:20:10.6338296Z Building wheels for collected packages: torchvision 2024-08-20T22:21:27.2817119Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2024-08-20T22:21:27.2849528Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2024978 sha256=278f3d7b8b1e766e475b07fb4bb6a7374bf4abe09e75f36547f1f248bf0eed77 2024-08-20T22:21:27.2851169Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-08-20T22:21:27.2884672Z Successfully built torchvision 2024-08-20T22:21:27.7673847Z Installing collected packages: torchvision 2024-08-20T22:21:28.1781867Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-20T22:21:28.3187516Z + '[' -n '' ']' 2024-08-20T22:21:28.3187907Z + id=0 2024-08-20T22:21:28.3188607Z + test_dynamo_benchmark timm_models 0 2024-08-20T22:21:28.3193284Z ++ pwd 2024-08-20T22:21:28.3196109Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:21:28.3196684Z + local suite=timm_models 2024-08-20T22:21:28.3197034Z + shift 2024-08-20T22:21:28.3197320Z + local shard_id=0 2024-08-20T22:21:28.3197906Z + shift 2024-08-20T22:21:28.3198236Z + [[ inductor_timm == *perf_compare* ]] 2024-08-20T22:21:28.3198671Z + [[ inductor_timm == *perf* ]] 2024-08-20T22:21:28.3199073Z + [[ inductor_timm == *cpu* ]] 2024-08-20T22:21:28.3199477Z + [[ inductor_timm == *aot_inductor* ]] 2024-08-20T22:21:28.3200255Z + test_single_dynamo_benchmark inference timm_models 0 --inference --bfloat16 2024-08-20T22:21:28.3201075Z ++ pwd 2024-08-20T22:21:28.3204857Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:21:28.3205607Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:21:28.3231549Z + local name=inference 2024-08-20T22:21:28.3232427Z + shift 2024-08-20T22:21:28.3233029Z + local suite=timm_models 2024-08-20T22:21:28.3233449Z + shift 2024-08-20T22:21:28.3233780Z + local shard_id=0 2024-08-20T22:21:28.3234412Z + shift 2024-08-20T22:21:28.3234866Z + partition_flags=() 2024-08-20T22:21:28.3235297Z + local partition_flags 2024-08-20T22:21:28.3235726Z + [[ -n 2 ]] 2024-08-20T22:21:28.3236048Z + [[ -n 0 ]] 2024-08-20T22:21:28.3236665Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-08-20T22:21:28.3237319Z + [[ inductor_timm == *perf_compare* ]] 2024-08-20T22:21:28.3237762Z + [[ inductor_timm == *perf* ]] 2024-08-20T22:21:28.3239447Z + [[ inductor_timm == *aot_inductor* ]] 2024-08-20T22:21:28.3240043Z + [[ inductor_timm == *_avx2* ]] 2024-08-20T22:21:28.3240500Z + [[ inductor_timm == *_avx512* ]] 2024-08-20T22:21:28.3242118Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --inductor --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv 2024-08-20T22:21:34.7762198Z 2024-08-20T22:21:35.0995229Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:21:35.0995565Z 2024-08-20T22:21:35.2031166Z model.safetensors: 0% 0.00/95.5M [00:00 will be ignored 2024-08-20T22:43:03.2806412Z pass 2024-08-20T22:43:03.3306958Z TIMING: entire_frame_compile:62.55337 code_gen:27.048 inductor_compile:43.94729 backend_compile:49.93862 total_wall_time:72.7089 2024-08-20T22:43:03.3308400Z STATS: call_* op count: 989 | FakeTensorMode.__torch_dispatch__:49959 | FakeTensor.__torch_dispatch__:15634 | ProxyTorchDispatchMode.__torch_dispatch__:21953 2024-08-20T22:43:03.3309533Z Dynamo produced 3 graphs covering 989 ops with 7 graph breaks (5 unique) 2024-08-20T22:43:10.4322825Z 2024-08-20T22:43:11.4635129Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:43:11.4635629Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:43:11.4636591Z cuda train botnet26t_256 2024-08-20T22:44:00.7383096Z pass 2024-08-20T22:44:00.7488988Z TIMING: entire_frame_compile:29.54523 code_gen:22.50027 inductor_compile:31.33398 backend_compile:23.95447 total_wall_time:45.30715 2024-08-20T22:44:00.7491383Z STATS: call_* op count: 264 | FakeTensorMode.__torch_dispatch__:23638 | FakeTensor.__torch_dispatch__:5542 | ProxyTorchDispatchMode.__torch_dispatch__:9029 2024-08-20T22:44:00.7492530Z Dynamo produced 2 graphs covering 264 ops with 6 graph breaks (5 unique) 2024-08-20T22:44:06.2977209Z 2024-08-20T22:44:10.4485887Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:44:10.4486470Z loading model: 0it [00:04, ?it/s] 2024-08-20T22:44:10.4487023Z cuda train cait_m36_384 2024-08-20T22:44:10.4493406Z Traceback (most recent call last): 2024-08-20T22:44:10.4494218Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2507, in validate_model 2024-08-20T22:44:10.4494976Z self.model_iter_fn(model, example_inputs) 2024-08-20T22:44:10.4495851Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 404, in forward_and_backward_pass 2024-08-20T22:44:10.4500361Z pred = mod(*cloned_inputs) 2024-08-20T22:44:10.4501607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:44:10.4502537Z return self._call_impl(*args, **kwargs) 2024-08-20T22:44:10.4503533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:44:10.4504378Z return forward_call(*args, **kwargs) 2024-08-20T22:44:10.4505649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 438, in forward 2024-08-20T22:44:10.4506550Z x = self.forward_features(x) 2024-08-20T22:44:10.4507450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 423, in forward_features 2024-08-20T22:44:10.4508251Z x = self.blocks(x) 2024-08-20T22:44:10.4509149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:44:10.4510050Z return self._call_impl(*args, **kwargs) 2024-08-20T22:44:10.4510992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:44:10.4512846Z return forward_call(*args, **kwargs) 2024-08-20T22:44:10.4514154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/container.py", line 250, in forward 2024-08-20T22:44:10.4515022Z input = module(input) 2024-08-20T22:44:10.4515937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:44:10.4516814Z return self._call_impl(*args, **kwargs) 2024-08-20T22:44:10.4517759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:44:10.4518591Z return forward_call(*args, **kwargs) 2024-08-20T22:44:10.4519438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 199, in forward 2024-08-20T22:44:10.4520309Z x = x + self.drop_path(self.gamma_1 * self.attn(self.norm1(x))) 2024-08-20T22:44:10.4521388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:44:10.4522271Z return self._call_impl(*args, **kwargs) 2024-08-20T22:44:10.4523202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:44:10.4524027Z return forward_call(*args, **kwargs) 2024-08-20T22:44:10.4524864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 153, in forward 2024-08-20T22:44:10.4525665Z x = (attn @ v).transpose(1, 2).reshape(B, N, C) 2024-08-20T22:44:10.4528989Z torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 21.98 GiB of which 16.44 MiB is free. Process 67157 has 21.95 GiB memory in use. Of the allocated memory 21.21 GiB is allocated by PyTorch, and 430.66 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) 2024-08-20T22:44:10.4531883Z 2024-08-20T22:44:10.4532182Z The above exception was the direct cause of the following exception: 2024-08-20T22:44:10.4532637Z 2024-08-20T22:44:10.4532785Z Traceback (most recent call last): 2024-08-20T22:44:10.4533440Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T22:44:10.4534099Z ) = runner.load_model( 2024-08-20T22:44:10.4534759Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 328, in load_model 2024-08-20T22:44:10.4535532Z self.validate_model(model, example_inputs) 2024-08-20T22:44:10.4536287Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2509, in validate_model 2024-08-20T22:44:10.4537050Z raise RuntimeError("Eager run failed") from e 2024-08-20T22:44:10.4537521Z RuntimeError: Eager run failed 2024-08-20T22:44:10.4537782Z 2024-08-20T22:44:10.4537902Z eager_fail_to_run 2024-08-20T22:44:14.3686523Z 2024-08-20T22:44:15.3306127Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:44:15.3306919Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:44:15.3307616Z cuda train coat_lite_mini 2024-08-20T22:45:06.9743185Z pass 2024-08-20T22:45:06.9860377Z TIMING: entire_frame_compile:27.65856 code_gen:22.18828 inductor_compile:31.97439 backend_compile:22.8592 total_wall_time:47.92914 2024-08-20T22:45:06.9861771Z STATS: call_* op count: 456 | FakeTensorMode.__torch_dispatch__:35432 | FakeTensor.__torch_dispatch__:5893 | ProxyTorchDispatchMode.__torch_dispatch__:14386 2024-08-20T22:45:06.9862898Z Dynamo produced 2 graphs covering 456 ops with 6 graph breaks (5 unique) 2024-08-20T22:45:12.7723256Z 2024-08-20T22:45:14.4969521Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:45:14.4970009Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:45:14.4970453Z cuda train convit_base 2024-08-20T22:45:14.5532274Z WARNING:common:fp64 golden ref were not generated for convit_base. Setting accuracy check to cosine 2024-08-20T22:46:40.7718131Z pass 2024-08-20T22:46:40.7964961Z TIMING: entire_frame_compile:52.61255 code_gen:32.37295 inductor_compile:51.04017 backend_compile:41.97455 total_wall_time:83.85341 2024-08-20T22:46:40.7967668Z STATS: call_* op count: 1142 | FakeTensorMode.__torch_dispatch__:85781 | FakeTensor.__torch_dispatch__:15645 | ProxyTorchDispatchMode.__torch_dispatch__:40285 2024-08-20T22:46:40.7968825Z Dynamo produced 3 graphs covering 1142 ops with 7 graph breaks (5 unique) 2024-08-20T22:46:47.8588461Z 2024-08-20T22:46:48.8078810Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:46:48.8079769Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:46:48.8080595Z cuda train convmixer_768_32 2024-08-20T22:47:46.5991029Z pass 2024-08-20T22:47:46.6180127Z TIMING: entire_frame_compile:35.75907 code_gen:16.18615 inductor_compile:28.74208 backend_compile:26.32795 total_wall_time:48.59353 2024-08-20T22:47:46.6181749Z STATS: call_* op count: 300 | FakeTensorMode.__torch_dispatch__:38513 | FakeTensor.__torch_dispatch__:11061 | ProxyTorchDispatchMode.__torch_dispatch__:15494 2024-08-20T22:47:46.6183087Z Dynamo produced 2 graphs covering 300 ops with 5 graph breaks (4 unique) 2024-08-20T22:47:52.9540474Z 2024-08-20T22:47:54.7839173Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:47:54.7840067Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:47:54.7840931Z cuda train convnext_base 2024-08-20T22:48:54.0562998Z W0820 22:48:54.055000 15454 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:50:16.0803936Z pass 2024-08-20T22:50:16.1309530Z TIMING: entire_frame_compile:106.32479 code_gen:47.8981 inductor_compile:77.57993 backend_compile:82.59672 total_wall_time:128.96341 2024-08-20T22:50:16.1311360Z STATS: call_* op count: 1508 | FakeTensorMode.__torch_dispatch__:80534 | FakeTensor.__torch_dispatch__:27882 | ProxyTorchDispatchMode.__torch_dispatch__:35464 2024-08-20T22:50:16.1312560Z Dynamo produced 3 graphs covering 1508 ops with 7 graph breaks (5 unique) 2024-08-20T22:50:25.7388259Z 2024-08-20T22:50:26.5963479Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:50:26.5964627Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:50:26.5965670Z cuda train crossvit_9_240 2024-08-20T22:51:22.9403140Z W0820 22:51:22.939000 16273 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:52:26.3718242Z pass 2024-08-20T22:52:26.3852527Z TIMING: entire_frame_compile:93.28522 code_gen:41.45333 inductor_compile:66.58511 backend_compile:75.27469 total_wall_time:112.0979 2024-08-20T22:52:26.3856657Z STATS: call_* op count: 1312 | FakeTensorMode.__torch_dispatch__:77628 | FakeTensor.__torch_dispatch__:23568 | ProxyTorchDispatchMode.__torch_dispatch__:34361 2024-08-20T22:52:26.3857803Z Dynamo produced 3 graphs covering 1312 ops with 7 graph breaks (5 unique) 2024-08-20T22:52:35.0994079Z 2024-08-20T22:52:36.4155452Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:52:36.4155933Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:52:36.4156382Z cuda train cspdarknet53 2024-08-20T22:53:40.4985437Z W0820 22:53:40.497000 17042 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:54:34.3844456Z pass 2024-08-20T22:54:34.3976916Z TIMING: entire_frame_compile:89.93904 code_gen:40.08818 inductor_compile:65.68409 backend_compile:70.21345 total_wall_time:108.87642 2024-08-20T22:54:34.3978488Z STATS: call_* op count: 950 | FakeTensorMode.__torch_dispatch__:60180 | FakeTensor.__torch_dispatch__:22264 | ProxyTorchDispatchMode.__torch_dispatch__:24624 2024-08-20T22:54:34.3979868Z Dynamo produced 3 graphs covering 950 ops with 7 graph breaks (5 unique) 2024-08-20T22:54:43.0982925Z 2024-08-20T22:54:44.7429974Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:54:44.7430585Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:54:44.7432355Z cuda train deit_base_distilled_patch16_224 2024-08-20T22:55:14.3343899Z W0820 22:55:14.333000 17969 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:55:50.8672774Z pass 2024-08-20T22:55:50.9184674Z TIMING: entire_frame_compile:48.76133 code_gen:21.10945 inductor_compile:34.9214 backend_compile:39.22681 total_wall_time:57.65447 2024-08-20T22:55:50.9186067Z STATS: call_* op count: 753 | FakeTensorMode.__torch_dispatch__:40529 | FakeTensor.__torch_dispatch__:13130 | ProxyTorchDispatchMode.__torch_dispatch__:18136 2024-08-20T22:55:50.9188443Z Dynamo produced 3 graphs covering 753 ops with 7 graph breaks (5 unique) 2024-08-20T22:55:57.3542331Z 2024-08-20T22:55:58.7853999Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:55:58.7854608Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:55:58.7855131Z cuda train dla102 2024-08-20T22:57:29.6442692Z W0820 22:57:29.643000 18284 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:58:53.1973363Z pass 2024-08-20T22:58:53.2197695Z TIMING: entire_frame_compile:136.52981 code_gen:60.91625 inductor_compile:97.29947 backend_compile:105.69356 total_wall_time:161.71561 2024-08-20T22:58:53.2200487Z STATS: call_* op count: 1457 | FakeTensorMode.__torch_dispatch__:90022 | FakeTensor.__torch_dispatch__:32600 | ProxyTorchDispatchMode.__torch_dispatch__:36994 2024-08-20T22:58:53.2202283Z Dynamo produced 3 graphs covering 1457 ops with 7 graph breaks (5 unique) 2024-08-20T22:59:04.5567251Z 2024-08-20T22:59:06.3893781Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:59:06.3894753Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:00:01.9096469Z cuda train dm_nfnet_f0 2024-08-20T23:00:01.9097108Z pass 2024-08-20T23:00:01.9612344Z TIMING: entire_frame_compile:29.89421 code_gen:17.89781 inductor_compile:30.53152 backend_compile:21.23124 total_wall_time:49.328 2024-08-20T23:00:01.9615712Z STATS: call_* op count: 594 | FakeTensorMode.__torch_dispatch__:38376 | ProxyTorchDispatchMode.__torch_dispatch__:16005 | FakeTensor.__torch_dispatch__:11566 2024-08-20T23:00:01.9617437Z Dynamo produced 2 graphs covering 594 ops with 6 graph breaks (5 unique) 2024-08-20T23:00:07.9598389Z 2024-08-20T23:00:09.6202729Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:00:09.6203361Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:00:09.6203940Z cuda train dpn107 2024-08-20T23:01:51.3319159Z pass 2024-08-20T23:01:51.3975100Z TIMING: entire_frame_compile:64.7904 code_gen:33.65684 inductor_compile:54.2894 backend_compile:48.04144 total_wall_time:90.29552 2024-08-20T23:01:51.3976622Z STATS: call_* op count: 634 | FakeTensorMode.__torch_dispatch__:69021 | FakeTensor.__torch_dispatch__:18035 | ProxyTorchDispatchMode.__torch_dispatch__:28636 2024-08-20T23:01:51.3977750Z Dynamo produced 2 graphs covering 634 ops with 6 graph breaks (5 unique) 2024-08-20T23:01:59.3900515Z 2024-08-20T23:02:00.4257501Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:02:00.4258122Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:02:00.4258649Z cuda train eca_botnext26ts_256 2024-08-20T23:02:48.8584928Z W0820 23:02:48.857000 21810 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:03:17.7328759Z pass 2024-08-20T23:03:17.7447086Z TIMING: entire_frame_compile:54.78439 code_gen:31.78081 inductor_compile:46.84381 backend_compile:44.30528 total_wall_time:72.15338 2024-08-20T23:03:17.7449883Z STATS: call_* op count: 634 | FakeTensorMode.__torch_dispatch__:37417 | FakeTensor.__torch_dispatch__:12499 | ProxyTorchDispatchMode.__torch_dispatch__:15134 2024-08-20T23:03:17.7451476Z Dynamo produced 3 graphs covering 634 ops with 7 graph breaks (5 unique) 2024-08-20T23:03:24.3052959Z 2024-08-20T23:03:25.3588304Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:03:25.3589379Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:03:25.3589832Z cuda train eca_halonext26ts 2024-08-20T23:04:18.1962894Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-20T23:04:18.1964099Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 404, in torch_dynamo_resume_in_forward_and_backward_pass_at_402 2024-08-20T23:04:18.1965205Z pred = mod(*cloned_inputs) 2024-08-20T23:04:18.1969140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/byobnet.py", line 1433, in forward 2024-08-20T23:04:18.1970099Z x = self.forward_features(x) 2024-08-20T23:04:18.1971330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/byobnet.py", line 1425, in forward_features 2024-08-20T23:04:18.1972159Z x = self.stages(x) 2024-08-20T23:04:18.1972968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/byobnet.py", line 894, in forward 2024-08-20T23:04:18.1973733Z x = self.self_attn(x) 2024-08-20T23:04:18.1974560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/layers/halo_attn.py", line 191, in forward 2024-08-20T23:04:18.1975640Z kv = kv.unfold(2, self.win_size, self.block_size).unfold(3, self.win_size, self.block_size).reshape( 2024-08-20T23:04:18.1976220Z 2024-08-20T23:04:18.3821763Z W0820 23:04:18.381000 22710 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:04:47.7847634Z pass 2024-08-20T23:04:47.7921724Z TIMING: entire_frame_compile:57.62266 code_gen:35.18329 inductor_compile:50.81919 backend_compile:45.32371 total_wall_time:78.02973 2024-08-20T23:04:47.7923625Z STATS: call_* op count: 666 | FakeTensorMode.__torch_dispatch__:38619 | FakeTensor.__torch_dispatch__:12313 | ProxyTorchDispatchMode.__torch_dispatch__:15754 2024-08-20T23:04:47.7924977Z Dynamo produced 3 graphs covering 666 ops with 7 graph breaks (5 unique) 2024-08-20T23:04:54.3861169Z 2024-08-20T23:04:55.3036208Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:04:55.3036828Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:04:55.3037337Z cuda train ese_vovnet19b_dw 2024-08-20T23:05:31.1616482Z W0820 23:05:31.160000 23173 torch/_logging/_internal.py:1058] [7/0] Profiler function will be ignored 2024-08-20T23:05:58.1761733Z pass 2024-08-20T23:05:58.1836667Z TIMING: entire_frame_compile:43.44992 code_gen:26.69353 inductor_compile:38.0636 backend_compile:33.75603 total_wall_time:57.59668 2024-08-20T23:05:58.1838040Z STATS: call_* op count: 434 | FakeTensorMode.__torch_dispatch__:26302 | FakeTensor.__torch_dispatch__:9349 | ProxyTorchDispatchMode.__torch_dispatch__:10392 2024-08-20T23:05:58.1840179Z Dynamo produced 4 graphs covering 434 ops with 7 graph breaks (5 unique) 2024-08-20T23:06:04.3199915Z 2024-08-20T23:06:05.2815976Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:06:05.2816494Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:06:05.2816970Z cuda train fbnetc_100 2024-08-20T23:06:59.6891063Z W0820 23:06:59.688000 23915 torch/_logging/_internal.py:1058] [7/0] Profiler function will be ignored 2024-08-20T23:07:53.8496610Z pass 2024-08-20T23:07:53.8543617Z TIMING: entire_frame_compile:85.67822 code_gen:37.2997 inductor_compile:59.36201 backend_compile:66.71092 total_wall_time:99.53398 2024-08-20T23:07:53.8545015Z STATS: call_* op count: 870 | FakeTensorMode.__torch_dispatch__:55101 | FakeTensor.__torch_dispatch__:20088 | ProxyTorchDispatchMode.__torch_dispatch__:22083 2024-08-20T23:07:53.8546158Z Dynamo produced 4 graphs covering 870 ops with 7 graph breaks (5 unique) 2024-08-20T23:08:02.2571257Z 2024-08-20T23:08:03.4052896Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:08:03.4053480Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:08:03.4055400Z cuda train fbnetv3_b 2024-08-20T23:09:38.8018854Z pass 2024-08-20T23:09:38.8108247Z TIMING: entire_frame_compile:59.5959 code_gen:35.07986 inductor_compile:58.26784 backend_compile:45.21307 total_wall_time:89.14199 2024-08-20T23:09:38.8109818Z STATS: call_* op count: 459 | FakeTensorMode.__torch_dispatch__:58710 | FakeTensor.__torch_dispatch__:19672 | ProxyTorchDispatchMode.__torch_dispatch__:23345 2024-08-20T23:09:38.8110956Z Dynamo produced 2 graphs covering 459 ops with 6 graph breaks (5 unique) 2024-08-20T23:09:47.0663431Z 2024-08-20T23:09:48.3765866Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:09:48.3766476Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:09:48.3767086Z cuda train gernet_l 2024-08-20T23:10:36.4785128Z pass 2024-08-20T23:10:36.4996174Z TIMING: entire_frame_compile:31.90675 code_gen:15.41759 inductor_compile:25.83545 backend_compile:23.23003 total_wall_time:43.27713 2024-08-20T23:10:36.4997548Z STATS: call_* op count: 250 | FakeTensorMode.__torch_dispatch__:30714 | FakeTensor.__torch_dispatch__:8886 | ProxyTorchDispatchMode.__torch_dispatch__:12116 2024-08-20T23:10:36.4999640Z Dynamo produced 2 graphs covering 250 ops with 6 graph breaks (5 unique) 2024-08-20T23:10:42.5110301Z 2024-08-20T23:10:43.5264829Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:10:43.5265393Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:10:43.5265835Z cuda train ghostnet_100 2024-08-20T23:12:04.4954571Z pass 2024-08-20T23:12:04.5005545Z TIMING: entire_frame_compile:53.53664 code_gen:31.47439 inductor_compile:49.08242 backend_compile:40.69 total_wall_time:75.81079 2024-08-20T23:12:04.5006959Z STATS: call_* op count: 405 | FakeTensorMode.__torch_dispatch__:50795 | FakeTensor.__torch_dispatch__:13895 | ProxyTorchDispatchMode.__torch_dispatch__:19864 2024-08-20T23:12:04.5008571Z Dynamo produced 2 graphs covering 405 ops with 6 graph breaks (5 unique) 2024-08-20T23:12:11.7986605Z 2024-08-20T23:12:13.1659638Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:12:13.1660132Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:12:13.1660583Z cuda train gluon_inception_v3 2024-08-20T23:12:51.5784485Z W0820 23:12:51.577000 28781 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:14:03.0264238Z pass 2024-08-20T23:14:03.0501273Z TIMING: entire_frame_compile:97.8302 inductor_compile:43.83486 backend_compile:70.98365 code_gen:21.30985 total_wall_time:101.28633 2024-08-20T23:14:03.0503020Z STATS: call_* op count: 1281 | FakeTensorMode.__torch_dispatch__:68829 | FakeTensor.__torch_dispatch__:14446 | ProxyTorchDispatchMode.__torch_dispatch__:33213 2024-08-20T23:14:03.0505327Z Dynamo produced 3 graphs covering 1281 ops with 7 graph breaks (5 unique) 2024-08-20T23:14:10.8479984Z 2024-08-20T23:14:11.8779448Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:14:11.8779947Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:14:11.8780401Z cuda train gmixer_24_224 2024-08-20T23:15:07.4943355Z pass 2024-08-20T23:15:07.5201282Z TIMING: entire_frame_compile:31.75965 code_gen:18.202 inductor_compile:30.18222 backend_compile:24.06335 total_wall_time:50.52299 2024-08-20T23:15:07.5202905Z STATS: call_* op count: 587 | FakeTensorMode.__torch_dispatch__:49688 | FakeTensor.__torch_dispatch__:11442 | ProxyTorchDispatchMode.__torch_dispatch__:23826 2024-08-20T23:15:07.5204019Z Dynamo produced 2 graphs covering 587 ops with 6 graph breaks (5 unique) 2024-08-20T23:15:13.6517011Z 2024-08-20T23:15:14.5892434Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:16:07.7391103Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:16:07.7391695Z cuda train gmlp_s16_224 2024-08-20T23:16:07.7393045Z W0820 23:16:07.738000 29300 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:17:18.4688066Z pass 2024-08-20T23:17:18.4896871Z TIMING: entire_frame_compile:95.13384 code_gen:41.37197 inductor_compile:68.72039 backend_compile:75.74374 total_wall_time:114.84315 2024-08-20T23:17:18.4898638Z STATS: call_* op count: 1396 | FakeTensorMode.__torch_dispatch__:77799 | FakeTensor.__torch_dispatch__:26803 | ProxyTorchDispatchMode.__torch_dispatch__:35364 2024-08-20T23:17:18.4899779Z Dynamo produced 3 graphs covering 1396 ops with 7 graph breaks (5 unique) 2024-08-20T23:17:27.4981109Z 2024-08-20T23:17:29.2919977Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:17:29.2920577Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:17:29.2921068Z cuda train hrnet_w18 2024-08-20T23:21:20.6967418Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-20T23:21:20.6968581Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 404, in torch_dynamo_resume_in_forward_and_backward_pass_at_402 2024-08-20T23:21:20.6969569Z pred = mod(*cloned_inputs) 2024-08-20T23:21:20.6972587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/hrnet.py", line 792, in forward 2024-08-20T23:21:20.6975341Z y = self.forward_features(x) 2024-08-20T23:21:20.6976843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/hrnet.py", line 770, in forward_features 2024-08-20T23:21:20.6977910Z yl = self.stages(x) 2024-08-20T23:21:20.6978857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/hrnet.py", line 751, in stages 2024-08-20T23:21:20.6979842Z yl = self.stage2(xl) 2024-08-20T23:21:20.6980719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/hrnet.py", line 506, in forward 2024-08-20T23:21:20.6981449Z x = module(x) 2024-08-20T23:21:20.6982187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/hrnet.py", line 484, in forward 2024-08-20T23:21:20.6983401Z y = y + f(x[j]) 2024-08-20T23:21:20.6983608Z 2024-08-20T23:21:25.0629377Z pass 2024-08-20T23:21:25.0881405Z TIMING: entire_frame_compile:169.00436 code_gen:71.73532 inductor_compile:124.9057 backend_compile:120.32845 total_wall_time:224.71228 2024-08-20T23:21:25.0882845Z STATS: call_* op count: 1475 | FakeTensorMode.__torch_dispatch__:173385 | FakeTensor.__torch_dispatch__:46888 | ProxyTorchDispatchMode.__torch_dispatch__:71305 2024-08-20T23:21:25.0883987Z Dynamo produced 2 graphs covering 1475 ops with 5 graph breaks (4 unique) 2024-08-20T23:21:39.7062512Z 2024-08-20T23:21:41.0703977Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:21:41.0704842Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:21:41.0705490Z cuda train inception_v3 2024-08-20T23:22:20.6161615Z pass 2024-08-20T23:22:20.6379056Z TIMING: entire_frame_compile:31.04311 inductor_compile:6.48534 backend_compile:18.00956 total_wall_time:34.46206 2024-08-20T23:22:20.6380470Z STATS: call_* op count: 412 | FakeTensorMode.__torch_dispatch__:39410 | FakeTensor.__torch_dispatch__:46 | ProxyTorchDispatchMode.__torch_dispatch__:20409 2024-08-20T23:22:20.6381577Z Dynamo produced 2 graphs covering 412 ops with 6 graph breaks (5 unique) 2024-08-20T23:22:25.5153551Z 2024-08-20T23:22:27.1968636Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:22:27.1969238Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:22:27.1969769Z cuda train jx_nest_base 2024-08-20T23:23:45.1995286Z W0820 23:23:45.198000 31005 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:24:58.7454148Z pass 2024-08-20T23:24:58.7861553Z TIMING: entire_frame_compile:110.30961 code_gen:55.06101 inductor_compile:86.46094 backend_compile:89.16438 total_wall_time:137.61664 2024-08-20T23:24:58.7864572Z STATS: call_* op count: 1671 | FakeTensorMode.__torch_dispatch__:98646 | FakeTensor.__torch_dispatch__:27894 | ProxyTorchDispatchMode.__torch_dispatch__:43281 2024-08-20T23:24:58.7865752Z Dynamo produced 3 graphs covering 1671 ops with 7 graph breaks (5 unique) 2024-08-20T23:25:08.5153223Z 2024-08-20T23:25:09.3102784Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:25:09.3103423Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:25:09.3104002Z cuda train lcnet_050 2024-08-20T23:25:40.3510308Z pass 2024-08-20T23:25:40.3535286Z TIMING: entire_frame_compile:18.96762 code_gen:11.53142 inductor_compile:19.18474 backend_compile:14.78534 total_wall_time:28.3645 2024-08-20T23:25:40.3536677Z STATS: call_* op count: 129 | FakeTensorMode.__torch_dispatch__:17856 | FakeTensor.__torch_dispatch__:6005 | ProxyTorchDispatchMode.__torch_dispatch__:6908 2024-08-20T23:25:40.3537787Z Dynamo produced 2 graphs covering 129 ops with 6 graph breaks (5 unique) 2024-08-20T23:25:45.5924762Z 2024-08-20T23:25:46.7398003Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:25:46.7398576Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:25:46.7399149Z cuda train levit_128 2024-08-20T23:27:04.0269373Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-20T23:27:04.0270682Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 404, in torch_dynamo_resume_in_forward_and_backward_pass_at_402 2024-08-20T23:27:04.0271643Z pred = mod(*cloned_inputs) 2024-08-20T23:27:04.0272814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/levit.py", line 718, in forward 2024-08-20T23:27:04.0273584Z x = self.forward_features(x) 2024-08-20T23:27:04.0274469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/levit.py", line 709, in forward_features 2024-08-20T23:27:04.0275250Z x = self.stages(x) 2024-08-20T23:27:04.0276015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/levit.py", line 520, in forward 2024-08-20T23:27:04.0276747Z x = self.blocks(x) 2024-08-20T23:27:04.0278050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/levit.py", line 458, in forward 2024-08-20T23:27:04.0278947Z x = x + self.drop_path1(self.attn(x)) 2024-08-20T23:27:04.0280109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/levit.py", line 237, in forward 2024-08-20T23:27:04.0281004Z attn = q @ k * self.scale + self.get_attention_biases(x.device) 2024-08-20T23:27:04.0282061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/levit.py", line 212, in get_attention_biases 2024-08-20T23:27:04.0282986Z return self.attention_biases[:, self.attention_bias_idxs] 2024-08-20T23:27:04.0283369Z 2024-08-20T23:27:04.3185495Z W0820 23:27:04.317000 32378 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:27:54.4668445Z pass 2024-08-20T23:27:54.4707898Z TIMING: entire_frame_compile:96.11117 code_gen:45.74186 inductor_compile:73.3091 backend_compile:77.26382 total_wall_time:121.58029 2024-08-20T23:27:54.4709504Z STATS: call_* op count: 1261 | FakeTensorMode.__torch_dispatch__:76161 | FakeTensor.__torch_dispatch__:22877 | ProxyTorchDispatchMode.__torch_dispatch__:32073 2024-08-20T23:27:54.4710624Z Dynamo produced 3 graphs covering 1261 ops with 7 graph breaks (5 unique) 2024-08-20T23:28:03.9479703Z 2024-08-20T23:28:05.2898685Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:28:05.2899318Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:28:05.2899850Z cuda train mixer_b16_224 2024-08-20T23:28:35.9954027Z W0820 23:28:35.994000 33442 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:29:11.3952409Z pass 2024-08-20T23:29:11.4384080Z TIMING: entire_frame_compile:49.2255 code_gen:21.63038 inductor_compile:35.32166 backend_compile:40.27825 total_wall_time:58.88844 2024-08-20T23:29:11.4385478Z STATS: call_* op count: 670 | FakeTensorMode.__torch_dispatch__:38728 | FakeTensor.__torch_dispatch__:13368 | ProxyTorchDispatchMode.__torch_dispatch__:17868 2024-08-20T23:29:11.4386613Z Dynamo produced 3 graphs covering 670 ops with 7 graph breaks (5 unique) 2024-08-20T23:29:14.6946244Z accuracy pass_rate=96.67% 2024-08-20T23:29:14.6946919Z calls_captured gmean=0.00x mean=779.033x 2024-08-20T23:29:14.6949577Z unique_graphs gmean=0.00x mean=2.533x 2024-08-20T23:29:14.6954890Z graph_breaks gmean=0.00x mean=6.267x 2024-08-20T23:29:14.6959638Z unique_graph_breaks gmean=0.00x mean=4.767x 2024-08-20T23:29:14.6962120Z autograd_captures gmean=0.00x mean=0.000x 2024-08-20T23:29:14.6965784Z autograd_compiles gmean=0.00x mean=0.000x 2024-08-20T23:29:14.6969763Z cudagraph_skips gmean=0.00x mean=0.133x 2024-08-20T23:29:17.1081638Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_timm_training.csv 2024-08-20T23:29:17.3710716Z adv_inception_v3 PASS 2024-08-20T23:29:17.3714722Z beit_base_patch16_224 PASS 2024-08-20T23:29:17.3719183Z botnet26t_256 PASS 2024-08-20T23:29:17.3723511Z cait_m36_384 XFAIL 2024-08-20T23:29:17.3728309Z coat_lite_mini PASS 2024-08-20T23:29:17.3732809Z convit_base PASS 2024-08-20T23:29:17.3737449Z convmixer_768_32 PASS 2024-08-20T23:29:17.3741738Z convnext_base PASS 2024-08-20T23:29:17.3746379Z crossvit_9_240 PASS 2024-08-20T23:29:17.3750590Z cspdarknet53 PASS 2024-08-20T23:29:17.3755426Z deit_base_distilled_patch16_224 PASS 2024-08-20T23:29:17.3759698Z dla102 PASS 2024-08-20T23:29:17.3764123Z dm_nfnet_f0 PASS 2024-08-20T23:29:17.3768693Z dpn107 PASS 2024-08-20T23:29:17.3773323Z eca_botnext26ts_256 PASS 2024-08-20T23:29:17.3777814Z eca_halonext26ts PASS 2024-08-20T23:29:17.3782403Z ese_vovnet19b_dw PASS 2024-08-20T23:29:17.3787078Z fbnetc_100 PASS 2024-08-20T23:29:17.3791406Z fbnetv3_b PASS 2024-08-20T23:29:17.3795972Z gernet_l PASS 2024-08-20T23:29:17.3800403Z ghostnet_100 PASS 2024-08-20T23:29:17.3805381Z gluon_inception_v3 PASS 2024-08-20T23:29:17.3810069Z gmixer_24_224 PASS 2024-08-20T23:29:17.3814307Z gmlp_s16_224 PASS 2024-08-20T23:29:17.3818822Z hrnet_w18 PASS 2024-08-20T23:29:17.3823310Z inception_v3 PASS 2024-08-20T23:29:17.3827932Z jx_nest_base PASS 2024-08-20T23:29:17.3832361Z lcnet_050 PASS 2024-08-20T23:29:17.3836811Z levit_128 PASS 2024-08-20T23:29:17.3841223Z mixer_b16_224 PASS 2024-08-20T23:29:17.4262295Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_timm_training.csv 2024-08-20T23:29:17.6895573Z adv_inception_v3 PASS 2024-08-20T23:29:17.6899156Z beit_base_patch16_224 PASS 2024-08-20T23:29:17.6905226Z botnet26t_256 PASS 2024-08-20T23:29:17.6910963Z cait_m36_384 PASS 2024-08-20T23:29:17.6916567Z coat_lite_mini PASS 2024-08-20T23:29:17.6922002Z convit_base PASS 2024-08-20T23:29:17.6928068Z convmixer_768_32 PASS 2024-08-20T23:29:17.6933320Z convnext_base PASS 2024-08-20T23:29:17.6938676Z crossvit_9_240 PASS 2024-08-20T23:29:17.6943324Z cspdarknet53 PASS 2024-08-20T23:29:17.6948513Z deit_base_distilled_patch16_224 PASS 2024-08-20T23:29:17.6953250Z dla102 PASS 2024-08-20T23:29:17.6958130Z dm_nfnet_f0 PASS 2024-08-20T23:29:17.6962880Z dpn107 PASS 2024-08-20T23:29:17.6967810Z eca_botnext26ts_256 PASS 2024-08-20T23:29:17.6973059Z eca_halonext26ts PASS 2024-08-20T23:29:17.6977652Z ese_vovnet19b_dw PASS 2024-08-20T23:29:17.6982799Z fbnetc_100 PASS 2024-08-20T23:29:17.6987286Z fbnetv3_b PASS 2024-08-20T23:29:17.6992181Z gernet_l PASS 2024-08-20T23:29:17.6997004Z ghostnet_100 PASS 2024-08-20T23:29:17.7001926Z gluon_inception_v3 PASS 2024-08-20T23:29:17.7007004Z gmixer_24_224 PASS 2024-08-20T23:29:17.7012436Z gmlp_s16_224 PASS 2024-08-20T23:29:17.7017220Z hrnet_w18 PASS 2024-08-20T23:29:17.7022076Z inception_v3 PASS 2024-08-20T23:29:17.7026849Z jx_nest_base PASS 2024-08-20T23:29:17.7031654Z lcnet_050 PASS 2024-08-20T23:29:17.7036548Z levit_128 PASS 2024-08-20T23:29:17.7041352Z mixer_b16_224 PASS 2024-08-20T23:29:17.7466486Z + cleanup_workspace 2024-08-20T23:29:17.7467475Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-20T23:29:17.7468709Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-20T23:29:17.7469643Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-20T23:29:17.7470282Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-20T23:29:17.7471082Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-20T23:29:17.7471980Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-20T23:29:17.7472916Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-20T23:29:18.4961857Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-20T23:29:18.4962376Z cat test/**/*_toprint.log || true 2024-08-20T23:29:18.4976319Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:18.4976818Z env: 2024-08-20T23:29:18.4977096Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:18.4977551Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:18.4978279Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:18.4978911Z ##[endgroup] 2024-08-20T23:29:18.5075738Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-20T23:29:18.5107253Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:29:18.5107721Z kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:29:18.5115894Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:18.5116394Z env: 2024-08-20T23:29:18.5116688Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:18.5117147Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:18.5117873Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:18.5118527Z MONITOR_SCRIPT_PID: 52429 2024-08-20T23:29:18.5118883Z ##[endgroup] 2024-08-20T23:29:18.5313540Z Prepare all required actions 2024-08-20T23:29:18.5313992Z Getting action download info 2024-08-20T23:29:18.7380946Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-20T23:29:18.8961743Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-20T23:29:18.8962211Z with: 2024-08-20T23:29:18.8962745Z file-suffix: test-inductor_timm-1-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426126 2024-08-20T23:29:18.8963404Z s3-bucket: gha-artifacts 2024-08-20T23:29:18.8963740Z env: 2024-08-20T23:29:18.8964017Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:18.8964469Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:18.8965214Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:18.8965842Z ##[endgroup] 2024-08-20T23:29:18.8993571Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-20T23:29:18.8994298Z # Remove any previous test jsons if they exist 2024-08-20T23:29:18.8994843Z rm -f test-jsons-*.zip 2024-08-20T23:29:18.8995356Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-20T23:29:18.9004454Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:18.9005277Z env: 2024-08-20T23:29:18.9005555Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:18.9005999Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:18.9006729Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:18.9007677Z FILE_SUFFIX: test-inductor_timm-1-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426126 2024-08-20T23:29:18.9008323Z ##[endgroup] 2024-08-20T23:29:18.9369212Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-20T23:29:18.9398833Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-20T23:29:18.9400205Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-20T23:29:18.9406325Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-20T23:29:18.9411631Z adding: test/slow_tests.json (deflated 82%) 2024-08-20T23:29:18.9444297Z ##[group]Run # Remove any previous test reports if they exist 2024-08-20T23:29:18.9444938Z # Remove any previous test reports if they exist 2024-08-20T23:29:18.9445463Z rm -f test-reports-*.zip 2024-08-20T23:29:18.9446052Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-20T23:29:18.9455107Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:18.9455595Z env: 2024-08-20T23:29:18.9455874Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:18.9456526Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:18.9457281Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:18.9458169Z FILE_SUFFIX: test-inductor_timm-1-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426126 2024-08-20T23:29:18.9458817Z ##[endgroup] 2024-08-20T23:29:18.9715436Z adding: test/test-reports/inference_timm_models.csv (deflated 68%) 2024-08-20T23:29:18.9716245Z adding: test/test-reports/training_timm_models.csv (deflated 65%) 2024-08-20T23:29:18.9720141Z adding: test/test-reports/training_timm_model_graph_breaks.csv (deflated 97%) 2024-08-20T23:29:18.9721085Z adding: test/test-reports/training_timm_model_graph_break_deduped.csv (deflated 73%) 2024-08-20T23:29:18.9752678Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-20T23:29:18.9753300Z # Remove any previous usage logs if they exist 2024-08-20T23:29:18.9753792Z rm -f logs-*.zip 2024-08-20T23:29:18.9754465Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-20T23:29:18.9755245Z # so check to see if the file exists first 2024-08-20T23:29:18.9755738Z if [ -f 'usage_log.txt' ]; then 2024-08-20T23:29:18.9756453Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-20T23:29:18.9756942Z fi 2024-08-20T23:29:18.9757308Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-20T23:29:18.9757875Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-20T23:29:18.9758366Z fi 2024-08-20T23:29:18.9766943Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:18.9767567Z env: 2024-08-20T23:29:18.9767839Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:18.9768291Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:18.9769020Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:18.9769921Z FILE_SUFFIX: test-inductor_timm-1-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426126 2024-08-20T23:29:18.9770557Z ##[endgroup] 2024-08-20T23:29:18.9856887Z adding: usage_log.txt (deflated 92%) 2024-08-20T23:29:18.9946778Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-20T23:29:18.9947505Z # Remove any previous debugging artifacts if they exist 2024-08-20T23:29:18.9948054Z rm -f debug-*.zip 2024-08-20T23:29:18.9948440Z if [ -d 'test/debug' ]; then 2024-08-20T23:29:18.9948936Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-20T23:29:18.9949409Z fi 2024-08-20T23:29:18.9958572Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:18.9959077Z env: 2024-08-20T23:29:18.9959358Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:18.9959820Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:18.9960556Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:18.9961462Z FILE_SUFFIX: test-inductor_timm-1-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426126 2024-08-20T23:29:18.9962101Z ##[endgroup] 2024-08-20T23:29:19.0055950Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:29:19.0056378Z with: 2024-08-20T23:29:19.0056658Z s3-bucket: gha-artifacts 2024-08-20T23:29:19.0057150Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:29:19.0057621Z retention-days: 14 2024-08-20T23:29:19.0057950Z if-no-files-found: warn 2024-08-20T23:29:19.0058303Z path: test-jsons-*.zip 2024-08-20T23:29:19.0058641Z name: artifact 2024-08-20T23:29:19.0058936Z region: us-east-1 2024-08-20T23:29:19.0059233Z env: 2024-08-20T23:29:19.0059507Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:19.0059952Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:19.0060671Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:19.0061302Z ##[endgroup] 2024-08-20T23:29:19.3613277Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:29:19.3614068Z With the provided path, there will be 1 file uploaded 2024-08-20T23:29:19.3614830Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:29:19.4933135Z Starting upload of test-jsons-test-inductor_timm-1-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426126.zip 2024-08-20T23:29:19.6268094Z Finished upload of test-jsons-test-inductor_timm-1-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426126.zip 2024-08-20T23:29:19.6552435Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:29:19.6552873Z with: 2024-08-20T23:29:19.6553160Z s3-bucket: gha-artifacts 2024-08-20T23:29:19.6553598Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:29:19.6554073Z retention-days: 14 2024-08-20T23:29:19.6554405Z if-no-files-found: error 2024-08-20T23:29:19.6554773Z path: test-reports-*.zip 2024-08-20T23:29:19.6555123Z name: artifact 2024-08-20T23:29:19.6555422Z region: us-east-1 2024-08-20T23:29:19.6555741Z env: 2024-08-20T23:29:19.6556017Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:19.6556464Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:19.6557193Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:19.6557830Z ##[endgroup] 2024-08-20T23:29:20.0081575Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:29:20.0082181Z With the provided path, there will be 1 file uploaded 2024-08-20T23:29:20.0082807Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:29:20.0138769Z Starting upload of test-reports-test-inductor_timm-1-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426126.zip 2024-08-20T23:29:20.0986879Z Finished upload of test-reports-test-inductor_timm-1-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426126.zip 2024-08-20T23:29:20.1271926Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:29:20.1272366Z with: 2024-08-20T23:29:20.1272653Z s3-bucket: gha-artifacts 2024-08-20T23:29:20.1273109Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:29:20.1273585Z retention-days: 14 2024-08-20T23:29:20.1273918Z if-no-files-found: ignore 2024-08-20T23:29:20.1274280Z path: logs-*.zip 2024-08-20T23:29:20.1274592Z name: artifact 2024-08-20T23:29:20.1274891Z region: us-east-1 2024-08-20T23:29:20.1275210Z env: 2024-08-20T23:29:20.1275487Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:20.1275941Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:20.1276665Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:20.1277295Z ##[endgroup] 2024-08-20T23:29:20.4890182Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:29:20.4891702Z With the provided path, there will be 1 file uploaded 2024-08-20T23:29:20.4893059Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:29:20.4946232Z Starting upload of logs-test-inductor_timm-1-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426126.zip 2024-08-20T23:29:20.5955935Z Finished upload of logs-test-inductor_timm-1-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026426126.zip 2024-08-20T23:29:20.6238566Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:29:20.6239003Z with: 2024-08-20T23:29:20.6239292Z s3-bucket: gha-artifacts 2024-08-20T23:29:20.6239750Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:29:20.6240222Z retention-days: 14 2024-08-20T23:29:20.6240557Z if-no-files-found: ignore 2024-08-20T23:29:20.6240921Z path: debug-*.zip 2024-08-20T23:29:20.6241240Z name: artifact 2024-08-20T23:29:20.6241544Z region: us-east-1 2024-08-20T23:29:20.6241845Z env: 2024-08-20T23:29:20.6242125Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:20.6242577Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:20.6243305Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:20.6243939Z ##[endgroup] 2024-08-20T23:29:20.9776080Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-20T23:29:21.0056620Z ##[group]Run # shellcheck disable=SC2156 2024-08-20T23:29:21.0057104Z # shellcheck disable=SC2156 2024-08-20T23:29:21.0057889Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-20T23:29:21.0067856Z shell: /usr/bin/bash -e {0} 2024-08-20T23:29:21.0068262Z env: 2024-08-20T23:29:21.0068544Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:21.0069003Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:21.0069731Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:21.0070365Z ##[endgroup] 2024-08-20T23:29:21.2697609Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-20T23:29:21.2698208Z with: 2024-08-20T23:29:21.2698469Z env: 2024-08-20T23:29:21.2698742Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:21.2699181Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:21.2699920Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:21.2700550Z ##[endgroup] 2024-08-20T23:29:21.2720127Z ##[group]Run set -eou pipefail 2024-08-20T23:29:21.2720534Z set -eou pipefail 2024-08-20T23:29:21.2720891Z  2024-08-20T23:29:21.2721403Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-20T23:29:21.2722029Z for _ in $(seq 1440); do 2024-08-20T23:29:21.2722505Z  # Break if no ssh session exists anymore 2024-08-20T23:29:21.2723000Z  if [ "$(who)" = "" ]; then 2024-08-20T23:29:21.2723407Z  break 2024-08-20T23:29:21.2723751Z  fi 2024-08-20T23:29:21.2724050Z  echo "." 2024-08-20T23:29:21.2724383Z  sleep 5 2024-08-20T23:29:21.2724704Z done 2024-08-20T23:29:21.2733587Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:21.2734082Z env: 2024-08-20T23:29:21.2734363Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:21.2734818Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:21.2735544Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:21.2736171Z ##[endgroup] 2024-08-20T23:29:21.2765403Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-20T23:29:21.2849340Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:29:21.2850104Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:29:21.2850690Z # shellcheck disable=SC2046 2024-08-20T23:29:21.2851169Z docker stop $(docker ps -q) || true 2024-08-20T23:29:21.2851645Z # Prune all of the docker images 2024-08-20T23:29:21.2852102Z docker system prune -af 2024-08-20T23:29:21.2860988Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:21.2861508Z env: 2024-08-20T23:29:21.2861799Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:21.2862261Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:29:21.2862998Z DOCKER_CONTAINER_ID: b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:21.2863640Z ##[endgroup] 2024-08-20T23:29:21.8136101Z b4c2664d5dd5 2024-08-20T23:29:23.2679059Z Deleted Containers: 2024-08-20T23:29:23.2679766Z b4c2664d5dd56fdb92869427f0d87101c428d155a22edffde72906ba26ce7c62 2024-08-20T23:29:23.2680208Z 2024-08-20T23:29:34.4185183Z Deleted Images: 2024-08-20T23:29:34.4187115Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T23:29:34.4189995Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks@sha256:4753b1ce6a27c0af5c669dd74e9e16618d0a1a015eb474ef14f6fd51409fc5f7 2024-08-20T23:29:34.4192220Z deleted: sha256:cee8d41a96d3e7eccb178c1529633c2dd768035bec694588cf53029b783ac292 2024-08-20T23:29:34.4193063Z deleted: sha256:387912ed740fe741bc7f78f2754e4da03169e3d9785348eeef65657209e6f778 2024-08-20T23:29:34.4193942Z deleted: sha256:fdabcba8f8fb280e19a152aa25e47e9651627e1f9f1bc23153068fdf8c7d630a 2024-08-20T23:29:34.4194802Z deleted: sha256:5c719bfedda0fa42f3cc857a021fb6d9166275ef47e6b7854bdf79cbd94371a7 2024-08-20T23:29:34.4195646Z deleted: sha256:1a7fa8fef08db4647f773032825124b3c7765761cdb2514100129cd118b53f37 2024-08-20T23:29:34.4196466Z deleted: sha256:f2e444a1c58fb82e012d6be36af6acb9d1f137330877be768e65749c12c6eb7a 2024-08-20T23:29:34.4197347Z deleted: sha256:33704a494ac956cb2ec1430c9126595eaac4741e5d6ae47ee20d64306ac05764 2024-08-20T23:29:34.4198418Z deleted: sha256:6a4ca9ff7195505d66fd28b827e4d52c95309ca34d455fe52bec2a3dfb6bf973 2024-08-20T23:29:34.4199265Z deleted: sha256:32a556656b7c0c0fedabcb4be518df66ae4b707ac63b52e7e5c256a151fd586f 2024-08-20T23:29:34.4200100Z deleted: sha256:21ffd21aadcd3065d2bd133ce57077e37347bb948615c4c71aa475d0fc9ed89b 2024-08-20T23:29:34.4200954Z deleted: sha256:d653a3b9ba44f6be8e819fcbd32a14c4a36ed536a5ef86f8bdc794ea46cef755 2024-08-20T23:29:34.4201813Z deleted: sha256:d345bcdfdea1caa63503188719bb1c5fa135b18bb0a8e816aefe3d2ded5c996a 2024-08-20T23:29:34.4202656Z deleted: sha256:a251f1284cbe4dfb7c5afb0f89272046796e77fb91a71471d89acca62f25a6db 2024-08-20T23:29:34.4203485Z deleted: sha256:f0189edd8227eb15aea7e1bfe18525327b825ee314882ebf3357b5093835dac3 2024-08-20T23:29:34.4204335Z deleted: sha256:5eacbc0fafbd39bf65cadd1e29a52ee9fd4222765cb56fcb878bda0344598406 2024-08-20T23:29:34.4205908Z deleted: sha256:7eddd13a99b3708e985984c95fc0c52cba171ff24a9c0fa45d57cb665d4e221f 2024-08-20T23:29:34.4206841Z deleted: sha256:bc8cac8e2879f26cb90afbbeb7e64ba11581b57648745399021fa45df7c2f7ae 2024-08-20T23:29:34.4207825Z deleted: sha256:57532a5c1bacb2650908c672b75fae5c8781cb7ee701ac1784967e50bfceaae5 2024-08-20T23:29:34.4208636Z deleted: sha256:21509edb6a37130a50af8008f246f00373a5bc735d4d1afa0a3fdbda622fbbbd 2024-08-20T23:29:34.4209470Z deleted: sha256:4db1e29af6ad7a5f28276ef7bdb6ac5a60244101f36d955aa34fb217ea4e45d4 2024-08-20T23:29:34.4210299Z deleted: sha256:5b97f357f15208f34242b48ed3cf670775a8fb58e04eb24ef8c8d9552848a918 2024-08-20T23:29:34.4211109Z deleted: sha256:968988c362f1c9b3b08fec85cf7062d8b29858681660ddfb3afd56840c40e25e 2024-08-20T23:29:34.4211934Z deleted: sha256:67b5029b1c3c8aa88fd34383f8ffc6e5eab1aa392fd01030d946abc467a47b59 2024-08-20T23:29:34.4212772Z deleted: sha256:a30fda61fc6efa33e06722afb4f61b07780cb21ca0668d135bbe551841828c65 2024-08-20T23:29:34.4213616Z deleted: sha256:bafaf280fd9d77b6d6cf03b502768cb7c77f215dad925746be1c31b7a58e48ef 2024-08-20T23:29:34.4214464Z deleted: sha256:2cec74a4397496f01bfa49616ffa41f1cfb9bfd83f333f1eabb5f35e82480699 2024-08-20T23:29:34.4215275Z deleted: sha256:44e532372f95c9294bef0fae8cb8a240437327fe41a7868842b9068423f1e9f2 2024-08-20T23:29:34.4216080Z deleted: sha256:9d3868b219eb14fb39fe0b22202580a15871a0ba15447569a89c096eb9a2cd06 2024-08-20T23:29:34.4216892Z deleted: sha256:033b1e94d1741cdceb2e0052d884cf58a98e7634482a5490385768dd53708747 2024-08-20T23:29:34.4217700Z deleted: sha256:11f9b7ea949d1065d500b6c408f8313c13734d192dbb2f6230e9b63fabe94ebf 2024-08-20T23:29:34.4218512Z deleted: sha256:10f56d39f97b51ff7fe2e2df9b80e1cfde2422660ac722c1d541139dbe1a704c 2024-08-20T23:29:34.4219334Z deleted: sha256:03a7c0c48c9a83e65ba5156403bc1fe60356ab4b1fc7e075f9141603833246f3 2024-08-20T23:29:34.4220156Z deleted: sha256:7fe9c7528f9ef7341782ed285475cb83eccc61ce0ab97d1e7cbae56ed0e01d42 2024-08-20T23:29:34.4220982Z deleted: sha256:0efbe5060cc721d5a732d3e34de1b9dd40573d304803cbd0b2437ca8872ce549 2024-08-20T23:29:34.4221802Z deleted: sha256:e6adbe3e7465b1b08f494e3561c42837c0116bc71ab409a7c62c56328fb9e1ef 2024-08-20T23:29:34.4222623Z deleted: sha256:2058c6a1cea3f868fabc404608ed1d273db9f63b5e1d3211d765e41ae51f8d61 2024-08-20T23:29:34.4223445Z deleted: sha256:33d25e35e9a4f1bfa5d8373c18e9f3449358445c62dc2708bc5036ed14c7e9bd 2024-08-20T23:29:34.4224277Z deleted: sha256:5f854f6a610a0bf5198cefce78fcb96dadf7ce7fdb02bfe6685b326f6cd3406b 2024-08-20T23:29:34.4225258Z deleted: sha256:98634d529353f4e760c15592834942206298353e233a2937a836a73c6b5e4168 2024-08-20T23:29:34.4226030Z deleted: sha256:071ec252097583d609a084e0f43a70395c334fe8d9b3e56a8d97135c49a3c3a5 2024-08-20T23:29:34.4226839Z deleted: sha256:5f818bbf177d0e18a7c5e5ce3470c6f6a2cc11017bed953cd5b78b2b26a35558 2024-08-20T23:29:34.4227718Z deleted: sha256:d2cce52317ffa022e42fdd14dc33292b40b1a273cc139c96d3a206a2173725a4 2024-08-20T23:29:34.4228556Z deleted: sha256:d5ce7ab9e4b134eb4efd1e5712bd0c87df578bdb5be9a1cbf5dcdb583044e9f8 2024-08-20T23:29:34.4229379Z deleted: sha256:7bf36330750a896b5b5581463f0e9cfdcd3920e3c5c20d728ee553c7f8cf6eeb 2024-08-20T23:29:34.4230590Z deleted: sha256:7f743fed415e07aca752031384d8262c5ea4c80e239710986b93e7118f67f1c2 2024-08-20T23:29:34.4231427Z deleted: sha256:aa2f660d8384dd4660a1080d54c22db28b4c8502c92255e97bac64b8172153b7 2024-08-20T23:29:34.4232244Z deleted: sha256:f872fb5e9a43d29fb03685b648ea38679b88586c783e5ec6815bb69895c0103c 2024-08-20T23:29:34.4233072Z deleted: sha256:2b39b70836b2330fe60c8a8e809f97b5deff7e1b3868fa4a7bf0e00e5f6540c4 2024-08-20T23:29:34.4233919Z deleted: sha256:fffd71ce3df39233b4a9d1e1c6a3e78fe0d22539f03b6f5e46ef323be746bb1b 2024-08-20T23:29:34.4234758Z deleted: sha256:8fa7767d8015b2f308374dc2573904630aff8009ae15ac5cd5b99986a9e5d38c 2024-08-20T23:29:34.4235576Z deleted: sha256:6d39ae4f4f81796c79703a115341abc214ea6402b95d7c4481b0860cca5411db 2024-08-20T23:29:34.4236400Z deleted: sha256:d635bd598f9c6aa8a28f9a417861a82ceddd0c28fb9e006ac1bdeffb6da92dc8 2024-08-20T23:29:34.4237292Z deleted: sha256:08458dac742f18d448f988793bd19c27c6129457b8b2ff2743ccdf2b705b86e3 2024-08-20T23:29:34.4238116Z deleted: sha256:7715bcb4ef88525b2b34380f2b81ceae25e87fb53a8c97e99dd319765db02f6f 2024-08-20T23:29:34.4238956Z deleted: sha256:ca9a40d26179de2d38203500cbd75fa2dbcb7da4b0badb3f02861714e1ee0e20 2024-08-20T23:29:34.4239804Z deleted: sha256:5076e56d3cbdded7799738aff3b50afcb3cb8f05b3aa4a9aa4a6e6fe6bb49465 2024-08-20T23:29:34.4240629Z deleted: sha256:c01100978013295b52ecbb0a196c82106506b48935a9c0a1d1dd66432d9b64b7 2024-08-20T23:29:34.4241435Z deleted: sha256:8747f391b7489623fc54b3813b0f79e3474c779d402b53e0c557e9b5b5f8e492 2024-08-20T23:29:34.4242250Z deleted: sha256:987974f3c4e180d9f3086ad4e1df30220d5ea68bafe243c0966d80cb45a0176d 2024-08-20T23:29:34.4243087Z deleted: sha256:1d2d9d89274b1d01c7b76fbc66fccafea8414d85482cb5c43d4b6631a09c1f4e 2024-08-20T23:29:34.4243914Z deleted: sha256:b5e92a2b04c427229eb25b5013dc2062bb64d7f751d3ea26292a7465f351a56b 2024-08-20T23:29:34.4244745Z deleted: sha256:6cf2fd73cc6550401f1b569cadf927d047ae3ff9624ed49a52c37e2f7e664346 2024-08-20T23:29:34.4245583Z deleted: sha256:dc09bad6e139f1dbf77f033a8bb913b50348ff30cca0459a1b65291309712b5a 2024-08-20T23:29:34.4246415Z deleted: sha256:69f8b41d1f6e2a349f9c4292928d43aa3c8750feff87e713ef27c9473db7e4bd 2024-08-20T23:29:34.4247278Z deleted: sha256:a88196e571b2b4c038c05542017cff4cf98568990a3483e4454739f789202a28 2024-08-20T23:29:34.4248099Z deleted: sha256:29f9f89c9d02abbd1cc7ea3a2e67211ffb77fa4fd4a6013dc8ac1328ed390133 2024-08-20T23:29:34.4248943Z deleted: sha256:fd5d6a109c3a5a89ed53f8b23d96494982c39df97ac3220edc4c4e4ba43c9af1 2024-08-20T23:29:34.4249775Z deleted: sha256:14436cfe80ca6ea0d09ba450a071ce28d8bc87fc3602d4c9f8b7cc3b474cbcc5 2024-08-20T23:29:34.4250599Z deleted: sha256:433544e4353c9e0269dc71fa16c00ddcecdcd5676ed0df618c7b51ac78e23053 2024-08-20T23:29:34.4251435Z deleted: sha256:fc24af9fd5121adfbc7b75df0abb1ef8c2d3868f33b7b6b1e8bf4e214fac4bd6 2024-08-20T23:29:34.4252269Z deleted: sha256:ce1492ad94b71204f8b3fb8903d0ad48a96652635e7fc9347e430639ca15a737 2024-08-20T23:29:34.4253071Z deleted: sha256:96f2ae9435e6e12f68a88dc404c8079b2796639589970c34894a86ccff3f3732 2024-08-20T23:29:34.4253885Z deleted: sha256:5310d2bc0952e7c6d8cbade7ecafd88ce3a04d9da303fa6419583e13a02b58a9 2024-08-20T23:29:34.4254706Z deleted: sha256:099076d608a55a027d60f5b572a3991a8d690a01500acf2c696933a7e6d38650 2024-08-20T23:29:34.4255519Z deleted: sha256:862b05d0f8f7e3dff2d04bd49784a9901f4e22c5dd6544ae72eb3be05e23fffe 2024-08-20T23:29:34.4256446Z deleted: sha256:f7a888e5f7904827f6d71cebcba5169dbc7a78b20f228b9b9b625f38e0c52f24 2024-08-20T23:29:34.4257285Z deleted: sha256:ab9f75a4a7a636aae480f630eaddc05133dda64f1c42445b617c97d3d209991c 2024-08-20T23:29:34.4258129Z deleted: sha256:56dda0c9cee64db5c10d5dd5f08c6f6707263f5ee98e8a075d635e5be996e855 2024-08-20T23:29:34.4258970Z deleted: sha256:5faf9c0a9efe4675ecd21a4ec417d51077d5e75da9e673161a94e7d6cd43f92c 2024-08-20T23:29:34.4259465Z 2024-08-20T23:29:34.4259623Z Total reclaimed space: 45.17GB 2024-08-20T23:29:34.4325921Z Post job cleanup. 2024-08-20T23:29:34.4376000Z Post job cleanup. 2024-08-20T23:29:34.5387065Z [command]/usr/bin/git version 2024-08-20T23:29:34.5439756Z git version 2.40.1 2024-08-20T23:29:34.5479128Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/c44d49b6-a1fd-41a6-94c0-2c100072f986' before making global git config changes 2024-08-20T23:29:34.5480503Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T23:29:34.5484324Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T23:29:34.5532491Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T23:29:34.5576645Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T23:29:34.5951465Z Entering 'android/libs/fbjni' 2024-08-20T23:29:34.6020465Z Entering 'third_party/FP16' 2024-08-20T23:29:34.6091067Z Entering 'third_party/FXdiv' 2024-08-20T23:29:34.6161449Z Entering 'third_party/NNPACK' 2024-08-20T23:29:34.6232085Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:29:34.6302072Z Entering 'third_party/XNNPACK' 2024-08-20T23:29:34.6388154Z Entering 'third_party/benchmark' 2024-08-20T23:29:34.6458844Z Entering 'third_party/cpp-httplib' 2024-08-20T23:29:34.6528637Z Entering 'third_party/cpuinfo' 2024-08-20T23:29:34.6599587Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:29:34.6674340Z Entering 'third_party/cutlass' 2024-08-20T23:29:34.6751328Z Entering 'third_party/eigen' 2024-08-20T23:29:34.6824627Z Entering 'third_party/fbgemm' 2024-08-20T23:29:34.6892072Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:29:34.6959113Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:29:34.7028899Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:29:34.7101403Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:29:34.7169473Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:29:34.7237867Z Entering 'third_party/flatbuffers' 2024-08-20T23:29:34.7310880Z Entering 'third_party/fmt' 2024-08-20T23:29:34.7380139Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:29:34.7452318Z Entering 'third_party/gloo' 2024-08-20T23:29:34.7521388Z Entering 'third_party/googletest' 2024-08-20T23:29:34.7589989Z Entering 'third_party/ideep' 2024-08-20T23:29:34.7658029Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:29:34.7734622Z Entering 'third_party/ittapi' 2024-08-20T23:29:34.7803912Z Entering 'third_party/kineto' 2024-08-20T23:29:34.7873943Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:29:34.7939760Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:29:34.8010034Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:29:34.8077271Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:29:34.8145258Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:29:34.8211229Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:29:34.8283025Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:29:34.8352678Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:29:34.8420452Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:29:34.8489299Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:29:34.8561378Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:29:34.8630145Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:29:34.8699006Z Entering 'third_party/mimalloc' 2024-08-20T23:29:34.8771251Z Entering 'third_party/nccl/nccl' 2024-08-20T23:29:34.8841555Z Entering 'third_party/nlohmann' 2024-08-20T23:29:34.8912048Z Entering 'third_party/onnx' 2024-08-20T23:29:34.8999060Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:29:34.9065742Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:29:34.9138279Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:29:34.9208530Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:29:34.9274278Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:29:34.9340920Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:29:34.9406556Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:29:34.9479135Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:29:34.9549849Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:29:34.9616774Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:29:34.9680287Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:29:34.9750130Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:29:34.9818969Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:29:34.9910226Z Entering 'third_party/pocketfft' 2024-08-20T23:29:34.9983902Z Entering 'third_party/protobuf' 2024-08-20T23:29:35.0056231Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:29:35.0126288Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:29:35.0195035Z Entering 'third_party/psimd' 2024-08-20T23:29:35.0264854Z Entering 'third_party/pthreadpool' 2024-08-20T23:29:35.0334555Z Entering 'third_party/pybind11' 2024-08-20T23:29:35.0404499Z Entering 'third_party/python-peachpy' 2024-08-20T23:29:35.0474500Z Entering 'third_party/sleef' 2024-08-20T23:29:35.0543756Z Entering 'third_party/tensorpipe' 2024-08-20T23:29:35.0612009Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:29:35.0677653Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:29:35.0743150Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:29:35.0809598Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:29:35.0872885Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:29:35.0968740Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T23:29:35.1005096Z http.https://github.com/.extraheader 2024-08-20T23:29:35.1015183Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-20T23:29:35.1061209Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T23:29:35.1431382Z Entering 'android/libs/fbjni' 2024-08-20T23:29:35.1473011Z http.https://github.com/.extraheader 2024-08-20T23:29:35.1518865Z Entering 'third_party/FP16' 2024-08-20T23:29:35.1562013Z http.https://github.com/.extraheader 2024-08-20T23:29:35.1605694Z Entering 'third_party/FXdiv' 2024-08-20T23:29:35.1648377Z http.https://github.com/.extraheader 2024-08-20T23:29:35.1690941Z Entering 'third_party/NNPACK' 2024-08-20T23:29:35.1734754Z http.https://github.com/.extraheader 2024-08-20T23:29:35.1777828Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:29:35.1821622Z http.https://github.com/.extraheader 2024-08-20T23:29:35.1864593Z Entering 'third_party/XNNPACK' 2024-08-20T23:29:35.1908416Z http.https://github.com/.extraheader 2024-08-20T23:29:35.1968104Z Entering 'third_party/benchmark' 2024-08-20T23:29:35.2011560Z http.https://github.com/.extraheader 2024-08-20T23:29:35.2054917Z Entering 'third_party/cpp-httplib' 2024-08-20T23:29:35.2098170Z http.https://github.com/.extraheader 2024-08-20T23:29:35.2142518Z Entering 'third_party/cpuinfo' 2024-08-20T23:29:35.2186520Z http.https://github.com/.extraheader 2024-08-20T23:29:35.2233193Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:29:35.2276596Z http.https://github.com/.extraheader 2024-08-20T23:29:35.2322835Z Entering 'third_party/cutlass' 2024-08-20T23:29:35.2366587Z http.https://github.com/.extraheader 2024-08-20T23:29:35.2417776Z Entering 'third_party/eigen' 2024-08-20T23:29:35.2460944Z http.https://github.com/.extraheader 2024-08-20T23:29:35.2506378Z Entering 'third_party/fbgemm' 2024-08-20T23:29:35.2551026Z http.https://github.com/.extraheader 2024-08-20T23:29:35.2594472Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:29:35.2637496Z http.https://github.com/.extraheader 2024-08-20T23:29:35.2682415Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:29:35.2725361Z http.https://github.com/.extraheader 2024-08-20T23:29:35.2767958Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:29:35.2818619Z http.https://github.com/.extraheader 2024-08-20T23:29:35.2868548Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:29:35.2911097Z http.https://github.com/.extraheader 2024-08-20T23:29:35.2953467Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:29:35.2995792Z http.https://github.com/.extraheader 2024-08-20T23:29:35.3046266Z Entering 'third_party/flatbuffers' 2024-08-20T23:29:35.3090220Z http.https://github.com/.extraheader 2024-08-20T23:29:35.3137635Z Entering 'third_party/fmt' 2024-08-20T23:29:35.3181394Z http.https://github.com/.extraheader 2024-08-20T23:29:35.3227574Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:29:35.3271530Z http.https://github.com/.extraheader 2024-08-20T23:29:35.3317633Z Entering 'third_party/gloo' 2024-08-20T23:29:35.3362698Z http.https://github.com/.extraheader 2024-08-20T23:29:35.3407138Z Entering 'third_party/googletest' 2024-08-20T23:29:35.3450912Z http.https://github.com/.extraheader 2024-08-20T23:29:35.3494400Z Entering 'third_party/ideep' 2024-08-20T23:29:35.3538662Z http.https://github.com/.extraheader 2024-08-20T23:29:35.3580469Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:29:35.3627489Z http.https://github.com/.extraheader 2024-08-20T23:29:35.3678695Z Entering 'third_party/ittapi' 2024-08-20T23:29:35.3727914Z http.https://github.com/.extraheader 2024-08-20T23:29:35.3771350Z Entering 'third_party/kineto' 2024-08-20T23:29:35.3816256Z http.https://github.com/.extraheader 2024-08-20T23:29:35.3858823Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:29:35.3906441Z http.https://github.com/.extraheader 2024-08-20T23:29:35.3950836Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:29:35.3992354Z http.https://github.com/.extraheader 2024-08-20T23:29:35.4039542Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:29:35.4082884Z http.https://github.com/.extraheader 2024-08-20T23:29:35.4128682Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:29:35.4171293Z http.https://github.com/.extraheader 2024-08-20T23:29:35.4216454Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:29:35.4260761Z http.https://github.com/.extraheader 2024-08-20T23:29:35.4303956Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:29:35.4348904Z http.https://github.com/.extraheader 2024-08-20T23:29:35.4398009Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:29:35.4441321Z http.https://github.com/.extraheader 2024-08-20T23:29:35.4487632Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:29:35.4531383Z http.https://github.com/.extraheader 2024-08-20T23:29:35.4576426Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:29:35.4620709Z http.https://github.com/.extraheader 2024-08-20T23:29:35.4666616Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:29:35.4711153Z http.https://github.com/.extraheader 2024-08-20T23:29:35.4758041Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:29:35.4805106Z http.https://github.com/.extraheader 2024-08-20T23:29:35.4848432Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:29:35.4891202Z http.https://github.com/.extraheader 2024-08-20T23:29:35.4939647Z Entering 'third_party/mimalloc' 2024-08-20T23:29:35.4984845Z http.https://github.com/.extraheader 2024-08-20T23:29:35.5030370Z Entering 'third_party/nccl/nccl' 2024-08-20T23:29:35.5075008Z http.https://github.com/.extraheader 2024-08-20T23:29:35.5121395Z Entering 'third_party/nlohmann' 2024-08-20T23:29:35.5165606Z http.https://github.com/.extraheader 2024-08-20T23:29:35.5214454Z Entering 'third_party/onnx' 2024-08-20T23:29:35.5258920Z http.https://github.com/.extraheader 2024-08-20T23:29:35.5316748Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:29:35.5359803Z http.https://github.com/.extraheader 2024-08-20T23:29:35.5404041Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:29:35.5447344Z http.https://github.com/.extraheader 2024-08-20T23:29:35.5500998Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:29:35.5546024Z http.https://github.com/.extraheader 2024-08-20T23:29:35.5589791Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:29:35.5632885Z http.https://github.com/.extraheader 2024-08-20T23:29:35.5675598Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:29:35.5718227Z http.https://github.com/.extraheader 2024-08-20T23:29:35.5760395Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:29:35.5803298Z http.https://github.com/.extraheader 2024-08-20T23:29:35.5847000Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:29:35.5893906Z http.https://github.com/.extraheader 2024-08-20T23:29:35.5940123Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:29:35.5982583Z http.https://github.com/.extraheader 2024-08-20T23:29:35.6030679Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:29:35.6073758Z http.https://github.com/.extraheader 2024-08-20T23:29:35.6116810Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:29:35.6159097Z http.https://github.com/.extraheader 2024-08-20T23:29:35.6203110Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:29:35.6245438Z http.https://github.com/.extraheader 2024-08-20T23:29:35.6290501Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:29:35.6332908Z http.https://github.com/.extraheader 2024-08-20T23:29:35.6379139Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:29:35.6421432Z http.https://github.com/.extraheader 2024-08-20T23:29:35.6487675Z Entering 'third_party/pocketfft' 2024-08-20T23:29:35.6533641Z http.https://github.com/.extraheader 2024-08-20T23:29:35.6578300Z Entering 'third_party/protobuf' 2024-08-20T23:29:35.6624527Z http.https://github.com/.extraheader 2024-08-20T23:29:35.6670974Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:29:35.6713969Z http.https://github.com/.extraheader 2024-08-20T23:29:35.6757033Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:29:35.6799662Z http.https://github.com/.extraheader 2024-08-20T23:29:35.6847205Z Entering 'third_party/psimd' 2024-08-20T23:29:35.6891220Z http.https://github.com/.extraheader 2024-08-20T23:29:35.6936841Z Entering 'third_party/pthreadpool' 2024-08-20T23:29:35.6980323Z http.https://github.com/.extraheader 2024-08-20T23:29:35.7024008Z Entering 'third_party/pybind11' 2024-08-20T23:29:35.7067744Z http.https://github.com/.extraheader 2024-08-20T23:29:35.7112250Z Entering 'third_party/python-peachpy' 2024-08-20T23:29:35.7156264Z http.https://github.com/.extraheader 2024-08-20T23:29:35.7201041Z Entering 'third_party/sleef' 2024-08-20T23:29:35.7245155Z http.https://github.com/.extraheader 2024-08-20T23:29:35.7289652Z Entering 'third_party/tensorpipe' 2024-08-20T23:29:35.7334035Z http.https://github.com/.extraheader 2024-08-20T23:29:35.7377414Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:29:35.7420342Z http.https://github.com/.extraheader 2024-08-20T23:29:35.7463252Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:29:35.7505815Z http.https://github.com/.extraheader 2024-08-20T23:29:35.7548330Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:29:35.7592401Z http.https://github.com/.extraheader 2024-08-20T23:29:35.7636326Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:29:35.7679411Z http.https://github.com/.extraheader 2024-08-20T23:29:35.7722864Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:29:35.7765235Z http.https://github.com/.extraheader 2024-08-20T23:29:35.7926659Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-20T23:29:35.7947649Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-20T23:29:35.7955704Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:35.7956226Z ##[endgroup] 2024-08-20T23:29:42.6376555Z Cleaning up orphan processes