2024-06-26T07:03:01.4915356Z Current runner version: '2.317.0' 2024-06-26T07:03:01.4921163Z Runner name: 'i-02ccdeb779ec2a6a3' 2024-06-26T07:03:01.4921926Z Runner group name: 'Default' 2024-06-26T07:03:01.4923024Z Machine name: 'ip-10-0-17-21' 2024-06-26T07:03:01.4927750Z ##[group]GITHUB_TOKEN Permissions 2024-06-26T07:03:01.4929595Z Actions: read 2024-06-26T07:03:01.4930072Z Attestations: read 2024-06-26T07:03:01.4930626Z Checks: read 2024-06-26T07:03:01.4931064Z Contents: read 2024-06-26T07:03:01.4931507Z Deployments: read 2024-06-26T07:03:01.4932005Z Discussions: read 2024-06-26T07:03:01.4932467Z Issues: read 2024-06-26T07:03:01.4932885Z Metadata: read 2024-06-26T07:03:01.4933384Z Packages: read 2024-06-26T07:03:01.4933828Z Pages: read 2024-06-26T07:03:01.4934296Z PullRequests: read 2024-06-26T07:03:01.4934778Z RepositoryProjects: read 2024-06-26T07:03:01.4935326Z SecurityEvents: read 2024-06-26T07:03:01.4935867Z Statuses: read 2024-06-26T07:03:01.4936290Z ##[endgroup] 2024-06-26T07:03:01.4939320Z Secret source: Actions 2024-06-26T07:03:01.4940061Z Prepare workflow directory 2024-06-26T07:03:01.8632666Z Prepare all required actions 2024-06-26T07:03:01.8818780Z Getting action download info 2024-06-26T07:03:02.0746377Z Download action repository 'pytorch/test-infra@main' (SHA:43a2ce341cc31288e9a38b65ce600a7f43021bd5) 2024-06-26T07:03:02.4036303Z Download action repository 'pytorch/pytorch@main' (SHA:c422a9549d2b320b47a9391f6cbcf592b2876c88) 2024-06-26T07:03:05.1319195Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-06-26T07:03:05.2579136Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-06-26T07:03:05.5689296Z Getting action download info 2024-06-26T07:03:05.6680252Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-06-26T07:03:05.8420022Z Getting action download info 2024-06-26T07:03:05.9313689Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-06-26T07:03:06.0789225Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/129470 (b8c4c54d347aa776934c60784e35936878ef18dc) 2024-06-26T07:03:06.0791315Z ##[group] Inputs 2024-06-26T07:03:06.0791787Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:03:06.0798667Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-06-26T07:03:06.0806450Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:03:06.0807604Z sync-tag: 2024-06-26T07:03:06.0808430Z timeout-minutes: 240 2024-06-26T07:03:06.0808761Z use-gha: 2024-06-26T07:03:06.0809048Z dashboard-tag: 2024-06-26T07:03:06.0809377Z s3-bucket: gha-artifacts 2024-06-26T07:03:06.0809726Z aws-role-to-assume: 2024-06-26T07:03:06.0810066Z ##[endgroup] 2024-06-26T07:03:06.0811060Z Complete job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:03:06.1362521Z A job started hook has been configured by the self-hosted runner administrator 2024-06-26T07:03:06.1498027Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T07:03:06.1508576Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:03:06.1509104Z ##[endgroup] 2024-06-26T07:03:06.9432340Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-06-26T07:03:06.9432884Z with: 2024-06-26T07:03:06.9433503Z github-secret: *** 2024-06-26T07:03:06.9434413Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-06-26T07:03:06.9435383Z activate-with-label: false 2024-06-26T07:03:06.9435740Z label: with-ssh 2024-06-26T07:03:06.9436081Z remove-existing-keys: true 2024-06-26T07:03:06.9436471Z fail-silently: true 2024-06-26T07:03:06.9436777Z env: 2024-06-26T07:03:06.9437045Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:03:06.9437383Z ##[endgroup] 2024-06-26T07:03:07.0295543Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-06-26T07:03:07.0296556Z ciflow reference detected, attempting to extract PR number 2024-06-26T07:03:07.2930863Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-06-26T07:03:07.3549332Z No SSH keys found for user pytorch-bot[bot] 2024-06-26T07:03:07.3550057Z Grabbing public ssh keys from https://github.com/leslie-fang-intel.keys 2024-06-26T07:03:07.4325717Z No SSH keys found for user leslie-fang-intel 2024-06-26T07:03:07.4430854Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-06-26T07:03:07.4431496Z with: 2024-06-26T07:03:07.4431825Z submodules: recursive 2024-06-26T07:03:07.4432191Z fetch-depth: 0 2024-06-26T07:03:07.4432546Z env: 2024-06-26T07:03:07.4432875Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:03:07.4433278Z ##[endgroup] 2024-06-26T07:03:07.4623723Z ##[group]Run retry () { 2024-06-26T07:03:07.4624250Z retry () { 2024-06-26T07:03:07.4624844Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-06-26T07:03:07.4625436Z } 2024-06-26T07:03:07.4625785Z echo "${GITHUB_WORKSPACE}" 2024-06-26T07:03:07.4626287Z if [ -z "${NO_SUDO}" ]; then 2024-06-26T07:03:07.4651986Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T07:03:07.4652476Z else 2024-06-26T07:03:07.4652840Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T07:03:07.4653272Z fi 2024-06-26T07:03:07.4653640Z mkdir "${GITHUB_WORKSPACE}" 2024-06-26T07:03:07.4661441Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:03:07.4661946Z env: 2024-06-26T07:03:07.4662225Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:03:07.4662563Z NO_SUDO: 2024-06-26T07:03:07.4662841Z ##[endgroup] 2024-06-26T07:03:07.4684882Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:03:09.6362790Z ##[group]Run malfet/checkout@silent-checkout 2024-06-26T07:03:09.6363263Z with: 2024-06-26T07:03:09.6363582Z ref: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:03:09.6364021Z fetch-depth: 0 2024-06-26T07:03:09.6364332Z submodules: recursive 2024-06-26T07:03:09.6364666Z quiet-checkout: true 2024-06-26T07:03:09.6365150Z repository: pytorch/pytorch 2024-06-26T07:03:09.6365627Z token: *** 2024-06-26T07:03:09.6365916Z ssh-strict: true 2024-06-26T07:03:09.6366239Z persist-credentials: true 2024-06-26T07:03:09.6366583Z clean: true 2024-06-26T07:03:09.6366915Z sparse-checkout-cone-mode: true 2024-06-26T07:03:09.6367298Z lfs: false 2024-06-26T07:03:09.6367588Z set-safe-directory: true 2024-06-26T07:03:09.6367924Z env: 2024-06-26T07:03:09.6368192Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:03:09.6368528Z ##[endgroup] 2024-06-26T07:03:09.7280773Z Syncing repository: pytorch/pytorch 2024-06-26T07:03:09.7282493Z ##[group]Getting Git version info 2024-06-26T07:03:09.7283399Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T07:03:09.7284252Z [command]/usr/bin/git version 2024-06-26T07:03:09.7284629Z git version 2.40.1 2024-06-26T07:03:09.7285935Z ##[endgroup] 2024-06-26T07:03:09.7297154Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/18d27c80-b973-4a75-8a7e-6a49272aadf0' before making global git config changes 2024-06-26T07:03:09.7298398Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T07:03:09.7301114Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:03:09.7330198Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T07:03:09.7333850Z ##[group]Initializing the repository 2024-06-26T07:03:09.7336685Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:03:09.7355489Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-06-26T07:03:09.7356716Z hint: is subject to change. To configure the initial branch name to use in all 2024-06-26T07:03:09.7357846Z hint: of your new repositories, which will suppress this warning, call: 2024-06-26T07:03:09.7358402Z hint: 2024-06-26T07:03:09.7358868Z hint: git config --global init.defaultBranch 2024-06-26T07:03:09.7359351Z hint: 2024-06-26T07:03:09.7359859Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-06-26T07:03:09.7360671Z hint: 'development'. The just-created branch can be renamed via this command: 2024-06-26T07:03:09.7361424Z hint: 2024-06-26T07:03:09.7361740Z hint: git branch -m 2024-06-26T07:03:09.7362852Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-06-26T07:03:09.7364438Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-06-26T07:03:09.7383403Z ##[endgroup] 2024-06-26T07:03:09.7384094Z ##[group]Disabling automatic garbage collection 2024-06-26T07:03:09.7386186Z [command]/usr/bin/git config --local gc.auto 0 2024-06-26T07:03:09.7412452Z ##[endgroup] 2024-06-26T07:03:09.7413006Z ##[group]Setting up auth 2024-06-26T07:03:09.7418784Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T07:03:09.7444904Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T07:03:09.7635755Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T07:03:09.7659201Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T07:03:09.7864838Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:03:09.7915117Z ##[endgroup] 2024-06-26T07:03:09.7915784Z ##[group]Fetching the repository 2024-06-26T07:03:09.7921365Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-06-26T07:03:12.0690658Z remote: Enumerating objects: 987612 2024-06-26T07:03:12.0692310Z remote: Enumerating objects: 990209, done. 2024-06-26T07:03:12.0694959Z remote: Counting objects: 0% (1/2597) 2024-06-26T07:03:12.0695775Z remote: Counting objects: 1% (26/2597) 2024-06-26T07:03:12.0696486Z remote: Counting objects: 2% (52/2597) 2024-06-26T07:03:12.0696994Z remote: Counting objects: 3% (78/2597) 2024-06-26T07:03:12.0697732Z remote: Counting objects: 4% (104/2597) 2024-06-26T07:03:12.0698408Z remote: Counting objects: 5% (130/2597) 2024-06-26T07:03:12.0699140Z remote: Counting objects: 6% (156/2597) 2024-06-26T07:03:12.0699652Z remote: Counting objects: 7% (182/2597) 2024-06-26T07:03:12.0700136Z remote: Counting objects: 8% (208/2597) 2024-06-26T07:03:12.0700635Z remote: Counting objects: 9% (234/2597) 2024-06-26T07:03:12.0701358Z remote: Counting objects: 10% (260/2597) 2024-06-26T07:03:12.0701945Z remote: Counting objects: 11% (286/2597) 2024-06-26T07:03:12.0702702Z remote: Counting objects: 12% (312/2597) 2024-06-26T07:03:12.0703319Z remote: Counting objects: 13% (338/2597) 2024-06-26T07:03:12.0703885Z remote: Counting objects: 14% (364/2597) 2024-06-26T07:03:12.0704492Z remote: Counting objects: 15% (390/2597) 2024-06-26T07:03:12.0704975Z remote: Counting objects: 16% (416/2597) 2024-06-26T07:03:12.0705687Z remote: Counting objects: 17% (442/2597) 2024-06-26T07:03:12.0706310Z remote: Counting objects: 18% (468/2597) 2024-06-26T07:03:12.0707353Z remote: Counting objects: 19% (494/2597) 2024-06-26T07:03:12.0707857Z remote: Counting objects: 20% (520/2597) 2024-06-26T07:03:12.0708561Z remote: Counting objects: 21% (546/2597) 2024-06-26T07:03:12.0709226Z remote: Counting objects: 22% (572/2597) 2024-06-26T07:03:12.0709969Z remote: Counting objects: 23% (598/2597) 2024-06-26T07:03:12.0710686Z remote: Counting objects: 24% (624/2597) 2024-06-26T07:03:12.0711200Z remote: Counting objects: 25% (650/2597) 2024-06-26T07:03:12.0711949Z remote: Counting objects: 26% (676/2597) 2024-06-26T07:03:12.0712496Z remote: Counting objects: 27% (702/2597) 2024-06-26T07:03:12.0713433Z remote: Counting objects: 28% (728/2597) 2024-06-26T07:03:12.0713984Z remote: Counting objects: 29% (754/2597) 2024-06-26T07:03:12.0714713Z remote: Counting objects: 30% (780/2597) 2024-06-26T07:03:12.0715614Z remote: Counting objects: 31% (806/2597) 2024-06-26T07:03:12.0716402Z remote: Counting objects: 32% (832/2597) 2024-06-26T07:03:12.0716951Z remote: Counting objects: 33% (858/2597) 2024-06-26T07:03:12.0717436Z remote: Counting objects: 34% (883/2597) 2024-06-26T07:03:12.0718314Z remote: Counting objects: 35% (909/2597) 2024-06-26T07:03:12.0719034Z remote: Counting objects: 36% (935/2597) 2024-06-26T07:03:12.0719797Z remote: Counting objects: 37% (961/2597) 2024-06-26T07:03:12.0720549Z remote: Counting objects: 38% (987/2597) 2024-06-26T07:03:12.0721473Z remote: Counting objects: 39% (1013/2597) 2024-06-26T07:03:12.0722468Z remote: Counting objects: 40% (1039/2597) 2024-06-26T07:03:12.0723002Z remote: Counting objects: 41% (1065/2597) 2024-06-26T07:03:12.0723517Z remote: Counting objects: 42% (1091/2597) 2024-06-26T07:03:12.0724251Z remote: Counting objects: 43% (1117/2597) 2024-06-26T07:03:12.0724885Z remote: Counting objects: 44% (1143/2597) 2024-06-26T07:03:12.0725658Z remote: Counting objects: 45% (1169/2597) 2024-06-26T07:03:12.0726196Z remote: Counting objects: 46% (1195/2597) 2024-06-26T07:03:12.0726699Z remote: Counting objects: 47% (1221/2597) 2024-06-26T07:03:12.0727208Z remote: Counting objects: 48% (1247/2597) 2024-06-26T07:03:12.0728195Z remote: Counting objects: 49% (1273/2597) 2024-06-26T07:03:12.0728999Z remote: Counting objects: 50% (1299/2597) 2024-06-26T07:03:12.0729801Z remote: Counting objects: 51% (1325/2597) 2024-06-26T07:03:12.0730311Z remote: Counting objects: 52% (1351/2597) 2024-06-26T07:03:12.0731048Z remote: Counting objects: 53% (1377/2597) 2024-06-26T07:03:12.0731749Z remote: Counting objects: 54% (1403/2597) 2024-06-26T07:03:12.0732438Z remote: Counting objects: 55% (1429/2597) 2024-06-26T07:03:12.0732947Z remote: Counting objects: 56% (1455/2597) 2024-06-26T07:03:12.0733448Z remote: Counting objects: 57% (1481/2597) 2024-06-26T07:03:12.0734179Z remote: Counting objects: 58% (1507/2597) 2024-06-26T07:03:12.0734853Z remote: Counting objects: 59% (1533/2597) 2024-06-26T07:03:12.0735564Z remote: Counting objects: 60% (1559/2597) 2024-06-26T07:03:12.0736132Z remote: Counting objects: 61% (1585/2597) 2024-06-26T07:03:12.0736857Z remote: Counting objects: 62% (1611/2597) 2024-06-26T07:03:12.0737650Z remote: Counting objects: 63% (1637/2597) 2024-06-26T07:03:12.0738327Z remote: Counting objects: 64% (1663/2597) 2024-06-26T07:03:12.0738831Z remote: Counting objects: 65% (1689/2597) 2024-06-26T07:03:12.0739334Z remote: Counting objects: 66% (1715/2597) 2024-06-26T07:03:12.0740072Z remote: Counting objects: 67% (1740/2597) 2024-06-26T07:03:12.0740694Z remote: Counting objects: 68% (1766/2597) 2024-06-26T07:03:12.0741464Z remote: Counting objects: 69% (1792/2597) 2024-06-26T07:03:12.0742200Z remote: Counting objects: 70% (1818/2597) 2024-06-26T07:03:12.0742752Z remote: Counting objects: 71% (1844/2597) 2024-06-26T07:03:12.0743252Z remote: Counting objects: 72% (1870/2597) 2024-06-26T07:03:12.0743752Z remote: Counting objects: 73% (1896/2597) 2024-06-26T07:03:12.0744309Z remote: Counting objects: 74% (1922/2597) 2024-06-26T07:03:12.0744811Z remote: Counting objects: 75% (1948/2597) 2024-06-26T07:03:12.0745374Z remote: Counting objects: 76% (1974/2597) 2024-06-26T07:03:12.0745879Z remote: Counting objects: 77% (2000/2597) 2024-06-26T07:03:12.0746377Z remote: Counting objects: 78% (2026/2597) 2024-06-26T07:03:12.0746997Z remote: Counting objects: 79% (2052/2597) 2024-06-26T07:03:12.0747500Z remote: Counting objects: 80% (2078/2597) 2024-06-26T07:03:12.0748003Z remote: Counting objects: 81% (2104/2597) 2024-06-26T07:03:12.0748495Z remote: Counting objects: 82% (2130/2597) 2024-06-26T07:03:12.0748996Z remote: Counting objects: 83% (2156/2597) 2024-06-26T07:03:12.0749500Z remote: Counting objects: 84% (2182/2597) 2024-06-26T07:03:12.0750000Z remote: Counting objects: 85% (2208/2597) 2024-06-26T07:03:12.0750493Z remote: Counting objects: 86% (2234/2597) 2024-06-26T07:03:12.0750992Z remote: Counting objects: 87% (2260/2597) 2024-06-26T07:03:12.0751490Z remote: Counting objects: 88% (2286/2597) 2024-06-26T07:03:12.0752004Z remote: Counting objects: 89% (2312/2597) 2024-06-26T07:03:12.0752505Z remote: Counting objects: 90% (2338/2597) 2024-06-26T07:03:12.0753017Z remote: Counting objects: 91% (2364/2597) 2024-06-26T07:03:12.0753531Z remote: Counting objects: 92% (2390/2597) 2024-06-26T07:03:12.0754031Z remote: Counting objects: 93% (2416/2597) 2024-06-26T07:03:12.0754526Z remote: Counting objects: 94% (2442/2597) 2024-06-26T07:03:12.0755024Z remote: Counting objects: 95% (2468/2597) 2024-06-26T07:03:12.0755531Z remote: Counting objects: 96% (2494/2597) 2024-06-26T07:03:12.0756036Z remote: Counting objects: 97% (2520/2597) 2024-06-26T07:03:12.0756529Z remote: Counting objects: 98% (2546/2597) 2024-06-26T07:03:12.0757028Z remote: Counting objects: 99% (2572/2597) 2024-06-26T07:03:12.0757532Z remote: Counting objects: 100% (2597/2597) 2024-06-26T07:03:12.0758075Z remote: Counting objects: 100% (2597/2597), done. 2024-06-26T07:03:12.0860845Z remote: Compressing objects: 0% (1/1074) 2024-06-26T07:03:12.1225403Z remote: Compressing objects: 1% (11/1074) 2024-06-26T07:03:12.1623429Z remote: Compressing objects: 2% (22/1074) 2024-06-26T07:03:12.1961098Z remote: Compressing objects: 3% (33/1074) 2024-06-26T07:03:12.2151669Z remote: Compressing objects: 4% (43/1074) 2024-06-26T07:03:12.2383407Z remote: Compressing objects: 5% (54/1074) 2024-06-26T07:03:12.2564103Z remote: Compressing objects: 6% (65/1074) 2024-06-26T07:03:12.2898888Z remote: Compressing objects: 7% (76/1074) 2024-06-26T07:03:12.3631456Z remote: Compressing objects: 8% (86/1074) 2024-06-26T07:03:12.4313115Z remote: Compressing objects: 9% (97/1074) 2024-06-26T07:03:12.4663521Z remote: Compressing objects: 10% (108/1074) 2024-06-26T07:03:12.4692816Z remote: Compressing objects: 11% (119/1074) 2024-06-26T07:03:12.4804398Z remote: Compressing objects: 12% (129/1074) 2024-06-26T07:03:12.4877967Z remote: Compressing objects: 13% (140/1074) 2024-06-26T07:03:12.4919845Z remote: Compressing objects: 14% (151/1074) 2024-06-26T07:03:12.4929772Z remote: Compressing objects: 15% (162/1074) 2024-06-26T07:03:12.4975696Z remote: Compressing objects: 16% (172/1074) 2024-06-26T07:03:12.4988359Z remote: Compressing objects: 17% (183/1074) 2024-06-26T07:03:12.4988900Z remote: Compressing objects: 18% (194/1074) 2024-06-26T07:03:12.4995742Z remote: Compressing objects: 19% (205/1074) 2024-06-26T07:03:12.5011328Z remote: Compressing objects: 20% (215/1074) 2024-06-26T07:03:12.5032581Z remote: Compressing objects: 21% (226/1074) 2024-06-26T07:03:12.5045872Z remote: Compressing objects: 22% (237/1074) 2024-06-26T07:03:12.5064604Z remote: Compressing objects: 23% (248/1074) 2024-06-26T07:03:12.5081139Z remote: Compressing objects: 24% (258/1074) 2024-06-26T07:03:12.5097616Z remote: Compressing objects: 25% (269/1074) 2024-06-26T07:03:12.5098154Z remote: Compressing objects: 26% (280/1074) 2024-06-26T07:03:12.5098860Z remote: Compressing objects: 27% (290/1074) 2024-06-26T07:03:12.5114477Z remote: Compressing objects: 28% (301/1074) 2024-06-26T07:03:12.5115018Z remote: Compressing objects: 29% (312/1074) 2024-06-26T07:03:12.5115547Z remote: Compressing objects: 30% (323/1074) 2024-06-26T07:03:12.5132118Z remote: Compressing objects: 31% (333/1074) 2024-06-26T07:03:12.5132651Z remote: Compressing objects: 32% (344/1074) 2024-06-26T07:03:12.5150287Z remote: Compressing objects: 33% (355/1074) 2024-06-26T07:03:12.5150821Z remote: Compressing objects: 34% (366/1074) 2024-06-26T07:03:12.5166535Z remote: Compressing objects: 35% (376/1074) 2024-06-26T07:03:12.5181892Z remote: Compressing objects: 36% (387/1074) 2024-06-26T07:03:12.5194120Z remote: Compressing objects: 37% (398/1074) 2024-06-26T07:03:12.5205109Z remote: Compressing objects: 38% (409/1074) 2024-06-26T07:03:12.5215608Z remote: Compressing objects: 39% (419/1074) 2024-06-26T07:03:12.5227951Z remote: Compressing objects: 40% (430/1074) 2024-06-26T07:03:12.5240244Z remote: Compressing objects: 41% (441/1074) 2024-06-26T07:03:12.5252802Z remote: Compressing objects: 42% (452/1074) 2024-06-26T07:03:12.5259091Z remote: Compressing objects: 43% (462/1074) 2024-06-26T07:03:12.5267333Z remote: Compressing objects: 44% (473/1074) 2024-06-26T07:03:12.5270279Z remote: Compressing objects: 45% (484/1074) 2024-06-26T07:03:12.5277750Z remote: Compressing objects: 46% (495/1074) 2024-06-26T07:03:12.5293440Z remote: Compressing objects: 47% (505/1074) 2024-06-26T07:03:12.5302709Z remote: Compressing objects: 48% (516/1074) 2024-06-26T07:03:12.5306770Z remote: Compressing objects: 49% (527/1074) 2024-06-26T07:03:12.5310956Z remote: Compressing objects: 50% (537/1074) 2024-06-26T07:03:12.5319032Z remote: Compressing objects: 51% (548/1074) 2024-06-26T07:03:12.5328921Z remote: Compressing objects: 52% (559/1074) 2024-06-26T07:03:12.5340123Z remote: Compressing objects: 53% (570/1074) 2024-06-26T07:03:12.5356834Z remote: Compressing objects: 54% (580/1074) 2024-06-26T07:03:12.5366510Z remote: Compressing objects: 55% (591/1074) 2024-06-26T07:03:12.5377384Z remote: Compressing objects: 56% (602/1074) 2024-06-26T07:03:12.5386742Z remote: Compressing objects: 57% (613/1074) 2024-06-26T07:03:12.5400313Z remote: Compressing objects: 58% (623/1074) 2024-06-26T07:03:12.5410168Z remote: Compressing objects: 59% (634/1074) 2024-06-26T07:03:12.5418829Z remote: Compressing objects: 60% (645/1074) 2024-06-26T07:03:12.5430926Z remote: Compressing objects: 61% (656/1074) 2024-06-26T07:03:12.5437089Z remote: Compressing objects: 62% (666/1074) 2024-06-26T07:03:12.5445617Z remote: Compressing objects: 63% (677/1074) 2024-06-26T07:03:12.5455484Z remote: Compressing objects: 64% (688/1074) 2024-06-26T07:03:12.5461051Z remote: Compressing objects: 65% (699/1074) 2024-06-26T07:03:12.5464072Z remote: Compressing objects: 66% (709/1074) 2024-06-26T07:03:12.5470158Z remote: Compressing objects: 67% (720/1074) 2024-06-26T07:03:12.5474662Z remote: Compressing objects: 68% (731/1074) 2024-06-26T07:03:12.5477362Z remote: Compressing objects: 69% (742/1074) 2024-06-26T07:03:12.5481267Z remote: Compressing objects: 70% (752/1074) 2024-06-26T07:03:12.5484146Z remote: Compressing objects: 71% (763/1074) 2024-06-26T07:03:12.5487046Z remote: Compressing objects: 72% (774/1074) 2024-06-26T07:03:12.5489912Z remote: Compressing objects: 73% (785/1074) 2024-06-26T07:03:12.5490481Z remote: Compressing objects: 74% (795/1074) 2024-06-26T07:03:12.5491096Z remote: Compressing objects: 75% (806/1074) 2024-06-26T07:03:12.5491624Z remote: Compressing objects: 76% (817/1074) 2024-06-26T07:03:12.5492270Z remote: Compressing objects: 77% (827/1074) 2024-06-26T07:03:12.5492818Z remote: Compressing objects: 78% (838/1074) 2024-06-26T07:03:12.5494733Z remote: Compressing objects: 79% (849/1074) 2024-06-26T07:03:12.5497494Z remote: Compressing objects: 80% (860/1074) 2024-06-26T07:03:12.5498029Z remote: Compressing objects: 81% (870/1074) 2024-06-26T07:03:12.5500356Z remote: Compressing objects: 82% (881/1074) 2024-06-26T07:03:12.5503461Z remote: Compressing objects: 83% (892/1074) 2024-06-26T07:03:12.5504096Z remote: Compressing objects: 84% (903/1074) 2024-06-26T07:03:12.5504629Z remote: Compressing objects: 85% (913/1074) 2024-06-26T07:03:12.5506905Z remote: Compressing objects: 86% (924/1074) 2024-06-26T07:03:12.5511385Z remote: Compressing objects: 87% (935/1074) 2024-06-26T07:03:12.5511912Z remote: Compressing objects: 88% (946/1074) 2024-06-26T07:03:12.5512440Z remote: Compressing objects: 89% (956/1074) 2024-06-26T07:03:12.5516417Z remote: Compressing objects: 90% (967/1074) 2024-06-26T07:03:12.5516954Z remote: Compressing objects: 91% (978/1074) 2024-06-26T07:03:12.5517475Z remote: Compressing objects: 92% (989/1074) 2024-06-26T07:03:12.5518002Z remote: Compressing objects: 93% (999/1074) 2024-06-26T07:03:12.5519965Z remote: Compressing objects: 94% (1010/1074) 2024-06-26T07:03:12.5520563Z remote: Compressing objects: 95% (1021/1074) 2024-06-26T07:03:12.5521188Z remote: Compressing objects: 96% (1032/1074) 2024-06-26T07:03:12.5521728Z remote: Compressing objects: 97% (1042/1074) 2024-06-26T07:03:12.5522424Z remote: Compressing objects: 98% (1053/1074) 2024-06-26T07:03:12.5522966Z remote: Compressing objects: 99% (1064/1074) 2024-06-26T07:03:12.5527215Z remote: Compressing objects: 100% (1074/1074) 2024-06-26T07:03:12.5527806Z remote: Compressing objects: 100% (1074/1074), done. 2024-06-26T07:03:32.5466142Z remote: Total 990209 (delta 2062), reused 1863 (delta 1522), pack-reused 987612 2024-06-26T07:03:57.7137245Z [command]/usr/bin/git rev-parse --verify --quiet b8c4c54d347aa776934c60784e35936878ef18dc^{object} 2024-06-26T07:03:57.7158033Z b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:03:57.7161701Z ##[endgroup] 2024-06-26T07:03:57.7162381Z ##[group]Determining the checkout info 2024-06-26T07:03:57.7165088Z ##[endgroup] 2024-06-26T07:03:57.7165649Z ##[group]Checking out the ref 2024-06-26T07:03:57.7166877Z [command]/usr/bin/git checkout --quiet --force b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:03:58.9228135Z ##[endgroup] 2024-06-26T07:03:58.9228868Z ##[group]Setting up auth for fetching submodules 2024-06-26T07:03:58.9232515Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:03:58.9274321Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-06-26T07:03:58.9294362Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-06-26T07:03:58.9313410Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-06-26T07:03:58.9330094Z ##[endgroup] 2024-06-26T07:03:58.9330623Z ##[group]Fetching submodules 2024-06-26T07:03:58.9334058Z [command]/usr/bin/git submodule sync --recursive 2024-06-26T07:03:58.9534014Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-06-26T07:03:58.9735311Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-06-26T07:03:58.9736730Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-06-26T07:03:58.9738140Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-06-26T07:03:58.9740082Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-06-26T07:03:58.9742926Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-06-26T07:03:58.9744894Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-06-26T07:03:58.9747683Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-06-26T07:03:58.9750294Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-06-26T07:03:58.9752934Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-06-26T07:03:58.9755719Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-06-26T07:03:58.9758344Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-06-26T07:03:58.9761181Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-06-26T07:03:58.9764468Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-06-26T07:03:58.9767504Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-06-26T07:03:58.9770420Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-06-26T07:03:58.9773512Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-06-26T07:03:58.9776811Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:03:58.9779846Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-06-26T07:03:58.9783217Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-06-26T07:03:58.9786763Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-06-26T07:03:58.9790176Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-06-26T07:03:58.9793695Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-06-26T07:03:58.9797404Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-06-26T07:03:58.9800953Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-06-26T07:03:58.9805089Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-06-26T07:03:58.9808703Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-06-26T07:03:58.9812817Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-06-26T07:03:58.9816505Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-06-26T07:03:58.9820469Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-06-26T07:03:58.9824592Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-06-26T07:03:58.9828825Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-06-26T07:03:58.9833026Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-06-26T07:03:58.9837137Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-06-26T07:03:58.9841258Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-06-26T07:03:58.9846166Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-06-26T07:03:58.9865576Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-06-26T07:03:59.2495728Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-06-26T07:03:59.4245983Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-06-26T07:03:59.5864722Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-06-26T07:03:59.8961829Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-06-26T07:04:01.9108948Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-06-26T07:04:11.7710775Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-06-26T07:04:12.1576190Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-06-26T07:04:12.6367181Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-06-26T07:04:13.2535300Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-06-26T07:04:14.4079621Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-06-26T07:04:16.1904754Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-06-26T07:04:22.0055359Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-06-26T07:04:23.2439883Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-06-26T07:04:24.9966845Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-06-26T07:04:26.1924845Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-06-26T07:04:26.3424838Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-06-26T07:04:26.7211007Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-06-26T07:04:27.0232466Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-06-26T07:04:27.9985800Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-06-26T07:04:28.3155718Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-06-26T07:04:28.5540427Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-06-26T07:04:30.0038800Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-06-26T07:04:30.7029951Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-06-26T07:04:31.3554129Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-06-26T07:04:36.9428888Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-06-26T07:04:39.3383606Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-06-26T07:04:44.8743468Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-06-26T07:04:45.0572253Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-06-26T07:04:53.2943808Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-06-26T07:04:53.4809918Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-06-26T07:04:53.7298972Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-06-26T07:04:54.6911436Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-06-26T07:04:55.0013458Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-06-26T07:04:55.6173609Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-06-26T07:04:56.0235148Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-06-26T07:04:56.0317257Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-06-26T07:04:56.0378615Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-06-26T07:04:56.0560369Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-06-26T07:04:56.0859673Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-06-26T07:04:56.8555821Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-06-26T07:04:56.8727372Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-06-26T07:04:56.9081463Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-06-26T07:04:56.9921947Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-06-26T07:04:57.0188393Z Submodule path 'third_party/cudnn_frontend': checked out 'aa3abd4bc689d6412979c7f55f9cd132848c9c6a' 2024-06-26T07:04:57.4423795Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-06-26T07:04:57.6529923Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-06-26T07:04:57.7161026Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-06-26T07:04:57.7172032Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:04:57.7173753Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:04:57.7176092Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:04:57.7178300Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:04:57.7180884Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:04:57.7198816Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-06-26T07:04:58.7316355Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-06-26T07:04:59.2982211Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-06-26T07:05:01.0609186Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-06-26T07:05:02.0013766Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-06-26T07:05:02.3414023Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-06-26T07:05:02.4245741Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-06-26T07:05:02.7638523Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-06-26T07:05:02.8180018Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-06-26T07:05:02.8263514Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-06-26T07:05:02.9175380Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-06-26T07:05:02.9457858Z Submodule path 'third_party/fmt': checked out 'e69e5f977d458f2650bb346dadf2ad30c5320281' 2024-06-26T07:05:02.9523071Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-06-26T07:05:02.9828007Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-06-26T07:05:03.0025184Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-06-26T07:05:03.0388688Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:05:03.0482610Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-06-26T07:05:03.0492421Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-06-26T07:05:03.0508983Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-06-26T07:05:15.2151319Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-06-26T07:05:15.2287580Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-06-26T07:05:15.3060653Z Submodule path 'third_party/kineto': checked out '8681ff11e1fa54da39023076c5c43eddd87b7a8a' 2024-06-26T07:05:15.3071063Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:05:15.3073117Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:05:15.3075083Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:05:15.3092208Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-06-26T07:05:15.8326428Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-06-26T07:05:17.0686084Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-06-26T07:05:18.1018678Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-06-26T07:05:18.1028555Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:05:18.1030270Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:05:18.1031949Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:05:18.1034053Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:05:18.1035863Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:05:18.1038128Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:05:18.1039988Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:05:18.1069405Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:05:18.1071026Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-06-26T07:05:19.1224804Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-06-26T07:05:19.5279590Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-06-26T07:05:20.7897437Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-06-26T07:05:21.0465931Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-06-26T07:05:21.5457667Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-06-26T07:05:22.4930238Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-06-26T07:05:28.0205269Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-06-26T07:05:28.4028831Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-06-26T07:05:28.4164671Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-06-26T07:05:28.4451753Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-06-26T07:05:28.4549837Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-06-26T07:05:28.4560835Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:05:28.4578275Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-06-26T07:05:28.7310347Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-06-26T07:05:28.7449883Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-06-26T07:05:28.7790465Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-06-26T07:05:28.8594998Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-06-26T07:05:28.8714526Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-06-26T07:05:28.9002222Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out 'a33701196adfad74917046096bf5a2aa0ab0bb50' 2024-06-26T07:05:28.9488885Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-06-26T07:05:28.9788937Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-06-26T07:05:28.9988201Z Submodule path 'third_party/nccl/nccl': checked out '48bb7fec7953112ff37499a272317f6663f8f600' 2024-06-26T07:05:29.0840510Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-06-26T07:05:29.3542406Z Submodule path 'third_party/onnx': checked out '990217f043af7222348ca8f0301e17fa7b841781' 2024-06-26T07:05:29.3567048Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-06-26T07:05:29.3568846Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-06-26T07:05:29.3584113Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-06-26T07:05:29.8107069Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-06-26T07:05:30.8972641Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-06-26T07:05:30.9251993Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-06-26T07:05:30.9771908Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-06-26T07:05:30.9783953Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:05:30.9786530Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:05:30.9788912Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:05:30.9791636Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:05:30.9794458Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:05:30.9797452Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:05:30.9800285Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:05:30.9802978Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:05:30.9819154Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-06-26T07:05:31.4350451Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-06-26T07:05:32.4573556Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-06-26T07:05:32.7662667Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-06-26T07:05:39.1601743Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-06-26T07:05:39.4530027Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-06-26T07:05:39.6624212Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-06-26T07:05:39.9347788Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-06-26T07:05:47.3273174Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-06-26T07:05:47.3604546Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-06-26T07:05:47.3727939Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-06-26T07:05:47.4556433Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-06-26T07:05:47.4652677Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-06-26T07:05:47.4757602Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-06-26T07:05:47.4868665Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-06-26T07:05:47.4878113Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:05:47.4880085Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:05:47.4897457Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-06-26T07:05:49.2879364Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-06-26T07:05:50.4174896Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-06-26T07:05:50.4533025Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:05:50.8061206Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-06-26T07:05:50.8141968Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-06-26T07:05:51.0287768Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-06-26T07:05:51.0300299Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:05:51.0302017Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-06-26T07:05:51.0318649Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-06-26T07:05:51.4184677Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-06-26T07:05:52.3814806Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-06-26T07:05:52.4427496Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-06-26T07:05:52.4486331Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-06-26T07:05:52.4575051Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-06-26T07:05:52.4851690Z Submodule path 'third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b' 2024-06-26T07:05:52.5078759Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-06-26T07:05:52.5410065Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-06-26T07:05:52.5608053Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-06-26T07:05:52.5619697Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:05:52.5621629Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:05:52.5623337Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:05:52.5625197Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:05:52.5640681Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-06-26T07:05:53.5113209Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-06-26T07:05:53.7471488Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-06-26T07:05:54.7606912Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-06-26T07:05:55.8672508Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-06-26T07:05:55.8788164Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-06-26T07:05:55.9347242Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-06-26T07:05:55.9574561Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-06-26T07:05:55.9584452Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:05:55.9601461Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-06-26T07:05:56.1573360Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-06-26T07:05:56.1597659Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-06-26T07:05:56.1801577Z Entering 'android/libs/fbjni' 2024-06-26T07:05:56.1829514Z Entering 'third_party/FP16' 2024-06-26T07:05:56.1858088Z Entering 'third_party/FXdiv' 2024-06-26T07:05:56.1884123Z Entering 'third_party/NNPACK' 2024-06-26T07:05:56.1911576Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:05:56.1939824Z Entering 'third_party/XNNPACK' 2024-06-26T07:05:56.1977590Z Entering 'third_party/benchmark' 2024-06-26T07:05:56.2004237Z Entering 'third_party/cpp-httplib' 2024-06-26T07:05:56.2031124Z Entering 'third_party/cpuinfo' 2024-06-26T07:05:56.2060804Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:05:56.2086957Z Entering 'third_party/cutlass' 2024-06-26T07:05:56.2121160Z Entering 'third_party/eigen' 2024-06-26T07:05:56.2151970Z Entering 'third_party/fbgemm' 2024-06-26T07:05:56.2178219Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:05:56.2205955Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:05:56.2234430Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:05:56.2267339Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:05:56.2294993Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:05:56.2324047Z Entering 'third_party/flatbuffers' 2024-06-26T07:05:56.2352480Z Entering 'third_party/fmt' 2024-06-26T07:05:56.2380998Z Entering 'third_party/foxi' 2024-06-26T07:05:56.2408891Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:05:56.2435560Z Entering 'third_party/gloo' 2024-06-26T07:05:56.2462750Z Entering 'third_party/googletest' 2024-06-26T07:05:56.2490471Z Entering 'third_party/ideep' 2024-06-26T07:05:56.2516357Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:05:56.2548486Z Entering 'third_party/ittapi' 2024-06-26T07:05:56.2577664Z Entering 'third_party/kineto' 2024-06-26T07:05:56.2602988Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:05:56.2630165Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:05:56.2658812Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:05:56.2683926Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:05:56.2712166Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:05:56.2737837Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:05:56.2767941Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:05:56.2796652Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:05:56.2823208Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:05:56.2855346Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:05:56.2884002Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:05:56.2912448Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:05:56.2940254Z Entering 'third_party/mimalloc' 2024-06-26T07:05:56.2970035Z Entering 'third_party/nccl/nccl' 2024-06-26T07:05:56.2996339Z Entering 'third_party/nlohmann' 2024-06-26T07:05:56.3024272Z Entering 'third_party/onnx' 2024-06-26T07:05:56.3065400Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:05:56.3092513Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:05:56.3120688Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:05:56.3151472Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:05:56.3177395Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:05:56.3204990Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:05:56.3232862Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:05:56.3260794Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:05:56.3286787Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:05:56.3312716Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:05:56.3340193Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:05:56.3369414Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:05:56.3397586Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:05:56.3442212Z Entering 'third_party/pocketfft' 2024-06-26T07:05:56.3469587Z Entering 'third_party/protobuf' 2024-06-26T07:05:56.3498294Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:05:56.3526035Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:05:56.3552303Z Entering 'third_party/psimd' 2024-06-26T07:05:56.3580266Z Entering 'third_party/pthreadpool' 2024-06-26T07:05:56.3607278Z Entering 'third_party/pybind11' 2024-06-26T07:05:56.3636351Z Entering 'third_party/python-peachpy' 2024-06-26T07:05:56.3661267Z Entering 'third_party/sleef' 2024-06-26T07:05:56.3686409Z Entering 'third_party/tensorpipe' 2024-06-26T07:05:56.3715422Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:05:56.3741314Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:05:56.3766653Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:05:56.3793275Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:05:56.3820974Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:05:56.3859436Z ##[endgroup] 2024-06-26T07:05:56.3861437Z ##[group]Persisting credentials for submodules 2024-06-26T07:05:56.3865035Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-06-26T07:05:56.4075612Z Entering 'android/libs/fbjni' 2024-06-26T07:05:56.4110553Z Entering 'third_party/FP16' 2024-06-26T07:05:56.4146106Z Entering 'third_party/FXdiv' 2024-06-26T07:05:56.4181490Z Entering 'third_party/NNPACK' 2024-06-26T07:05:56.4217423Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:05:56.4251838Z Entering 'third_party/XNNPACK' 2024-06-26T07:05:56.4298205Z Entering 'third_party/benchmark' 2024-06-26T07:05:56.4333907Z Entering 'third_party/cpp-httplib' 2024-06-26T07:05:56.4369237Z Entering 'third_party/cpuinfo' 2024-06-26T07:05:56.4405865Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:05:56.4441200Z Entering 'third_party/cutlass' 2024-06-26T07:05:56.4483151Z Entering 'third_party/eigen' 2024-06-26T07:05:56.4516607Z Entering 'third_party/fbgemm' 2024-06-26T07:05:56.4551839Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:05:56.4584929Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:05:56.4617443Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:05:56.4656682Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:05:56.4689638Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:05:56.4725379Z Entering 'third_party/flatbuffers' 2024-06-26T07:05:56.4762641Z Entering 'third_party/fmt' 2024-06-26T07:05:56.4799015Z Entering 'third_party/foxi' 2024-06-26T07:05:56.4834807Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:05:56.4870360Z Entering 'third_party/gloo' 2024-06-26T07:05:56.4906824Z Entering 'third_party/googletest' 2024-06-26T07:05:56.4940582Z Entering 'third_party/ideep' 2024-06-26T07:05:56.4976344Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:05:56.5013759Z Entering 'third_party/ittapi' 2024-06-26T07:05:56.5049460Z Entering 'third_party/kineto' 2024-06-26T07:05:56.5084929Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:05:56.5119742Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:05:56.5155205Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:05:56.5191352Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:05:56.5228183Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:05:56.5261000Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:05:56.5296812Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:05:56.5331497Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:05:56.5366725Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:05:56.5402944Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:05:56.5438433Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:05:56.5471300Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:05:56.5505922Z Entering 'third_party/mimalloc' 2024-06-26T07:05:56.5541445Z Entering 'third_party/nccl/nccl' 2024-06-26T07:05:56.5577732Z Entering 'third_party/nlohmann' 2024-06-26T07:05:56.5610926Z Entering 'third_party/onnx' 2024-06-26T07:05:56.5657696Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:05:56.5694315Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:05:56.5730825Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:05:56.5768310Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:05:56.5802431Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:05:56.5836622Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:05:56.5871090Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:05:56.5908282Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:05:56.5942324Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:05:56.5976723Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:05:56.6007906Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:05:56.6043466Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:05:56.6079361Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:05:56.6129532Z Entering 'third_party/pocketfft' 2024-06-26T07:05:56.6163490Z Entering 'third_party/protobuf' 2024-06-26T07:05:56.6202666Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:05:56.6236615Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:05:56.6271521Z Entering 'third_party/psimd' 2024-06-26T07:05:56.6307745Z Entering 'third_party/pthreadpool' 2024-06-26T07:05:56.6340648Z Entering 'third_party/pybind11' 2024-06-26T07:05:56.6376290Z Entering 'third_party/python-peachpy' 2024-06-26T07:05:56.6410303Z Entering 'third_party/sleef' 2024-06-26T07:05:56.6445839Z Entering 'third_party/tensorpipe' 2024-06-26T07:05:56.6479514Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:05:56.6515445Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:05:56.6551688Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:05:56.6584094Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:05:56.6616626Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:05:56.6664960Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-06-26T07:05:56.6873110Z Entering 'android/libs/fbjni' 2024-06-26T07:05:56.6904862Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-06-26T07:05:56.6916808Z Entering 'third_party/FP16' 2024-06-26T07:05:56.6949268Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-06-26T07:05:56.6962219Z Entering 'third_party/FXdiv' 2024-06-26T07:05:56.6994140Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-06-26T07:05:56.7008119Z Entering 'third_party/NNPACK' 2024-06-26T07:05:56.7038933Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-06-26T07:05:56.7052322Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:05:56.7083360Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-06-26T07:05:56.7096670Z Entering 'third_party/XNNPACK' 2024-06-26T07:05:56.7126800Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-06-26T07:05:56.7149953Z Entering 'third_party/benchmark' 2024-06-26T07:05:56.7180918Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:05:56.7193737Z Entering 'third_party/cpp-httplib' 2024-06-26T07:05:56.7225765Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-06-26T07:05:56.7237437Z Entering 'third_party/cpuinfo' 2024-06-26T07:05:56.7268763Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:05:56.7281901Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:05:56.7313790Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-06-26T07:05:56.7327143Z Entering 'third_party/cutlass' 2024-06-26T07:05:56.7358469Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:05:56.7378357Z Entering 'third_party/eigen' 2024-06-26T07:05:56.7410426Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-06-26T07:05:56.7424880Z Entering 'third_party/fbgemm' 2024-06-26T07:05:56.7456737Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-06-26T07:05:56.7470958Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:05:56.7503480Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-06-26T07:05:56.7516218Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:05:56.7549616Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:05:56.7560698Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:05:56.7594154Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:05:56.7611949Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:05:56.7643004Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:05:56.7654617Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:05:56.7687083Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-06-26T07:05:56.7700993Z Entering 'third_party/flatbuffers' 2024-06-26T07:05:56.7732412Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-06-26T07:05:56.7748523Z Entering 'third_party/fmt' 2024-06-26T07:05:56.7782256Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:05:56.7794096Z Entering 'third_party/foxi' 2024-06-26T07:05:56.7825417Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-06-26T07:05:56.7838220Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:05:56.7870981Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-06-26T07:05:56.7884199Z Entering 'third_party/gloo' 2024-06-26T07:05:56.7915896Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-06-26T07:05:56.7928476Z Entering 'third_party/googletest' 2024-06-26T07:05:56.7957090Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:05:56.7968846Z Entering 'third_party/ideep' 2024-06-26T07:05:56.8000209Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-06-26T07:05:56.8013159Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:05:56.8044988Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-06-26T07:05:56.8062815Z Entering 'third_party/ittapi' 2024-06-26T07:05:56.8094513Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-06-26T07:05:56.8107999Z Entering 'third_party/kineto' 2024-06-26T07:05:56.8140857Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-06-26T07:05:56.8153418Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:05:56.8186139Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-06-26T07:05:56.8198802Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:05:56.8231392Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-06-26T07:05:56.8244856Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:05:56.8277264Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-06-26T07:05:56.8289428Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:05:56.8320697Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:05:56.8332038Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:05:56.8366791Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-06-26T07:05:56.8378271Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:05:56.8410332Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-06-26T07:05:56.8425152Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:05:56.8457632Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-06-26T07:05:56.8470530Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:05:56.8501426Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:05:56.8512203Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:05:56.8545046Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-06-26T07:05:56.8557612Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:05:56.8590556Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-06-26T07:05:56.8604962Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:05:56.8635834Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-06-26T07:05:56.8649043Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:05:56.8680239Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-06-26T07:05:56.8693563Z Entering 'third_party/mimalloc' 2024-06-26T07:05:56.8724724Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-06-26T07:05:56.8739239Z Entering 'third_party/nccl/nccl' 2024-06-26T07:05:56.8769546Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-06-26T07:05:56.8780162Z Entering 'third_party/nlohmann' 2024-06-26T07:05:56.8811851Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-06-26T07:05:56.8825688Z Entering 'third_party/onnx' 2024-06-26T07:05:56.8856811Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-06-26T07:05:56.8883069Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:05:56.8915901Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:05:56.8930675Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:05:56.8961772Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:05:56.8978811Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:05:56.9010380Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-06-26T07:05:56.9025526Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:05:56.9056201Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:05:56.9069976Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:05:56.9101022Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:05:56.9113991Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:05:56.9145989Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-06-26T07:05:56.9157623Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:05:56.9189945Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-06-26T07:05:56.9203649Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:05:56.9234363Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-06-26T07:05:56.9246659Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:05:56.9275540Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-06-26T07:05:56.9287022Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:05:56.9318772Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-06-26T07:05:56.9331328Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:05:56.9364177Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-06-26T07:05:56.9378543Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:05:56.9408388Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-06-26T07:05:56.9421745Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:05:56.9452954Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-06-26T07:05:56.9481180Z Entering 'third_party/pocketfft' 2024-06-26T07:05:56.9512925Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-06-26T07:05:56.9527134Z Entering 'third_party/protobuf' 2024-06-26T07:05:56.9559416Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-06-26T07:05:56.9573865Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:05:56.9605827Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:05:56.9618891Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:05:56.9649638Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:05:56.9662951Z Entering 'third_party/psimd' 2024-06-26T07:05:56.9695064Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-06-26T07:05:56.9707495Z Entering 'third_party/pthreadpool' 2024-06-26T07:05:56.9739438Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-06-26T07:05:56.9753030Z Entering 'third_party/pybind11' 2024-06-26T07:05:56.9785875Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:05:56.9798130Z Entering 'third_party/python-peachpy' 2024-06-26T07:05:56.9828239Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-06-26T07:05:56.9840431Z Entering 'third_party/sleef' 2024-06-26T07:05:56.9873930Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-06-26T07:05:56.9886922Z Entering 'third_party/tensorpipe' 2024-06-26T07:05:56.9918243Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-06-26T07:05:56.9930922Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:05:56.9961715Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:05:56.9973782Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:05:57.0004748Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-06-26T07:05:57.0017458Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:05:57.0048354Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-06-26T07:05:57.0060412Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:05:57.0092498Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:05:57.0103939Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:05:57.0135096Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-06-26T07:05:57.1003390Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-06-26T07:05:57.1205799Z Entering 'android/libs/fbjni' 2024-06-26T07:05:57.1232155Z Entering 'third_party/FP16' 2024-06-26T07:05:57.1258387Z Entering 'third_party/FXdiv' 2024-06-26T07:05:57.1284271Z Entering 'third_party/NNPACK' 2024-06-26T07:05:57.1313025Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:05:57.1341476Z Entering 'third_party/XNNPACK' 2024-06-26T07:05:57.1378825Z Entering 'third_party/benchmark' 2024-06-26T07:05:57.1408182Z Entering 'third_party/cpp-httplib' 2024-06-26T07:05:57.1437270Z Entering 'third_party/cpuinfo' 2024-06-26T07:05:57.1464464Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:05:57.1493618Z Entering 'third_party/cutlass' 2024-06-26T07:05:57.1526811Z Entering 'third_party/eigen' 2024-06-26T07:05:57.1555975Z Entering 'third_party/fbgemm' 2024-06-26T07:05:57.1585955Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:05:57.1611884Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:05:57.1640574Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:05:57.1674383Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:05:57.1701389Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:05:57.1730410Z Entering 'third_party/flatbuffers' 2024-06-26T07:05:57.1760290Z Entering 'third_party/fmt' 2024-06-26T07:05:57.1787415Z Entering 'third_party/foxi' 2024-06-26T07:05:57.1816178Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:05:57.1842537Z Entering 'third_party/gloo' 2024-06-26T07:05:57.1870620Z Entering 'third_party/googletest' 2024-06-26T07:05:57.1899357Z Entering 'third_party/ideep' 2024-06-26T07:05:57.1927535Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:05:57.1958179Z Entering 'third_party/ittapi' 2024-06-26T07:05:57.1987618Z Entering 'third_party/kineto' 2024-06-26T07:05:57.2014771Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:05:57.2043130Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:05:57.2071602Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:05:57.2099026Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:05:57.2127580Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:05:57.2155398Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:05:57.2184734Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:05:57.2211525Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:05:57.2239362Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:05:57.2266288Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:05:57.2292475Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:05:57.2320971Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:05:57.2350283Z Entering 'third_party/mimalloc' 2024-06-26T07:05:57.2379098Z Entering 'third_party/nccl/nccl' 2024-06-26T07:05:57.2409323Z Entering 'third_party/nlohmann' 2024-06-26T07:05:57.2438132Z Entering 'third_party/onnx' 2024-06-26T07:05:57.2476131Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:05:57.2503153Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:05:57.2533946Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:05:57.2563179Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:05:57.2591618Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:05:57.2618384Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:05:57.2646805Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:05:57.2674325Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:05:57.2702882Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:05:57.2728752Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:05:57.2755199Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:05:57.2782539Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:05:57.2809049Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:05:57.2851447Z Entering 'third_party/pocketfft' 2024-06-26T07:05:57.2878545Z Entering 'third_party/protobuf' 2024-06-26T07:05:57.2910239Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:05:57.2937101Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:05:57.2963808Z Entering 'third_party/psimd' 2024-06-26T07:05:57.2992572Z Entering 'third_party/pthreadpool' 2024-06-26T07:05:57.3020400Z Entering 'third_party/pybind11' 2024-06-26T07:05:57.3048231Z Entering 'third_party/python-peachpy' 2024-06-26T07:05:57.3078545Z Entering 'third_party/sleef' 2024-06-26T07:05:57.3109159Z Entering 'third_party/tensorpipe' 2024-06-26T07:05:57.3136098Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:05:57.3164599Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:05:57.3190617Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:05:57.3217167Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:05:57.3242049Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:05:57.3284907Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-06-26T07:05:57.3490914Z Entering 'android/libs/fbjni' 2024-06-26T07:05:57.3519993Z Entering 'third_party/FP16' 2024-06-26T07:05:57.3550082Z Entering 'third_party/FXdiv' 2024-06-26T07:05:57.3577018Z Entering 'third_party/NNPACK' 2024-06-26T07:05:57.3604726Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:05:57.3634376Z Entering 'third_party/XNNPACK' 2024-06-26T07:05:57.3672913Z Entering 'third_party/benchmark' 2024-06-26T07:05:57.3702261Z Entering 'third_party/cpp-httplib' 2024-06-26T07:05:57.3732039Z Entering 'third_party/cpuinfo' 2024-06-26T07:05:57.3761907Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:05:57.3789786Z Entering 'third_party/cutlass' 2024-06-26T07:05:57.3822903Z Entering 'third_party/eigen' 2024-06-26T07:05:57.3853874Z Entering 'third_party/fbgemm' 2024-06-26T07:05:57.3880622Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:05:57.3908503Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:05:57.3936068Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:05:57.3968891Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:05:57.3996547Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:05:57.4025965Z Entering 'third_party/flatbuffers' 2024-06-26T07:05:57.4054122Z Entering 'third_party/fmt' 2024-06-26T07:05:57.4082039Z Entering 'third_party/foxi' 2024-06-26T07:05:57.4110956Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:05:57.4139353Z Entering 'third_party/gloo' 2024-06-26T07:05:57.4167331Z Entering 'third_party/googletest' 2024-06-26T07:05:57.4196360Z Entering 'third_party/ideep' 2024-06-26T07:05:57.4222152Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:05:57.4255521Z Entering 'third_party/ittapi' 2024-06-26T07:05:57.4286494Z Entering 'third_party/kineto' 2024-06-26T07:05:57.4313729Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:05:57.4339752Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:05:57.4367597Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:05:57.4394452Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:05:57.4422004Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:05:57.4448712Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:05:57.4477759Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:05:57.4507001Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:05:57.4532915Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:05:57.4560642Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:05:57.4591038Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:05:57.4619291Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:05:57.4648236Z Entering 'third_party/mimalloc' 2024-06-26T07:05:57.4674548Z Entering 'third_party/nccl/nccl' 2024-06-26T07:05:57.4700688Z Entering 'third_party/nlohmann' 2024-06-26T07:05:57.4731775Z Entering 'third_party/onnx' 2024-06-26T07:05:57.4770574Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:05:57.4798181Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:05:57.4829940Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:05:57.4860123Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:05:57.4889203Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:05:57.4916229Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:05:57.4944707Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:05:57.4972975Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:05:57.5000247Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:05:57.5027292Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:05:57.5055033Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:05:57.5084765Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:05:57.5112218Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:05:57.5153971Z Entering 'third_party/pocketfft' 2024-06-26T07:05:57.5184232Z Entering 'third_party/protobuf' 2024-06-26T07:05:57.5212846Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:05:57.5241145Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:05:57.5271737Z Entering 'third_party/psimd' 2024-06-26T07:05:57.5297445Z Entering 'third_party/pthreadpool' 2024-06-26T07:05:57.5321426Z Entering 'third_party/pybind11' 2024-06-26T07:05:57.5350164Z Entering 'third_party/python-peachpy' 2024-06-26T07:05:57.5377029Z Entering 'third_party/sleef' 2024-06-26T07:05:57.5407502Z Entering 'third_party/tensorpipe' 2024-06-26T07:05:57.5433863Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:05:57.5462567Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:05:57.5488260Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:05:57.5514929Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:05:57.5541940Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:05:57.5581089Z ##[endgroup] 2024-06-26T07:05:57.5610873Z [command]/usr/bin/git log -1 --format='%H' 2024-06-26T07:05:57.5631922Z 'b8c4c54d347aa776934c60784e35936878ef18dc' 2024-06-26T07:05:57.5799870Z Prepare all required actions 2024-06-26T07:05:57.5800319Z Getting action download info 2024-06-26T07:05:57.7410290Z ##[group]Run ./.github/actions/setup-linux 2024-06-26T07:05:57.7410723Z env: 2024-06-26T07:05:57.7411011Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:57.7411366Z ##[endgroup] 2024-06-26T07:05:57.7469668Z ##[group]Run set -euo pipefail 2024-06-26T07:05:57.7470102Z set -euo pipefail 2024-06-26T07:05:57.7470493Z function get_ec2_metadata() { 2024-06-26T07:05:57.7471027Z  # Pulled from instance metadata endpoint for EC2 2024-06-26T07:05:57.7471903Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-06-26T07:05:57.7472663Z  category=$1 2024-06-26T07:05:57.7473186Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-06-26T07:05:57.7473821Z  runner_name_str=i-02ccdeb779ec2a6a3 2024-06-26T07:05:57.7474317Z  if [[ -f /.inarc ]]; then 2024-06-26T07:05:57.7474803Z  echo "ARC Runner, no info on ec2 metadata" 2024-06-26T07:05:57.7475366Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-06-26T07:05:57.7476050Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-06-26T07:05:57.7476648Z  else 2024-06-26T07:05:57.7477133Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-06-26T07:05:57.7477708Z  fi 2024-06-26T07:05:57.7477996Z } 2024-06-26T07:05:57.7478360Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-06-26T07:05:57.7478937Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-06-26T07:05:57.7479584Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-06-26T07:05:57.7480148Z echo "system info $(uname -a)" 2024-06-26T07:05:57.7488120Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:05:57.7488605Z env: 2024-06-26T07:05:57.7488887Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:57.7489241Z ##[endgroup] 2024-06-26T07:05:57.7559651Z ami-id: ami-0ce0c36d7a00b20e2 2024-06-26T07:05:57.7600328Z instance-id: i-02ccdeb779ec2a6a3 2024-06-26T07:05:57.7640733Z instance-type: g5.4xlarge 2024-06-26T07:05:57.7647127Z system info Linux ip-10-0-17-21.ec2.internal 4.14.336-257.562.amzn2.x86_64 #1 SMP Sat Feb 24 09:50:35 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-06-26T07:05:57.7667508Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:05:57.7668402Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:05:57.7676220Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:05:57.7676717Z env: 2024-06-26T07:05:57.7676995Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:57.7677344Z ##[endgroup] 2024-06-26T07:05:57.7736620Z ##[group]Run if systemctl is-active --quiet docker; then 2024-06-26T07:05:57.7737320Z if systemctl is-active --quiet docker; then 2024-06-26T07:05:57.7737858Z  echo "Docker daemon is running..."; 2024-06-26T07:05:57.7738444Z else 2024-06-26T07:05:57.7738927Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-06-26T07:05:57.7739501Z fi 2024-06-26T07:05:57.7746396Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:05:57.7746886Z env: 2024-06-26T07:05:57.7747168Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:57.7747518Z ##[endgroup] 2024-06-26T07:05:57.7783130Z Docker daemon is running... 2024-06-26T07:05:57.7825159Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:05:57.7825704Z with: 2024-06-26T07:05:57.7825976Z shell: bash 2024-06-26T07:05:57.7826260Z timeout_minutes: 5 2024-06-26T07:05:57.7826579Z max_attempts: 3 2024-06-26T07:05:57.7826891Z retry_wait_seconds: 30 2024-06-26T07:05:57.7828351Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-06-26T07:05:57.7829753Z polling_interval_seconds: 1 2024-06-26T07:05:57.7830117Z warning_on_retry: true 2024-06-26T07:05:57.7830459Z continue_on_error: false 2024-06-26T07:05:57.7830790Z env: 2024-06-26T07:05:57.7831051Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:57.7831402Z AWS_RETRY_MODE: standard 2024-06-26T07:05:57.7831754Z AWS_MAX_ATTEMPTS: 5 2024-06-26T07:05:57.7832088Z AWS_DEFAULT_REGION: us-east-1 2024-06-26T07:05:57.7832444Z ##[endgroup] 2024-06-26T07:05:58.6372830Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:05:58.6373659Z Configure a credential helper to remove this warning. See 2024-06-26T07:05:58.6374534Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:05:58.6375061Z 2024-06-26T07:05:58.6375213Z Login Succeeded 2024-06-26T07:05:58.8327739Z Command completed after 1 attempt(s). 2024-06-26T07:05:58.8371053Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:05:58.8371785Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:05:58.8372435Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:05:58.8380094Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:05:58.8380584Z env: 2024-06-26T07:05:58.8380864Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:58.8381215Z ##[endgroup] 2024-06-26T07:05:58.8437936Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:05:58.8438697Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:05:58.8439281Z # shellcheck disable=SC2046 2024-06-26T07:05:58.8439741Z docker stop $(docker ps -q) || true 2024-06-26T07:05:58.8440226Z # Prune all of the docker images 2024-06-26T07:05:58.8440681Z docker system prune -af 2024-06-26T07:05:58.8448430Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:05:58.8448936Z env: 2024-06-26T07:05:58.8449206Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:58.8449560Z ##[endgroup] 2024-06-26T07:05:58.8690444Z "docker stop" requires at least 1 argument. 2024-06-26T07:05:58.8691202Z See 'docker stop --help'. 2024-06-26T07:05:58.8691536Z 2024-06-26T07:05:58.8691850Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-06-26T07:05:58.8692356Z 2024-06-26T07:05:58.8692555Z Stop one or more running containers 2024-06-26T07:05:58.8840492Z Total reclaimed space: 0B 2024-06-26T07:05:58.8869194Z ##[group]Run set +e 2024-06-26T07:05:58.8869561Z set +e 2024-06-26T07:05:58.8869867Z set -x 2024-06-26T07:05:58.8870164Z  2024-06-26T07:05:58.8870483Z PT_DOMAIN=download.pytorch.org 2024-06-26T07:05:58.8871272Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-06-26T07:05:58.8872334Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-06-26T07:05:58.8873182Z # one is returned at random 2024-06-26T07:05:58.8873723Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-06-26T07:05:58.8874226Z  2024-06-26T07:05:58.8874551Z if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:05:58.8875168Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-06-26T07:05:58.8875918Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-06-26T07:05:58.8876460Z  2024-06-26T07:05:58.8876792Z  if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:05:58.8877347Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-06-26T07:05:58.8877855Z  exit 1 2024-06-26T07:05:58.8878164Z  fi 2024-06-26T07:05:58.8878453Z fi 2024-06-26T07:05:58.8878846Z  2024-06-26T07:05:58.8879207Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-06-26T07:05:58.8879716Z  # Clean up any old records first 2024-06-26T07:05:58.8880229Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-06-26T07:05:58.8880689Z fi 2024-06-26T07:05:58.8880964Z  2024-06-26T07:05:58.8881396Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-06-26T07:05:58.8881937Z cat /etc/hosts 2024-06-26T07:05:58.8889882Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:05:58.8890405Z env: 2024-06-26T07:05:58.8890691Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:58.8891052Z ##[endgroup] 2024-06-26T07:05:58.8910594Z + PT_DOMAIN=download.pytorch.org 2024-06-26T07:05:58.8914218Z ++ dig -4 +short download.pytorch.org 2024-06-26T07:05:58.8914829Z ++ tail -n1 2024-06-26T07:05:58.9016757Z + RESOLVED_IP=18.160.10.22 2024-06-26T07:05:58.9017226Z + '[' -z 18.160.10.22 ']' 2024-06-26T07:05:58.9017683Z + grep -r download.pytorch.org /etc/hosts 2024-06-26T07:05:58.9022786Z 18.160.18.56 download.pytorch.org 2024-06-26T07:05:58.9023473Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-06-26T07:05:58.9111330Z + echo '18.160.10.22 download.pytorch.org' 2024-06-26T07:05:58.9111904Z + sudo tee -a /etc/hosts 2024-06-26T07:05:58.9171898Z 18.160.10.22 download.pytorch.org 2024-06-26T07:05:58.9185224Z + cat /etc/hosts 2024-06-26T07:05:58.9190166Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-06-26T07:05:58.9202686Z ::1 localhost6 localhost6.localdomain6 2024-06-26T07:05:58.9203203Z 18.160.10.22 download.pytorch.org 2024-06-26T07:05:58.9407697Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-06-26T07:05:58.9408284Z with: 2024-06-26T07:05:58.9409342Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:58.9410525Z docker-build-dir: .ci/docker 2024-06-26T07:05:58.9410886Z working-directory: . 2024-06-26T07:05:58.9411355Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:05:58.9411870Z force-push: false 2024-06-26T07:05:58.9412163Z env: 2024-06-26T07:05:58.9412426Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:58.9412757Z ##[endgroup] 2024-06-26T07:05:58.9429609Z ##[group]Run set -ex 2024-06-26T07:05:58.9429980Z set -ex 2024-06-26T07:05:58.9430264Z  2024-06-26T07:05:58.9430814Z # If the docker build directory or the build script doesn't exist, the action will 2024-06-26T07:05:58.9431770Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-06-26T07:05:58.9432552Z # job could then download the pre-built image as usual 2024-06-26T07:05:58.9433280Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-06-26T07:05:58.9433944Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:05:58.9434566Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:05:58.9435241Z  2024-06-26T07:05:58.9435740Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-06-26T07:05:58.9436355Z  exit 0 2024-06-26T07:05:58.9436658Z else 2024-06-26T07:05:58.9437023Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-06-26T07:05:58.9437461Z fi 2024-06-26T07:05:58.9437738Z  2024-06-26T07:05:58.9438203Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-06-26T07:05:58.9439033Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-06-26T07:05:58.9439791Z  # use it as it is, but first let's extract the tag 2024-06-26T07:05:58.9440483Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-06-26T07:05:58.9441199Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:05:58.9441889Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:05:58.9442754Z else 2024-06-26T07:05:58.9443313Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-06-26T07:05:58.9444011Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:05:58.9444879Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:05:58.9445602Z fi 2024-06-26T07:05:58.9453159Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:05:58.9453649Z env: 2024-06-26T07:05:58.9453921Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:58.9454260Z REPO_NAME: pytorch 2024-06-26T07:05:58.9455364Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:58.9456535Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:05:58.9457026Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:05:58.9457546Z ##[endgroup] 2024-06-26T07:05:58.9477946Z + [[ ! -d .ci/docker ]] 2024-06-26T07:05:58.9478370Z + [[ ! -f .ci/docker/build.sh ]] 2024-06-26T07:05:58.9478768Z + echo skip=false 2024-06-26T07:05:58.9480907Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-06-26T07:05:58.9484239Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:58.9485406Z ++ awk -F '[:,]' '{print $2}' 2024-06-26T07:05:58.9488934Z + DOCKER_TAG=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:58.9489590Z + echo docker-tag=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:58.9491250Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:58.9515840Z ##[group]Run set +e 2024-06-26T07:05:58.9516213Z set +e 2024-06-26T07:05:58.9516516Z set -x 2024-06-26T07:05:58.9516806Z  2024-06-26T07:05:58.9517086Z login() { 2024-06-26T07:05:58.9517747Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:05:58.9518452Z } 2024-06-26T07:05:58.9518725Z  2024-06-26T07:05:58.9519003Z retry () { 2024-06-26T07:05:58.9519393Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:05:58.9519836Z } 2024-06-26T07:05:58.9520104Z  2024-06-26T07:05:58.9520425Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:05:58.9520836Z  2024-06-26T07:05:58.9521299Z # Check if image already exists, if it does then skip building it 2024-06-26T07:05:58.9522097Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-06-26T07:05:58.9522892Z  exit 0 2024-06-26T07:05:58.9523195Z fi 2024-06-26T07:05:58.9523463Z  2024-06-26T07:05:58.9523957Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-06-26T07:05:58.9524772Z # be empty. The default action would be to continue rebuild the image 2024-06-26T07:05:58.9525500Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-06-26T07:05:58.9526159Z  # if we're on the base branch then use the parent commit 2024-06-26T07:05:58.9526745Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-06-26T07:05:58.9527171Z else 2024-06-26T07:05:58.9527633Z  # otherwise we're on a PR, so use the most recent base commit 2024-06-26T07:05:58.9528306Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-06-26T07:05:58.9528818Z fi 2024-06-26T07:05:58.9529099Z  2024-06-26T07:05:58.9529429Z if [[ -z "${MERGE_BASE}" ]]; then 2024-06-26T07:05:58.9529921Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:05:58.9530373Z  2024-06-26T07:05:58.9531021Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-06-26T07:05:58.9531767Z  exit 0 2024-06-26T07:05:58.9532060Z fi 2024-06-26T07:05:58.9532339Z  2024-06-26T07:05:58.9532785Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-06-26T07:05:58.9533738Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-06-26T07:05:58.9534517Z  exit 1 2024-06-26T07:05:58.9534825Z fi 2024-06-26T07:05:58.9535106Z  2024-06-26T07:05:58.9535586Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-06-26T07:05:58.9536509Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-06-26T07:05:58.9537359Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-06-26T07:05:58.9538305Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-06-26T07:05:58.9539373Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-06-26T07:05:58.9540174Z fi 2024-06-26T07:05:58.9540456Z  2024-06-26T07:05:58.9540814Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:05:58.9547829Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:05:58.9548320Z env: 2024-06-26T07:05:58.9548590Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:58.9548951Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:05:58.9549408Z BASE_REVISION: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:05:58.9550658Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:58.9551871Z DOCKER_TAG: 91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:58.9552461Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:05:58.9552978Z ##[endgroup] 2024-06-26T07:05:58.9571506Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:05:58.9572156Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:05:58.9573086Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:05:58.9574317Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:05:59.3791063Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:05:59.3792081Z Configure a credential helper to remove this warning. See 2024-06-26T07:05:59.3792963Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:05:59.3793688Z 2024-06-26T07:05:59.3793813Z Login Succeeded 2024-06-26T07:05:59.3802537Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:59.5757882Z { 2024-06-26T07:05:59.5758494Z "schemaVersion": 2, 2024-06-26T07:05:59.5759229Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-06-26T07:05:59.5759865Z "config": { 2024-06-26T07:05:59.5760484Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-06-26T07:05:59.5761161Z "size": 48681, 2024-06-26T07:05:59.5761807Z "digest": "sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd" 2024-06-26T07:05:59.5762682Z }, 2024-06-26T07:05:59.5762945Z "layers": [ 2024-06-26T07:05:59.5763221Z { 2024-06-26T07:05:59.5763650Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5764453Z "size": 28580681, 2024-06-26T07:05:59.5765167Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-06-26T07:05:59.5765978Z }, 2024-06-26T07:05:59.5766297Z { 2024-06-26T07:05:59.5766866Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5767587Z "size": 7943451, 2024-06-26T07:05:59.5768290Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-06-26T07:05:59.5769020Z }, 2024-06-26T07:05:59.5769581Z { 2024-06-26T07:05:59.5770289Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5770857Z "size": 55728572, 2024-06-26T07:05:59.5771392Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-06-26T07:05:59.5771991Z }, 2024-06-26T07:05:59.5772237Z { 2024-06-26T07:05:59.5772662Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5773201Z "size": 186, 2024-06-26T07:05:59.5773734Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-06-26T07:05:59.5774338Z }, 2024-06-26T07:05:59.5774584Z { 2024-06-26T07:05:59.5774999Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5775540Z "size": 6886, 2024-06-26T07:05:59.5776076Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-06-26T07:05:59.5776667Z }, 2024-06-26T07:05:59.5777179Z { 2024-06-26T07:05:59.5777613Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5778158Z "size": 1329001756, 2024-06-26T07:05:59.5778714Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-06-26T07:05:59.5779332Z }, 2024-06-26T07:05:59.5779580Z { 2024-06-26T07:05:59.5780000Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5780535Z "size": 62501, 2024-06-26T07:05:59.5781072Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-06-26T07:05:59.5781669Z }, 2024-06-26T07:05:59.5781916Z { 2024-06-26T07:05:59.5782331Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5782872Z "size": 1684, 2024-06-26T07:05:59.5783396Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-06-26T07:05:59.5784107Z }, 2024-06-26T07:05:59.5784354Z { 2024-06-26T07:05:59.5784782Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5785320Z "size": 1523, 2024-06-26T07:05:59.5785859Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-06-26T07:05:59.5786470Z }, 2024-06-26T07:05:59.5786712Z { 2024-06-26T07:05:59.5787133Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5787666Z "size": 2528295403, 2024-06-26T07:05:59.5788216Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-06-26T07:05:59.5788919Z }, 2024-06-26T07:05:59.5789165Z { 2024-06-26T07:05:59.5789585Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5790173Z "size": 86016, 2024-06-26T07:05:59.5790715Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-06-26T07:05:59.5791404Z + exit 0 2024-06-26T07:05:59.5791662Z }, 2024-06-26T07:05:59.5791910Z { 2024-06-26T07:05:59.5792342Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5792873Z "size": 1823, 2024-06-26T07:05:59.5793390Z "digest": "sha256:309b872da805fcdf74137a887464876fa0615dc8b24199307119eb65fc2bfb49" 2024-06-26T07:05:59.5793976Z }, 2024-06-26T07:05:59.5794221Z { 2024-06-26T07:05:59.5794632Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5795175Z "size": 246737542, 2024-06-26T07:05:59.5795712Z "digest": "sha256:e82932cb002c06a782077b8e5965f8733eb8563406a23cab260d9d8c4a994f7d" 2024-06-26T07:05:59.5796307Z }, 2024-06-26T07:05:59.5796548Z { 2024-06-26T07:05:59.5796968Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5797501Z "size": 545, 2024-06-26T07:05:59.5798028Z "digest": "sha256:ca9dd73e3e9378225ebdcfd12199ef44649e34fe55003cfbc8afe2e8a4527c5b" 2024-06-26T07:05:59.5798633Z }, 2024-06-26T07:05:59.5798876Z { 2024-06-26T07:05:59.5799299Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5799826Z "size": 1283, 2024-06-26T07:05:59.5800339Z "digest": "sha256:16d76a79d5d6616e228537785ec83a869f68a402520bbc6e826201621ca6cb98" 2024-06-26T07:05:59.5800921Z }, 2024-06-26T07:05:59.5801168Z { 2024-06-26T07:05:59.5801581Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5802117Z "size": 485, 2024-06-26T07:05:59.5802927Z "digest": "sha256:4bdf41c9ecbe1651e0986ccbea4663a0556af27b7145a679cfa45c35dc96b891" 2024-06-26T07:05:59.5803532Z }, 2024-06-26T07:05:59.5803783Z { 2024-06-26T07:05:59.5804203Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5804742Z "size": 91710937, 2024-06-26T07:05:59.5805292Z "digest": "sha256:f4863801e30cdbcd1d4e6cb1cbbcde7dbcaf6ae317ca80429295741ce60ac31d" 2024-06-26T07:05:59.5805901Z }, 2024-06-26T07:05:59.5806144Z { 2024-06-26T07:05:59.5806569Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5807197Z "size": 3233, 2024-06-26T07:05:59.5807736Z "digest": "sha256:d37dd9ed8948a0d02e1198bbedadbca36477a0980dd553196385aab0c025581d" 2024-06-26T07:05:59.5808339Z }, 2024-06-26T07:05:59.5808576Z { 2024-06-26T07:05:59.5808996Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5809535Z "size": 1905, 2024-06-26T07:05:59.5810072Z "digest": "sha256:cf2f34f225dbed17c1a9a8b9ea4ad030aba797774a716dc0a82fe212324aa751" 2024-06-26T07:05:59.5810677Z }, 2024-06-26T07:05:59.5810919Z { 2024-06-26T07:05:59.5811342Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5811879Z "size": 702, 2024-06-26T07:05:59.5812401Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:05:59.5813003Z }, 2024-06-26T07:05:59.5813247Z { 2024-06-26T07:05:59.5813668Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5814202Z "size": 2784789002, 2024-06-26T07:05:59.5814753Z "digest": "sha256:3d914950f8addfc4d657a2365c76e1143eb80b710cd3f6e9ae05640217fc6347" 2024-06-26T07:05:59.5815345Z }, 2024-06-26T07:05:59.5815582Z { 2024-06-26T07:05:59.5816002Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5816536Z "size": 381, 2024-06-26T07:05:59.5817060Z "digest": "sha256:96d0e2032184c57e62a652aaffcf0241f0d0c7a23e894ca0ad192c4a64a36583" 2024-06-26T07:05:59.5817654Z }, 2024-06-26T07:05:59.5817993Z { 2024-06-26T07:05:59.5818411Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5818945Z "size": 11879, 2024-06-26T07:05:59.5819464Z "digest": "sha256:06e12ce59790cb8b091e5d619c2a62a28329008ee184a8b9a562e3db12471f30" 2024-06-26T07:05:59.5820055Z }, 2024-06-26T07:05:59.5820304Z { 2024-06-26T07:05:59.5820715Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5821246Z "size": 803, 2024-06-26T07:05:59.5821781Z "digest": "sha256:ed712f63cde7070bdec2a6179c4c35e752c6991ddbc69fe9f2724a9903250ceb" 2024-06-26T07:05:59.5822381Z }, 2024-06-26T07:05:59.5822617Z { 2024-06-26T07:05:59.5823037Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5823577Z "size": 106, 2024-06-26T07:05:59.5824184Z "digest": "sha256:a97a71ef8a4e66234dfc2098834f8395ac5630240e60164d1a251fdb0115ba85" 2024-06-26T07:05:59.5824764Z }, 2024-06-26T07:05:59.5825012Z { 2024-06-26T07:05:59.5825439Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5825977Z "size": 503, 2024-06-26T07:05:59.5826494Z "digest": "sha256:25cd0c482ad612f936b35b4f220c580e16be982226aac8801a6a849a6bfdc221" 2024-06-26T07:05:59.5827092Z }, 2024-06-26T07:05:59.5827339Z { 2024-06-26T07:05:59.5827757Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5828295Z "size": 121477177, 2024-06-26T07:05:59.5828849Z "digest": "sha256:9bf94bc456584365fda21d0ed83b7cc690b08a71727b71c4c1e667e15028a386" 2024-06-26T07:05:59.5829448Z }, 2024-06-26T07:05:59.5829690Z { 2024-06-26T07:05:59.5830111Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5830653Z "size": 109, 2024-06-26T07:05:59.5831176Z "digest": "sha256:de45a14c9b227e4b0c9d45d42223b50a71c03522ddd9e2ca0d9517835206c823" 2024-06-26T07:05:59.5831761Z }, 2024-06-26T07:05:59.5832005Z { 2024-06-26T07:05:59.5832423Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5832959Z "size": 491, 2024-06-26T07:05:59.5833480Z "digest": "sha256:e99241e77209b0b35a0fb59a61c0821fc161190178af7e4af45e6eeaa043375e" 2024-06-26T07:05:59.5834071Z }, 2024-06-26T07:05:59.5834318Z { 2024-06-26T07:05:59.5834733Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5835268Z "size": 291, 2024-06-26T07:05:59.5835804Z "digest": "sha256:35e8f4e2a42f9df0c969dedd2199cf71719cf5ecfa676dfa014bbdeb4fdb295f" 2024-06-26T07:05:59.5836590Z }, 2024-06-26T07:05:59.5836830Z { 2024-06-26T07:05:59.5837254Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5837793Z "size": 101, 2024-06-26T07:05:59.5838317Z "digest": "sha256:a823ffa4c541394b52953391dc0a6867e26c37fe4a71fa022a2cc3f07f0ba70d" 2024-06-26T07:05:59.5838906Z }, 2024-06-26T07:05:59.5839149Z { 2024-06-26T07:05:59.5839568Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5840101Z "size": 1473, 2024-06-26T07:05:59.5840616Z "digest": "sha256:44ba272eae2064603605a6238b3f35cc6578859d660068e702a047a9cd8ab2fd" 2024-06-26T07:05:59.5841200Z }, 2024-06-26T07:05:59.5841444Z { 2024-06-26T07:05:59.5841865Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5842617Z "size": 446710781, 2024-06-26T07:05:59.5843172Z "digest": "sha256:da02d278e5240c285bc855ce4e42cd6121d5b903787f5c14f03e575ad1268650" 2024-06-26T07:05:59.5843771Z }, 2024-06-26T07:05:59.5844014Z { 2024-06-26T07:05:59.5844443Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5844978Z "size": 161, 2024-06-26T07:05:59.5845482Z "digest": "sha256:73a0ce681c295135456e190578733273276b38b4dd5babbb39298f384d43b1c6" 2024-06-26T07:05:59.5846068Z }, 2024-06-26T07:05:59.5846313Z { 2024-06-26T07:05:59.5846731Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5847261Z "size": 564, 2024-06-26T07:05:59.5847900Z "digest": "sha256:49737bd5ae709dbf7956e21fc8b05ba009e9e6c9b130b0126a73bd3d3f5efc78" 2024-06-26T07:05:59.5848499Z }, 2024-06-26T07:05:59.5848747Z { 2024-06-26T07:05:59.5849159Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5849701Z "size": 35874600, 2024-06-26T07:05:59.5850298Z "digest": "sha256:9cd4c745f8e4f9d5c9bb8050923575afad21f5b7852975a4a1981bc9eff282c5" 2024-06-26T07:05:59.5850897Z }, 2024-06-26T07:05:59.5851133Z { 2024-06-26T07:05:59.5851557Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5852089Z "size": 104, 2024-06-26T07:05:59.5852592Z "digest": "sha256:b2950a0365345d4c17a48627c905fd8930b3b3154c727f0526eb9ca547102bb7" 2024-06-26T07:05:59.5853177Z }, 2024-06-26T07:05:59.5853421Z { 2024-06-26T07:05:59.5853848Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5854372Z "size": 425, 2024-06-26T07:05:59.5854883Z "digest": "sha256:25e19d81d9fc081962ff4cbc09e098f4035419822712c19a410f5282300dfb9e" 2024-06-26T07:05:59.5855485Z }, 2024-06-26T07:05:59.5855728Z { 2024-06-26T07:05:59.5856146Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5856684Z "size": 20262084, 2024-06-26T07:05:59.5857236Z "digest": "sha256:60156b40fcbec2c25f0170b0a82fa8cd9303e0daae8bb1811fd0fc6562e88e8e" 2024-06-26T07:05:59.5857837Z }, 2024-06-26T07:05:59.5858077Z { 2024-06-26T07:05:59.5858503Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5859034Z "size": 439, 2024-06-26T07:05:59.5859542Z "digest": "sha256:7a868239227fd19b30a19927bbc8f058d247403fe5b343c55f4b4b8987918911" 2024-06-26T07:05:59.5860124Z }, 2024-06-26T07:05:59.5860366Z { 2024-06-26T07:05:59.5860790Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5861315Z "size": 702, 2024-06-26T07:05:59.5861841Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:05:59.5862447Z }, 2024-06-26T07:05:59.5862689Z { 2024-06-26T07:05:59.5863100Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5863705Z "size": 143, 2024-06-26T07:05:59.5864248Z "digest": "sha256:83911d7c7ecf11b5932bb7449975c52e7f737a5b867691935e1ccb85ceb25b52" 2024-06-26T07:05:59.5864829Z }, 2024-06-26T07:05:59.5865071Z { 2024-06-26T07:05:59.5865492Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5866118Z "size": 135, 2024-06-26T07:05:59.5866655Z "digest": "sha256:b3c5d66c679f58c71febdd502889bd8cd3d36e5001754cdf1e30eabe21135cce" 2024-06-26T07:05:59.5867266Z }, 2024-06-26T07:05:59.5867510Z { 2024-06-26T07:05:59.5867928Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5868461Z "size": 4911495247, 2024-06-26T07:05:59.5869001Z "digest": "sha256:84b2c7a899820f6acdb22148c5e8346d8b26c33269fd01842c9573515251e1e1" 2024-06-26T07:05:59.5869597Z }, 2024-06-26T07:05:59.5869833Z { 2024-06-26T07:05:59.5870253Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5870786Z "size": 190, 2024-06-26T07:05:59.5871325Z "digest": "sha256:c1068d03a811d8dead5da9e82fdc8869205fc4ee9cb92d4f7a4acfbb5ba83950" 2024-06-26T07:05:59.5871926Z }, 2024-06-26T07:05:59.5872168Z { 2024-06-26T07:05:59.5872586Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5873118Z "size": 565, 2024-06-26T07:05:59.5873640Z "digest": "sha256:671b4a301d81074fca17d36851f81801552c4ecb2f90ff3412aa9ab6a1a91371" 2024-06-26T07:05:59.5874230Z }, 2024-06-26T07:05:59.5874471Z { 2024-06-26T07:05:59.5874892Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5875420Z "size": 43163095, 2024-06-26T07:05:59.5875964Z "digest": "sha256:67760cf5ec46ab58d3ca98637fd9b26cb47a67446bfd0db176fe5e023606aa0d" 2024-06-26T07:05:59.5876566Z }, 2024-06-26T07:05:59.5876869Z { 2024-06-26T07:05:59.5877290Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5877822Z "size": 106, 2024-06-26T07:05:59.5878346Z "digest": "sha256:ddc9740bcea2b301e99b50525059aea8d9c7121a1c8eab566561a8968438a8c2" 2024-06-26T07:05:59.5878940Z }, 2024-06-26T07:05:59.5879188Z { 2024-06-26T07:05:59.5879605Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5880171Z "size": 1213, 2024-06-26T07:05:59.5880691Z "digest": "sha256:fb2e977685609fcc01de170ba7f50a573e73e6520e47253717e92a28952c5ee8" 2024-06-26T07:05:59.5881279Z }, 2024-06-26T07:05:59.5881525Z { 2024-06-26T07:05:59.5881954Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5882675Z "size": 702, 2024-06-26T07:05:59.5883207Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:05:59.5883808Z }, 2024-06-26T07:05:59.5884050Z { 2024-06-26T07:05:59.5884478Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5885010Z "size": 138, 2024-06-26T07:05:59.5885543Z "digest": "sha256:b599c5a8c7cb461cbd7b3c98b99960b8eb04506e05e79db1cc0cfadb14a47e91" 2024-06-26T07:05:59.5886141Z }, 2024-06-26T07:05:59.5886384Z { 2024-06-26T07:05:59.5886804Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5887335Z "size": 120, 2024-06-26T07:05:59.5887865Z "digest": "sha256:e08e88c1366827455ee7ddbf92b6ae615ceb3cbd3fb11cd03ddd31bd3fa94a9c" 2024-06-26T07:05:59.5888469Z }, 2024-06-26T07:05:59.5888714Z { 2024-06-26T07:05:59.5889127Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5889663Z "size": 1904157495, 2024-06-26T07:05:59.5890215Z "digest": "sha256:12ba7431ff38de8cf6af3052a2a52e79f4e712b8c99f5bafde3c20e63b1f570f" 2024-06-26T07:05:59.5890811Z }, 2024-06-26T07:05:59.5891053Z { 2024-06-26T07:05:59.5891478Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5892022Z "size": 172, 2024-06-26T07:05:59.5892549Z "digest": "sha256:cf7c2cb896cba6a594028cc0fd246e127a2f4481d48905a3aa12c5f5872332d4" 2024-06-26T07:05:59.5893142Z }, 2024-06-26T07:05:59.5893384Z { 2024-06-26T07:05:59.5893802Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5894340Z "size": 908, 2024-06-26T07:05:59.5894866Z "digest": "sha256:3a778ddfac2af5b3a11f49ddf2f3f1ad1731f74ba691f434ab69966d1e2a189f" 2024-06-26T07:05:59.5895578Z }, 2024-06-26T07:05:59.5895824Z { 2024-06-26T07:05:59.5896239Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5896775Z "size": 702, 2024-06-26T07:05:59.5897306Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:05:59.5897910Z }, 2024-06-26T07:05:59.5898143Z { 2024-06-26T07:05:59.5898565Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5899102Z "size": 134, 2024-06-26T07:05:59.5899621Z "digest": "sha256:236c8f0245f4b40616a4322eaf1e1b3e178d5990db1fcaba39622ea4e79c06a1" 2024-06-26T07:05:59.5900204Z }, 2024-06-26T07:05:59.5900446Z { 2024-06-26T07:05:59.5900868Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5901395Z "size": 32, 2024-06-26T07:05:59.5901933Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:05:59.5902532Z }, 2024-06-26T07:05:59.5902780Z { 2024-06-26T07:05:59.5903193Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5903806Z "size": 156, 2024-06-26T07:05:59.5904339Z "digest": "sha256:dda1ce302b9a9df0c8f1145d2e3eaa03c96909bb45a792451f1fcb555b06738d" 2024-06-26T07:05:59.5904945Z }, 2024-06-26T07:05:59.5905192Z { 2024-06-26T07:05:59.5905610Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5906145Z "size": 1841, 2024-06-26T07:05:59.5906770Z "digest": "sha256:e797fe86caef19319e03f8dca92cf96838ab3bb3644f4447c6dfc04230118afa" 2024-06-26T07:05:59.5907362Z }, 2024-06-26T07:05:59.5907603Z { 2024-06-26T07:05:59.5908028Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5908560Z "size": 7529784, 2024-06-26T07:05:59.5909097Z "digest": "sha256:967190076fc0ab2bb2fc0d74e58edd3d662a537d702ff3f8b6e876230154a16f" 2024-06-26T07:05:59.5909695Z }, 2024-06-26T07:05:59.5909936Z { 2024-06-26T07:05:59.5910352Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5910888Z "size": 165, 2024-06-26T07:05:59.5911412Z "digest": "sha256:4fb0a83f06855534ef82b80dcf7c685303edb2bcbd446196066dc2ad2dd71281" 2024-06-26T07:05:59.5912005Z }, 2024-06-26T07:05:59.5912240Z { 2024-06-26T07:05:59.5912658Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5913192Z "size": 7944, 2024-06-26T07:05:59.5913721Z "digest": "sha256:68af2ec4e30a33f26947ece9f3a31b7d3b04c2bca5865d4b48618457832a1da2" 2024-06-26T07:05:59.5914325Z }, 2024-06-26T07:05:59.5914568Z { 2024-06-26T07:05:59.5914988Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5915518Z "size": 8070, 2024-06-26T07:05:59.5916041Z "digest": "sha256:cc91933e985a044c6df53caa15488d2285896f5e991f38ec068c253ceec53f14" 2024-06-26T07:05:59.5916635Z }, 2024-06-26T07:05:59.5916879Z { 2024-06-26T07:05:59.5917305Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5917841Z "size": 303, 2024-06-26T07:05:59.5918368Z "digest": "sha256:9f3187916e7dcad33cc572f8f0e949d2f507b887f99cfd6acefd14826b44960d" 2024-06-26T07:05:59.5918970Z }, 2024-06-26T07:05:59.5919205Z { 2024-06-26T07:05:59.5919622Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5920157Z "size": 7629071, 2024-06-26T07:05:59.5920663Z "digest": "sha256:52608399485738d10d779254946c9e1d536010dd45593c69bf1eb9f62f85dc24" 2024-06-26T07:05:59.5921246Z }, 2024-06-26T07:05:59.5921487Z { 2024-06-26T07:05:59.5921911Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5922622Z "size": 108, 2024-06-26T07:05:59.5923149Z "digest": "sha256:05a6f6dcc1c00bac6660923a22adc7d7c34416718f8c89daf9b3351d9365f694" 2024-06-26T07:05:59.5923749Z }, 2024-06-26T07:05:59.5923993Z { 2024-06-26T07:05:59.5924404Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5925040Z "size": 54145775, 2024-06-26T07:05:59.5925592Z "digest": "sha256:6bc9dc8315b09b27c976bfba9700767ba412d72fd0528b18bbe8f8e62005ec22" 2024-06-26T07:05:59.5926189Z }, 2024-06-26T07:05:59.5926424Z { 2024-06-26T07:05:59.5926843Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5927378Z "size": 474, 2024-06-26T07:05:59.5927897Z "digest": "sha256:1b0e27c35f07b3fbf0aeb548d1949bf1d60a420136da21789d94d196c891d981" 2024-06-26T07:05:59.5928500Z }, 2024-06-26T07:05:59.5928742Z { 2024-06-26T07:05:59.5929167Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5929700Z "size": 1374858747, 2024-06-26T07:05:59.5930236Z "digest": "sha256:95237e7311a1c5ce6937db7196c866632f44e8788514a5c0eb4a3f04da4017e6" 2024-06-26T07:05:59.5930821Z }, 2024-06-26T07:05:59.5931065Z { 2024-06-26T07:05:59.5931480Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5932016Z "size": 106, 2024-06-26T07:05:59.5932556Z "digest": "sha256:789eff6c51aada70bf75eb1acafed63c0d839b768638001500616ff6e53b854a" 2024-06-26T07:05:59.5933157Z }, 2024-06-26T07:05:59.5933400Z { 2024-06-26T07:05:59.5933821Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5934357Z "size": 558, 2024-06-26T07:05:59.5934882Z "digest": "sha256:de0f8a8c45c6077b3c5ba3b84d7f07de98353fd618dad9a1e2f368ffe00493c7" 2024-06-26T07:05:59.5935483Z }, 2024-06-26T07:05:59.5935817Z { 2024-06-26T07:05:59.5936238Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5936765Z "size": 46248513, 2024-06-26T07:05:59.5937300Z "digest": "sha256:2060028f74888d218bebdd10d723f66c33ee1c708935c4ffd4654d34912ac4dc" 2024-06-26T07:05:59.5937889Z }, 2024-06-26T07:05:59.5938124Z { 2024-06-26T07:05:59.5938547Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5939078Z "size": 111, 2024-06-26T07:05:59.5939608Z "digest": "sha256:e412c7f3b2ded440992955bd92512345cbece8a8894cd3592f714ebc758ad1a0" 2024-06-26T07:05:59.5940197Z }, 2024-06-26T07:05:59.5940442Z { 2024-06-26T07:05:59.5940861Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5941394Z "size": 32, 2024-06-26T07:05:59.5941913Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:05:59.5942513Z }, 2024-06-26T07:05:59.5942755Z { 2024-06-26T07:05:59.5943180Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5943785Z "size": 32, 2024-06-26T07:05:59.5944314Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:05:59.5944911Z }, 2024-06-26T07:05:59.5945145Z { 2024-06-26T07:05:59.5945561Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5946107Z "size": 32, 2024-06-26T07:05:59.5946657Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:05:59.5947247Z }, 2024-06-26T07:05:59.5947490Z { 2024-06-26T07:05:59.5947914Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:05:59.5948452Z "size": 32, 2024-06-26T07:05:59.5948966Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:05:59.5949566Z } 2024-06-26T07:05:59.5949809Z ] 2024-06-26T07:05:59.5950046Z } 2024-06-26T07:05:59.6133067Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:05:59.6133776Z tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:05:59.6134382Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-06-26T07:05:59.6142642Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:05:59.6143139Z env: 2024-06-26T07:05:59.6143424Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:59.6144927Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:59.6146387Z ##[endgroup] 2024-06-26T07:05:59.6170925Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:59.6218748Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-06-26T07:05:59.6219302Z with: 2024-06-26T07:05:59.6220344Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:59.6221633Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:05:59.6222135Z env: 2024-06-26T07:05:59.6222404Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:59.6222758Z ##[endgroup] 2024-06-26T07:05:59.6238040Z ##[group]Run set -x 2024-06-26T07:05:59.6238380Z set -x 2024-06-26T07:05:59.6238727Z set +e 2024-06-26T07:05:59.6239076Z  2024-06-26T07:05:59.6239349Z login() { 2024-06-26T07:05:59.6240004Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:05:59.6240698Z } 2024-06-26T07:05:59.6240971Z  2024-06-26T07:05:59.6241271Z retry () { 2024-06-26T07:05:59.6241657Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:05:59.6242090Z } 2024-06-26T07:05:59.6242575Z  2024-06-26T07:05:59.6242888Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:05:59.6243411Z  2024-06-26T07:05:59.6243683Z set -e 2024-06-26T07:05:59.6244145Z # ignore output since only exit code is used for conditional 2024-06-26T07:05:59.6244826Z # only pull docker image if it's not available locally 2024-06-26T07:05:59.6245566Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-06-26T07:05:59.6246242Z  retry docker pull "${DOCKER_IMAGE}" 2024-06-26T07:05:59.6246675Z fi 2024-06-26T07:05:59.6253475Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:05:59.6253953Z env: 2024-06-26T07:05:59.6254230Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:59.6255344Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:59.6256606Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:05:59.6257118Z ##[endgroup] 2024-06-26T07:05:59.6275304Z + set +e 2024-06-26T07:05:59.6275989Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:05:59.6276634Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:05:59.6277479Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:05:59.6278697Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:06:00.0314181Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:06:00.0314993Z Configure a credential helper to remove this warning. See 2024-06-26T07:06:00.0315955Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:06:00.0316496Z 2024-06-26T07:06:00.0316621Z Login Succeeded 2024-06-26T07:06:00.0324913Z + set -e 2024-06-26T07:06:00.0326198Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:06:00.0450920Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:06:00.0453002Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:06:00.2627365Z 91382da70d5719cd7007b6b80b71d2f48398f6b7: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-06-26T07:06:00.2628618Z 7a2c55901189: Pulling fs layer 2024-06-26T07:06:00.2629121Z 224fe954d725: Pulling fs layer 2024-06-26T07:06:00.2629817Z 75722010b82e: Pulling fs layer 2024-06-26T07:06:00.2630309Z d527cbbb87e3: Pulling fs layer 2024-06-26T07:06:00.2630777Z b57676e46aee: Pulling fs layer 2024-06-26T07:06:00.2631313Z a8c1e85b5e14: Pulling fs layer 2024-06-26T07:06:00.2631792Z a41a8d1c11c8: Pulling fs layer 2024-06-26T07:06:00.2632230Z 0c1227890755: Pulling fs layer 2024-06-26T07:06:00.2632681Z d8d1234baab3: Pulling fs layer 2024-06-26T07:06:00.2633198Z 7ed32bc8e469: Pulling fs layer 2024-06-26T07:06:00.2634817Z ec1e7978c1fe: Pulling fs layer 2024-06-26T07:06:00.2635399Z 309b872da805: Pulling fs layer 2024-06-26T07:06:00.2635948Z e82932cb002c: Pulling fs layer 2024-06-26T07:06:00.2636436Z ca9dd73e3e93: Pulling fs layer 2024-06-26T07:06:00.2636811Z 16d76a79d5d6: Pulling fs layer 2024-06-26T07:06:00.2637275Z 4bdf41c9ecbe: Pulling fs layer 2024-06-26T07:06:00.2637798Z f4863801e30c: Pulling fs layer 2024-06-26T07:06:00.2638333Z d37dd9ed8948: Pulling fs layer 2024-06-26T07:06:00.2638838Z cf2f34f225db: Pulling fs layer 2024-06-26T07:06:00.2639353Z df546d7b3e34: Pulling fs layer 2024-06-26T07:06:00.2639945Z 3d914950f8ad: Pulling fs layer 2024-06-26T07:06:00.2640444Z 96d0e2032184: Pulling fs layer 2024-06-26T07:06:00.2641012Z 06e12ce59790: Pulling fs layer 2024-06-26T07:06:00.2641519Z ed712f63cde7: Pulling fs layer 2024-06-26T07:06:00.2642227Z a97a71ef8a4e: Pulling fs layer 2024-06-26T07:06:00.2642957Z a8c1e85b5e14: Waiting 2024-06-26T07:06:00.2643380Z 25cd0c482ad6: Pulling fs layer 2024-06-26T07:06:00.2643866Z a41a8d1c11c8: Waiting 2024-06-26T07:06:00.2644307Z 9bf94bc45658: Pulling fs layer 2024-06-26T07:06:00.2644757Z de45a14c9b22: Pulling fs layer 2024-06-26T07:06:00.2645126Z e99241e77209: Pulling fs layer 2024-06-26T07:06:00.2645486Z 35e8f4e2a42f: Pulling fs layer 2024-06-26T07:06:00.2645946Z a823ffa4c541: Pulling fs layer 2024-06-26T07:06:00.2646378Z 44ba272eae20: Pulling fs layer 2024-06-26T07:06:00.2646783Z da02d278e524: Pulling fs layer 2024-06-26T07:06:00.2647148Z 73a0ce681c29: Pulling fs layer 2024-06-26T07:06:00.2647507Z 0c1227890755: Waiting 2024-06-26T07:06:00.2647831Z 49737bd5ae70: Pulling fs layer 2024-06-26T07:06:00.2648246Z 9cd4c745f8e4: Pulling fs layer 2024-06-26T07:06:00.2648612Z b2950a036534: Pulling fs layer 2024-06-26T07:06:00.2649080Z d8d1234baab3: Waiting 2024-06-26T07:06:00.2649535Z 25e19d81d9fc: Pulling fs layer 2024-06-26T07:06:00.2649966Z 60156b40fcbe: Pulling fs layer 2024-06-26T07:06:00.2650346Z 7a868239227f: Pulling fs layer 2024-06-26T07:06:00.2650827Z 7ed32bc8e469: Waiting 2024-06-26T07:06:00.2651241Z d527cbbb87e3: Waiting 2024-06-26T07:06:00.2651597Z 83911d7c7ecf: Pulling fs layer 2024-06-26T07:06:00.2652108Z b3c5d66c679f: Pulling fs layer 2024-06-26T07:06:00.2652566Z 84b2c7a89982: Pulling fs layer 2024-06-26T07:06:00.2652915Z ec1e7978c1fe: Waiting 2024-06-26T07:06:00.2653232Z c1068d03a811: Pulling fs layer 2024-06-26T07:06:00.2653659Z 309b872da805: Waiting 2024-06-26T07:06:00.2653977Z 16d76a79d5d6: Waiting 2024-06-26T07:06:00.2654312Z e82932cb002c: Waiting 2024-06-26T07:06:00.2654625Z 671b4a301d81: Pulling fs layer 2024-06-26T07:06:00.2654987Z 4bdf41c9ecbe: Waiting 2024-06-26T07:06:00.2655300Z ca9dd73e3e93: Waiting 2024-06-26T07:06:00.2655620Z 67760cf5ec46: Pulling fs layer 2024-06-26T07:06:00.2655972Z ed712f63cde7: Waiting 2024-06-26T07:06:00.2656301Z ddc9740bcea2: Pulling fs layer 2024-06-26T07:06:00.2656689Z f4863801e30c: Waiting 2024-06-26T07:06:00.2657031Z fb2e97768560: Pulling fs layer 2024-06-26T07:06:00.2657384Z b57676e46aee: Waiting 2024-06-26T07:06:00.2657710Z b599c5a8c7cb: Pulling fs layer 2024-06-26T07:06:00.2658062Z a823ffa4c541: Waiting 2024-06-26T07:06:00.2658365Z d37dd9ed8948: Waiting 2024-06-26T07:06:00.2658684Z e08e88c13668: Pulling fs layer 2024-06-26T07:06:00.2659034Z 3d914950f8ad: Waiting 2024-06-26T07:06:00.2659353Z 12ba7431ff38: Pulling fs layer 2024-06-26T07:06:00.2659705Z 44ba272eae20: Waiting 2024-06-26T07:06:00.2660039Z cf2f34f225db: Waiting 2024-06-26T07:06:00.2660374Z 96d0e2032184: Waiting 2024-06-26T07:06:00.2660696Z cf7c2cb896cb: Pulling fs layer 2024-06-26T07:06:00.2661177Z a97a71ef8a4e: Waiting 2024-06-26T07:06:00.2661504Z de45a14c9b22: Waiting 2024-06-26T07:06:00.2661831Z 3a778ddfac2a: Pulling fs layer 2024-06-26T07:06:00.2662174Z 83911d7c7ecf: Waiting 2024-06-26T07:06:00.2662480Z b3c5d66c679f: Waiting 2024-06-26T07:06:00.2662789Z 25cd0c482ad6: Waiting 2024-06-26T07:06:00.2663112Z 236c8f0245f4: Pulling fs layer 2024-06-26T07:06:00.2663466Z e99241e77209: Waiting 2024-06-26T07:06:00.2663901Z da02d278e524: Waiting 2024-06-26T07:06:00.2664228Z 4f4fb700ef54: Pulling fs layer 2024-06-26T07:06:00.2664578Z 9bf94bc45658: Waiting 2024-06-26T07:06:00.2664884Z 35e8f4e2a42f: Waiting 2024-06-26T07:06:00.2665216Z dda1ce302b9a: Pulling fs layer 2024-06-26T07:06:00.2665567Z df546d7b3e34: Waiting 2024-06-26T07:06:00.2665897Z e797fe86caef: Pulling fs layer 2024-06-26T07:06:00.2666249Z 25e19d81d9fc: Waiting 2024-06-26T07:06:00.2666571Z 967190076fc0: Pulling fs layer 2024-06-26T07:06:00.2666930Z 4fb0a83f0685: Pulling fs layer 2024-06-26T07:06:00.2667277Z e08e88c13668: Waiting 2024-06-26T07:06:00.2667587Z b2950a036534: Waiting 2024-06-26T07:06:00.2667888Z 49737bd5ae70: Waiting 2024-06-26T07:06:00.2668186Z 9cd4c745f8e4: Waiting 2024-06-26T07:06:00.2668505Z 68af2ec4e30a: Pulling fs layer 2024-06-26T07:06:00.2668853Z 84b2c7a89982: Waiting 2024-06-26T07:06:00.2669232Z 06e12ce59790: Waiting 2024-06-26T07:06:00.2669541Z cf7c2cb896cb: Waiting 2024-06-26T07:06:00.2673697Z cc91933e985a: Pulling fs layer 2024-06-26T07:06:00.2674218Z 671b4a301d81: Waiting 2024-06-26T07:06:00.2674650Z 67760cf5ec46: Waiting 2024-06-26T07:06:00.2675050Z 60156b40fcbe: Waiting 2024-06-26T07:06:00.2675458Z 3a778ddfac2a: Waiting 2024-06-26T07:06:00.2675944Z ddc9740bcea2: Waiting 2024-06-26T07:06:00.2676281Z 9f3187916e7d: Pulling fs layer 2024-06-26T07:06:00.2676682Z 526083994857: Pulling fs layer 2024-06-26T07:06:00.2677062Z b599c5a8c7cb: Waiting 2024-06-26T07:06:00.2677406Z fb2e97768560: Waiting 2024-06-26T07:06:00.2677708Z 967190076fc0: Waiting 2024-06-26T07:06:00.2678034Z 05a6f6dcc1c0: Pulling fs layer 2024-06-26T07:06:00.2678398Z 4f4fb700ef54: Waiting 2024-06-26T07:06:00.2678714Z e797fe86caef: Waiting 2024-06-26T07:06:00.2679037Z 6bc9dc8315b0: Pulling fs layer 2024-06-26T07:06:00.2679394Z 4fb0a83f0685: Waiting 2024-06-26T07:06:00.2679703Z 526083994857: Waiting 2024-06-26T07:06:00.2680020Z 1b0e27c35f07: Pulling fs layer 2024-06-26T07:06:00.2680386Z dda1ce302b9a: Waiting 2024-06-26T07:06:00.2680713Z 95237e7311a1: Pulling fs layer 2024-06-26T07:06:00.2681068Z 6bc9dc8315b0: Waiting 2024-06-26T07:06:00.2681390Z 789eff6c51aa: Pulling fs layer 2024-06-26T07:06:00.2681745Z 1b0e27c35f07: Waiting 2024-06-26T07:06:00.2682082Z de0f8a8c45c6: Pulling fs layer 2024-06-26T07:06:00.2682792Z c1068d03a811: Waiting 2024-06-26T07:06:00.2683120Z 2060028f7488: Pulling fs layer 2024-06-26T07:06:00.2683492Z e412c7f3b2de: Pulling fs layer 2024-06-26T07:06:00.2683848Z 95237e7311a1: Waiting 2024-06-26T07:06:00.2684161Z 789eff6c51aa: Waiting 2024-06-26T07:06:00.2684476Z de0f8a8c45c6: Waiting 2024-06-26T07:06:00.2684787Z 2060028f7488: Waiting 2024-06-26T07:06:00.2685114Z e412c7f3b2de: Waiting 2024-06-26T07:06:00.4371808Z 224fe954d725: Verifying Checksum 2024-06-26T07:06:00.4372241Z 224fe954d725: Download complete 2024-06-26T07:06:00.5057874Z d527cbbb87e3: Verifying Checksum 2024-06-26T07:06:00.5058338Z d527cbbb87e3: Download complete 2024-06-26T07:06:00.5898808Z b57676e46aee: Download complete 2024-06-26T07:06:00.6199466Z 7a2c55901189: Verifying Checksum 2024-06-26T07:06:00.6199997Z 7a2c55901189: Download complete 2024-06-26T07:06:00.7026306Z a41a8d1c11c8: Verifying Checksum 2024-06-26T07:06:00.7026934Z a41a8d1c11c8: Download complete 2024-06-26T07:06:00.7772754Z 0c1227890755: Verifying Checksum 2024-06-26T07:06:00.8541051Z d8d1234baab3: Verifying Checksum 2024-06-26T07:06:00.8541661Z d8d1234baab3: Download complete 2024-06-26T07:06:00.9007121Z 75722010b82e: Verifying Checksum 2024-06-26T07:06:00.9007586Z 75722010b82e: Download complete 2024-06-26T07:06:00.9715132Z ec1e7978c1fe: Download complete 2024-06-26T07:06:01.0471773Z 309b872da805: Download complete 2024-06-26T07:06:01.6529475Z 7a2c55901189: Pull complete 2024-06-26T07:06:01.9390158Z 224fe954d725: Pull complete 2024-06-26T07:06:02.9080603Z 75722010b82e: Pull complete 2024-06-26T07:06:02.9696414Z d527cbbb87e3: Pull complete 2024-06-26T07:06:03.0246957Z b57676e46aee: Pull complete 2024-06-26T07:06:03.5692427Z e82932cb002c: Verifying Checksum 2024-06-26T07:06:03.5693087Z e82932cb002c: Download complete 2024-06-26T07:06:03.6547211Z ca9dd73e3e93: Verifying Checksum 2024-06-26T07:06:03.6547757Z ca9dd73e3e93: Download complete 2024-06-26T07:06:03.7353501Z 16d76a79d5d6: Download complete 2024-06-26T07:06:03.8342502Z 4bdf41c9ecbe: Verifying Checksum 2024-06-26T07:06:03.8343777Z 4bdf41c9ecbe: Download complete 2024-06-26T07:06:04.8212756Z f4863801e30c: Verifying Checksum 2024-06-26T07:06:04.8213241Z f4863801e30c: Download complete 2024-06-26T07:06:04.9074391Z d37dd9ed8948: Verifying Checksum 2024-06-26T07:06:04.9075033Z d37dd9ed8948: Download complete 2024-06-26T07:06:04.9965916Z cf2f34f225db: Verifying Checksum 2024-06-26T07:06:04.9966809Z cf2f34f225db: Download complete 2024-06-26T07:06:05.1188679Z df546d7b3e34: Verifying Checksum 2024-06-26T07:06:05.1189458Z df546d7b3e34: Download complete 2024-06-26T07:06:13.9735755Z a8c1e85b5e14: Verifying Checksum 2024-06-26T07:06:13.9736404Z a8c1e85b5e14: Download complete 2024-06-26T07:06:14.0639350Z 96d0e2032184: Verifying Checksum 2024-06-26T07:06:14.0639939Z 96d0e2032184: Download complete 2024-06-26T07:06:14.1446697Z 06e12ce59790: Verifying Checksum 2024-06-26T07:06:14.1447232Z 06e12ce59790: Download complete 2024-06-26T07:06:14.2235021Z ed712f63cde7: Download complete 2024-06-26T07:06:14.3007562Z a97a71ef8a4e: Verifying Checksum 2024-06-26T07:06:14.3008228Z a97a71ef8a4e: Download complete 2024-06-26T07:06:14.3781366Z 25cd0c482ad6: Verifying Checksum 2024-06-26T07:06:14.3781876Z 25cd0c482ad6: Download complete 2024-06-26T07:06:15.6435055Z 9bf94bc45658: Verifying Checksum 2024-06-26T07:06:15.6435709Z 9bf94bc45658: Download complete 2024-06-26T07:06:15.7622303Z de45a14c9b22: Download complete 2024-06-26T07:06:15.8361292Z e99241e77209: Verifying Checksum 2024-06-26T07:06:15.8361956Z e99241e77209: Download complete 2024-06-26T07:06:15.9133212Z 35e8f4e2a42f: Verifying Checksum 2024-06-26T07:06:15.9133829Z 35e8f4e2a42f: Download complete 2024-06-26T07:06:16.0079733Z a823ffa4c541: Verifying Checksum 2024-06-26T07:06:16.0080375Z a823ffa4c541: Download complete 2024-06-26T07:06:16.0910293Z 44ba272eae20: Download complete 2024-06-26T07:06:20.6172916Z da02d278e524: Verifying Checksum 2024-06-26T07:06:20.6173381Z da02d278e524: Download complete 2024-06-26T07:06:20.6859418Z 73a0ce681c29: Download complete 2024-06-26T07:06:20.7721782Z 49737bd5ae70: Download complete 2024-06-26T07:06:21.1941809Z 9cd4c745f8e4: Verifying Checksum 2024-06-26T07:06:21.1942384Z 9cd4c745f8e4: Download complete 2024-06-26T07:06:21.2887399Z b2950a036534: Verifying Checksum 2024-06-26T07:06:21.2887950Z b2950a036534: Download complete 2024-06-26T07:06:21.3716307Z 25e19d81d9fc: Verifying Checksum 2024-06-26T07:06:21.3716944Z 25e19d81d9fc: Download complete 2024-06-26T07:06:21.7177782Z 60156b40fcbe: Download complete 2024-06-26T07:06:21.8079981Z 7a868239227f: Download complete 2024-06-26T07:06:21.8810600Z 83911d7c7ecf: Download complete 2024-06-26T07:06:21.9635962Z b3c5d66c679f: Verifying Checksum 2024-06-26T07:06:21.9636504Z b3c5d66c679f: Download complete 2024-06-26T07:06:26.2465381Z 7ed32bc8e469: Verifying Checksum 2024-06-26T07:06:26.2466028Z 7ed32bc8e469: Download complete 2024-06-26T07:06:26.3408433Z c1068d03a811: Verifying Checksum 2024-06-26T07:06:26.3409102Z c1068d03a811: Download complete 2024-06-26T07:06:26.4366715Z 671b4a301d81: Download complete 2024-06-26T07:06:26.6173661Z a8c1e85b5e14: Pull complete 2024-06-26T07:06:26.7624603Z a41a8d1c11c8: Pull complete 2024-06-26T07:06:26.9499127Z 67760cf5ec46: Verifying Checksum 2024-06-26T07:06:26.9499591Z 67760cf5ec46: Download complete 2024-06-26T07:06:26.9820155Z 0c1227890755: Pull complete 2024-06-26T07:06:27.0290298Z ddc9740bcea2: Verifying Checksum 2024-06-26T07:06:27.0290926Z ddc9740bcea2: Download complete 2024-06-26T07:06:27.1238696Z fb2e97768560: Verifying Checksum 2024-06-26T07:06:27.1239308Z fb2e97768560: Download complete 2024-06-26T07:06:27.2086870Z d8d1234baab3: Pull complete 2024-06-26T07:06:27.2212963Z b599c5a8c7cb: Download complete 2024-06-26T07:06:27.2932949Z e08e88c13668: Download complete 2024-06-26T07:06:33.0289703Z 3d914950f8ad: Verifying Checksum 2024-06-26T07:06:33.0290239Z 3d914950f8ad: Download complete 2024-06-26T07:06:33.1054071Z cf7c2cb896cb: Download complete 2024-06-26T07:06:33.1828702Z 3a778ddfac2a: Verifying Checksum 2024-06-26T07:06:33.1829134Z 3a778ddfac2a: Download complete 2024-06-26T07:06:33.2598247Z 236c8f0245f4: Download complete 2024-06-26T07:06:33.2717133Z 4f4fb700ef54: Verifying Checksum 2024-06-26T07:06:33.2717577Z 4f4fb700ef54: Download complete 2024-06-26T07:06:33.3463746Z dda1ce302b9a: Verifying Checksum 2024-06-26T07:06:33.3464226Z dda1ce302b9a: Download complete 2024-06-26T07:06:33.4264852Z e797fe86caef: Download complete 2024-06-26T07:06:33.5980148Z 967190076fc0: Verifying Checksum 2024-06-26T07:06:33.5980613Z 967190076fc0: Download complete 2024-06-26T07:06:33.6685536Z 4fb0a83f0685: Verifying Checksum 2024-06-26T07:06:33.6686081Z 4fb0a83f0685: Download complete 2024-06-26T07:06:33.7455165Z 68af2ec4e30a: Download complete 2024-06-26T07:06:33.8298291Z cc91933e985a: Download complete 2024-06-26T07:06:33.9088204Z 9f3187916e7d: Verifying Checksum 2024-06-26T07:06:33.9088650Z 9f3187916e7d: Download complete 2024-06-26T07:06:34.0506054Z 526083994857: Verifying Checksum 2024-06-26T07:06:34.0506611Z 526083994857: Download complete 2024-06-26T07:06:34.1559287Z 05a6f6dcc1c0: Verifying Checksum 2024-06-26T07:06:34.1559776Z 05a6f6dcc1c0: Download complete 2024-06-26T07:06:34.7654680Z 6bc9dc8315b0: Verifying Checksum 2024-06-26T07:06:34.7655234Z 6bc9dc8315b0: Download complete 2024-06-26T07:06:34.8477484Z 1b0e27c35f07: Verifying Checksum 2024-06-26T07:06:34.8477945Z 1b0e27c35f07: Download complete 2024-06-26T07:06:57.6309344Z 12ba7431ff38: Verifying Checksum 2024-06-26T07:06:57.6310985Z 12ba7431ff38: Download complete 2024-06-26T07:06:57.7151667Z 789eff6c51aa: Verifying Checksum 2024-06-26T07:06:57.7152166Z 789eff6c51aa: Download complete 2024-06-26T07:06:57.7960071Z de0f8a8c45c6: Verifying Checksum 2024-06-26T07:06:57.7960559Z de0f8a8c45c6: Download complete 2024-06-26T07:06:59.1509560Z 2060028f7488: Verifying Checksum 2024-06-26T07:06:59.1510015Z 2060028f7488: Download complete 2024-06-26T07:06:59.2376592Z e412c7f3b2de: Download complete 2024-06-26T07:07:02.7257718Z 95237e7311a1: Verifying Checksum 2024-06-26T07:07:02.7258165Z 95237e7311a1: Download complete 2024-06-26T07:07:25.7299980Z 84b2c7a89982: Verifying Checksum 2024-06-26T07:07:25.7300483Z 84b2c7a89982: Download complete 2024-06-26T07:07:42.9458914Z 7ed32bc8e469: Pull complete 2024-06-26T07:07:43.1579876Z ec1e7978c1fe: Pull complete 2024-06-26T07:07:43.3694879Z 309b872da805: Pull complete 2024-06-26T07:07:50.7549130Z e82932cb002c: Pull complete 2024-06-26T07:07:50.9790285Z ca9dd73e3e93: Pull complete 2024-06-26T07:07:51.1949309Z 16d76a79d5d6: Pull complete 2024-06-26T07:07:51.4077398Z 4bdf41c9ecbe: Pull complete 2024-06-26T07:07:53.7714610Z f4863801e30c: Pull complete 2024-06-26T07:07:53.9951175Z d37dd9ed8948: Pull complete 2024-06-26T07:07:54.2098701Z cf2f34f225db: Pull complete 2024-06-26T07:07:54.4293139Z df546d7b3e34: Pull complete 2024-06-26T07:08:43.8350832Z 3d914950f8ad: Pull complete 2024-06-26T07:08:43.9951418Z 96d0e2032184: Pull complete 2024-06-26T07:08:44.1850450Z 06e12ce59790: Pull complete 2024-06-26T07:08:44.3095701Z ed712f63cde7: Pull complete 2024-06-26T07:08:44.3593318Z a97a71ef8a4e: Pull complete 2024-06-26T07:08:44.4471278Z 25cd0c482ad6: Pull complete 2024-06-26T07:08:47.7902128Z 9bf94bc45658: Pull complete 2024-06-26T07:08:47.8851516Z de45a14c9b22: Pull complete 2024-06-26T07:08:47.9810026Z e99241e77209: Pull complete 2024-06-26T07:08:48.1164345Z 35e8f4e2a42f: Pull complete 2024-06-26T07:08:48.2561281Z a823ffa4c541: Pull complete 2024-06-26T07:08:48.4886067Z 44ba272eae20: Pull complete 2024-06-26T07:08:56.7685330Z da02d278e524: Pull complete 2024-06-26T07:08:56.9768946Z 73a0ce681c29: Pull complete 2024-06-26T07:08:57.2077155Z 49737bd5ae70: Pull complete 2024-06-26T07:08:58.1868770Z 9cd4c745f8e4: Pull complete 2024-06-26T07:08:58.3698166Z b2950a036534: Pull complete 2024-06-26T07:08:58.5302431Z 25e19d81d9fc: Pull complete 2024-06-26T07:08:59.0033084Z 60156b40fcbe: Pull complete 2024-06-26T07:08:59.1759864Z 7a868239227f: Pull complete 2024-06-26T07:08:59.5463908Z 83911d7c7ecf: Pull complete 2024-06-26T07:08:59.7404510Z b3c5d66c679f: Pull complete 2024-06-26T07:09:38.7197780Z 84b2c7a89982: Pull complete 2024-06-26T07:09:38.9195776Z c1068d03a811: Pull complete 2024-06-26T07:09:39.1401242Z 671b4a301d81: Pull complete 2024-06-26T07:09:41.1206410Z 67760cf5ec46: Pull complete 2024-06-26T07:09:41.1658180Z ddc9740bcea2: Pull complete 2024-06-26T07:09:41.2156972Z fb2e97768560: Pull complete 2024-06-26T07:09:41.2936423Z b599c5a8c7cb: Pull complete 2024-06-26T07:09:41.3301556Z e08e88c13668: Pull complete 2024-06-26T07:10:15.7233686Z 12ba7431ff38: Pull complete 2024-06-26T07:10:15.9144921Z cf7c2cb896cb: Pull complete 2024-06-26T07:10:16.1213840Z 3a778ddfac2a: Pull complete 2024-06-26T07:10:16.5707623Z 236c8f0245f4: Pull complete 2024-06-26T07:10:16.7835568Z 4f4fb700ef54: Pull complete 2024-06-26T07:10:16.9951850Z dda1ce302b9a: Pull complete 2024-06-26T07:10:17.2183256Z e797fe86caef: Pull complete 2024-06-26T07:10:17.5820834Z 967190076fc0: Pull complete 2024-06-26T07:10:17.7997222Z 4fb0a83f0685: Pull complete 2024-06-26T07:10:18.0246908Z 68af2ec4e30a: Pull complete 2024-06-26T07:10:18.2226887Z cc91933e985a: Pull complete 2024-06-26T07:10:18.4369628Z 9f3187916e7d: Pull complete 2024-06-26T07:10:19.5234802Z 526083994857: Pull complete 2024-06-26T07:10:19.7474049Z 05a6f6dcc1c0: Pull complete 2024-06-26T07:10:21.7870718Z 6bc9dc8315b0: Pull complete 2024-06-26T07:10:22.0012622Z 1b0e27c35f07: Pull complete 2024-06-26T07:10:35.5296206Z 95237e7311a1: Pull complete 2024-06-26T07:10:35.5947951Z 789eff6c51aa: Pull complete 2024-06-26T07:10:35.6383025Z de0f8a8c45c6: Pull complete 2024-06-26T07:10:36.3512730Z 2060028f7488: Pull complete 2024-06-26T07:10:36.3977906Z e412c7f3b2de: Pull complete 2024-06-26T07:10:36.6756185Z Digest: sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb 2024-06-26T07:10:36.6806180Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:10:36.6837243Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:10:36.6877103Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:10:36.6878011Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:10:36.6886417Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:10:36.6886901Z env: 2024-06-26T07:10:36.6887164Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:36.6887509Z ##[endgroup] 2024-06-26T07:10:36.7036699Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-06-26T07:10:36.7037252Z with: 2024-06-26T07:10:36.7037541Z driver-version: 550.54.15 2024-06-26T07:10:36.7037875Z env: 2024-06-26T07:10:36.7038149Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:36.7038490Z ##[endgroup] 2024-06-26T07:10:36.7077249Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:10:36.7077803Z with: 2024-06-26T07:10:36.7078082Z timeout_minutes: 10 2024-06-26T07:10:36.7078403Z max_attempts: 3 2024-06-26T07:10:36.7108909Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" 2024-06-26T07:10:36.7139743Z retry_wait_seconds: 10 2024-06-26T07:10:36.7140109Z polling_interval_seconds: 1 2024-06-26T07:10:36.7140478Z warning_on_retry: true 2024-06-26T07:10:36.7140827Z continue_on_error: false 2024-06-26T07:10:36.7141160Z env: 2024-06-26T07:10:36.7141436Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:36.7141795Z DRIVER_VERSION: 550.54.15 2024-06-26T07:10:36.7142143Z ##[endgroup] 2024-06-26T07:10:36.7663301Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-06-26T07:10:36.7664023Z + pre_install_nvidia_driver_amzn2 2024-06-26T07:10:36.7664633Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-06-26T07:10:37.0647139Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:10:37.1031533Z No Match for argument: nvidia-driver-latest-dkms 2024-06-26T07:10:37.1270688Z No Packages marked for removal 2024-06-26T07:10:37.1393760Z + install_nvidia_driver_common 2024-06-26T07:10:37.1396159Z + echo 'Before installing NVIDIA driver' 2024-06-26T07:10:37.1396587Z + lspci 2024-06-26T07:10:37.1397011Z Before installing NVIDIA driver 2024-06-26T07:10:37.1475264Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-06-26T07:10:37.1476057Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T07:10:37.1477004Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-06-26T07:10:37.1477958Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-06-26T07:10:37.1478987Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 2024-06-26T07:10:37.1479851Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T07:10:37.1480761Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1) 2024-06-26T07:10:37.1481552Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-06-26T07:10:37.1482134Z + lsmod 2024-06-26T07:10:37.1488388Z Module Size Used by 2024-06-26T07:10:37.1489034Z nvidia_modeset 1339392 0 2024-06-26T07:10:37.1489578Z veth 16384 0 2024-06-26T07:10:37.1490090Z nvidia_uvm 4599808 0 2024-06-26T07:10:37.1490694Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:10:37.1491282Z drm 421888 1 nvidia 2024-06-26T07:10:37.1492145Z i2c_core 77824 2 nvidia,drm 2024-06-26T07:10:37.1492835Z backlight 16384 1 nvidia_modeset 2024-06-26T07:10:37.1493445Z xt_conntrack 16384 1 2024-06-26T07:10:37.1493952Z ipt_MASQUERADE 16384 1 2024-06-26T07:10:37.1494527Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T07:10:37.1495116Z nf_conntrack_netlink 49152 0 2024-06-26T07:10:37.1495605Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T07:10:37.1496195Z xfrm_user 45056 1 2024-06-26T07:10:37.1496711Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:10:37.1497213Z iptable_nat 16384 1 2024-06-26T07:10:37.1497666Z nf_conntrack_ipv4 16384 3 2024-06-26T07:10:37.1498228Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T07:10:37.1498889Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T07:10:37.1499576Z nf_nat 32768 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 2024-06-26T07:10:37.1500681Z nf_conntrack 155648 7 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink 2024-06-26T07:10:37.1501565Z xt_addrtype 16384 2 2024-06-26T07:10:37.1501934Z iptable_filter 16384 1 2024-06-26T07:10:37.1502388Z br_netfilter 24576 0 2024-06-26T07:10:37.1502800Z bridge 172032 1 br_netfilter 2024-06-26T07:10:37.1503229Z stp 16384 1 bridge 2024-06-26T07:10:37.1503717Z llc 16384 2 bridge,stp 2024-06-26T07:10:37.1504276Z overlay 86016 0 2024-06-26T07:10:37.1504643Z sunrpc 393216 1 2024-06-26T07:10:37.1505001Z dm_mirror 28672 0 2024-06-26T07:10:37.1505390Z dm_region_hash 20480 1 dm_mirror 2024-06-26T07:10:37.1505866Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T07:10:37.1506365Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T07:10:37.1506794Z dax 69632 1 dm_mod 2024-06-26T07:10:37.1507208Z crc32_pclmul 16384 0 2024-06-26T07:10:37.1507588Z ghash_clmulni_intel 16384 0 2024-06-26T07:10:37.1508125Z pcbc 16384 0 2024-06-26T07:10:37.1508486Z aesni_intel 188416 0 2024-06-26T07:10:37.1508875Z aes_x86_64 20480 1 aesni_intel 2024-06-26T07:10:37.1509313Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:10:37.1509755Z glue_helper 16384 1 aesni_intel 2024-06-26T07:10:37.1510402Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T07:10:37.1510982Z mousedev 24576 0 2024-06-26T07:10:37.1511398Z evdev 20480 3 2024-06-26T07:10:37.1511760Z psmouse 32768 0 2024-06-26T07:10:37.1512123Z button 16384 0 2024-06-26T07:10:37.1512486Z ena 139264 0 2024-06-26T07:10:37.1512851Z ptp 20480 1 ena 2024-06-26T07:10:37.1513240Z pps_core 20480 1 ptp 2024-06-26T07:10:37.1513634Z crc32c_intel 24576 0 2024-06-26T07:10:37.1513999Z autofs4 49152 2 2024-06-26T07:10:37.1514355Z + modinfo nvidia 2024-06-26T07:10:37.1515009Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T07:10:37.1515689Z alias: char-major-195-* 2024-06-26T07:10:37.1516063Z version: 550.54.15 2024-06-26T07:10:37.1516393Z supported: external 2024-06-26T07:10:37.1516737Z license: NVIDIA 2024-06-26T07:10:37.1517109Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:10:37.1517584Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:10:37.1518032Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T07:10:37.1518488Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:10:37.1518976Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:10:37.1519456Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:10:37.1519923Z depends: i2c-core,drm 2024-06-26T07:10:37.1520358Z retpoline: Y 2024-06-26T07:10:37.1520667Z name: nvidia 2024-06-26T07:10:37.1521212Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T07:10:37.1521865Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:10:37.1522778Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:10:37.1523373Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:10:37.1523806Z parm: NVreg_RmLogonRC:int 2024-06-26T07:10:37.1524238Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:10:37.1524695Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:10:37.1525128Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:10:37.1525556Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:10:37.1526085Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:10:37.1526644Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:10:37.1527118Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:10:37.1527542Z parm: NVreg_EnableMSI:int 2024-06-26T07:10:37.1527956Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:10:37.1528417Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:10:37.1528940Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:10:37.1529508Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:10:37.1530060Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:10:37.1530666Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:10:37.1531262Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:10:37.1531861Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:10:37.1532451Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:10:37.1532939Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:10:37.1533471Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:10:37.1534012Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:10:37.1534506Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:10:37.1535069Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:10:37.1535546Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:10:37.1536015Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:10:37.1536459Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:10:37.1536958Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:10:37.1537475Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:10:37.1537954Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:10:37.1538439Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:10:37.1538932Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:10:37.1539406Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:10:37.1539904Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:10:37.1540381Z parm: NVreg_RmMsg:charp 2024-06-26T07:10:37.1540842Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:10:37.1541310Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:10:37.1541785Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:10:37.1542242Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:10:37.1542720Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:10:37.1543190Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:10:37.1543644Z parm: rm_firmware_active:charp 2024-06-26T07:10:37.1544133Z + HAS_NVIDIA_DRIVER=0 2024-06-26T07:10:37.1544514Z ++ command -v nvidia-smi 2024-06-26T07:10:37.1544906Z + '[' -x /usr/bin/nvidia-smi ']' 2024-06-26T07:10:37.1545274Z + set +e 2024-06-26T07:10:37.1545776Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-06-26T07:10:39.1500839Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-06-26T07:10:39.1501431Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:10:39.1502033Z + '[' 0 -ne 0 ']' 2024-06-26T07:10:39.1502504Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-06-26T07:10:39.1503346Z + HAS_NVIDIA_DRIVER=1 2024-06-26T07:10:39.1504411Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-06-26T07:10:39.1505378Z + set -e 2024-06-26T07:10:39.1505698Z + '[' 1 -eq 0 ']' 2024-06-26T07:10:39.1506269Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-06-26T07:10:39.1506952Z + post_install_nvidia_driver_common 2024-06-26T07:10:39.1507357Z + sudo modprobe nvidia 2024-06-26T07:10:39.1589874Z + echo 'After installing NVIDIA driver' 2024-06-26T07:10:39.1590351Z + lspci 2024-06-26T07:10:39.1590652Z After installing NVIDIA driver 2024-06-26T07:10:39.1668527Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-06-26T07:10:39.1669506Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T07:10:39.1670441Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-06-26T07:10:39.1671250Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-06-26T07:10:39.1671994Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 2024-06-26T07:10:39.1672749Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T07:10:39.1673461Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1) 2024-06-26T07:10:39.1674236Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-06-26T07:10:39.1674819Z + lsmod 2024-06-26T07:10:39.1682788Z Module Size Used by 2024-06-26T07:10:39.1683359Z nvidia_modeset 1339392 0 2024-06-26T07:10:39.1683872Z veth 16384 0 2024-06-26T07:10:39.1684376Z nvidia_uvm 4599808 0 2024-06-26T07:10:39.1684958Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:10:39.1685543Z drm 421888 1 nvidia 2024-06-26T07:10:39.1686034Z i2c_core 77824 2 nvidia,drm 2024-06-26T07:10:39.1686506Z backlight 16384 1 nvidia_modeset 2024-06-26T07:10:39.1686944Z xt_conntrack 16384 1 2024-06-26T07:10:39.1687326Z ipt_MASQUERADE 16384 1 2024-06-26T07:10:39.1687945Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T07:10:39.1688409Z nf_conntrack_netlink 49152 0 2024-06-26T07:10:39.1688843Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T07:10:39.1689297Z xfrm_user 45056 1 2024-06-26T07:10:39.1689686Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:10:39.1690090Z iptable_nat 16384 1 2024-06-26T07:10:39.1690464Z nf_conntrack_ipv4 16384 3 2024-06-26T07:10:39.1690906Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T07:10:39.1691544Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T07:10:39.1692244Z nf_nat 32768 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 2024-06-26T07:10:39.1693455Z nf_conntrack 155648 7 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink 2024-06-26T07:10:39.1694430Z xt_addrtype 16384 2 2024-06-26T07:10:39.1694812Z iptable_filter 16384 1 2024-06-26T07:10:39.1695198Z br_netfilter 24576 0 2024-06-26T07:10:39.1695595Z bridge 172032 1 br_netfilter 2024-06-26T07:10:39.1696034Z stp 16384 1 bridge 2024-06-26T07:10:39.1696449Z llc 16384 2 bridge,stp 2024-06-26T07:10:39.1696870Z overlay 86016 0 2024-06-26T07:10:39.1697231Z sunrpc 393216 1 2024-06-26T07:10:39.1697600Z dm_mirror 28672 0 2024-06-26T07:10:39.1697990Z dm_region_hash 20480 1 dm_mirror 2024-06-26T07:10:39.1698465Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T07:10:39.1699086Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T07:10:39.1699541Z dax 69632 1 dm_mod 2024-06-26T07:10:39.1699983Z crc32_pclmul 16384 0 2024-06-26T07:10:39.1700350Z ghash_clmulni_intel 16384 0 2024-06-26T07:10:39.1700862Z pcbc 16384 0 2024-06-26T07:10:39.1701236Z aesni_intel 188416 0 2024-06-26T07:10:39.1701623Z aes_x86_64 20480 1 aesni_intel 2024-06-26T07:10:39.1702066Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:10:39.1702518Z glue_helper 16384 1 aesni_intel 2024-06-26T07:10:39.1703080Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T07:10:39.1703613Z mousedev 24576 0 2024-06-26T07:10:39.1703976Z evdev 20480 3 2024-06-26T07:10:39.1704420Z psmouse 32768 0 2024-06-26T07:10:39.1704784Z button 16384 0 2024-06-26T07:10:39.1705140Z ena 139264 0 2024-06-26T07:10:39.1705509Z ptp 20480 1 ena 2024-06-26T07:10:39.1705906Z pps_core 20480 1 ptp 2024-06-26T07:10:39.1706298Z crc32c_intel 24576 0 2024-06-26T07:10:39.1706660Z autofs4 49152 2 2024-06-26T07:10:39.1707021Z + modinfo nvidia 2024-06-26T07:10:39.1707685Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T07:10:39.1708406Z alias: char-major-195-* 2024-06-26T07:10:39.1708820Z version: 550.54.15 2024-06-26T07:10:39.1709198Z supported: external 2024-06-26T07:10:39.1709536Z license: NVIDIA 2024-06-26T07:10:39.1709909Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:10:39.1710376Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:10:39.1710833Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T07:10:39.1711297Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:10:39.1711791Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:10:39.1712270Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:10:39.1712757Z depends: i2c-core,drm 2024-06-26T07:10:39.1713117Z retpoline: Y 2024-06-26T07:10:39.1713432Z name: nvidia 2024-06-26T07:10:39.1713979Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T07:10:39.1714634Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:10:39.1715351Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:10:39.1715939Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:10:39.1716378Z parm: NVreg_RmLogonRC:int 2024-06-26T07:10:39.1716812Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:10:39.1717266Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:10:39.1717697Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:10:39.1718164Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:10:39.1718691Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:10:39.1719248Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:10:39.1719725Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:10:39.1720148Z parm: NVreg_EnableMSI:int 2024-06-26T07:10:39.1720568Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:10:39.1721032Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:10:39.1721615Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:10:39.1722194Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:10:39.1723060Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:10:39.1723672Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:10:39.1724266Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:10:39.1724865Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:10:39.1725452Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:10:39.1725938Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:10:39.1726467Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:10:39.1727010Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:10:39.1727508Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:10:39.1728066Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:10:39.1728545Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:10:39.1729014Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:10:39.1729468Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:10:39.1729967Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:10:39.1730483Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:10:39.1730957Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:10:39.1731442Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:10:39.1731942Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:10:39.1732420Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:10:39.1732913Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:10:39.1733402Z parm: NVreg_RmMsg:charp 2024-06-26T07:10:39.1733812Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:10:39.1734269Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:10:39.1734750Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:10:39.1735215Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:10:39.1735680Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:10:39.1736158Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:10:39.1736613Z parm: rm_firmware_active:charp 2024-06-26T07:10:39.1737010Z + set +e 2024-06-26T07:10:39.1737309Z + nvidia-smi 2024-06-26T07:10:40.7440482Z Wed Jun 26 07:10:40 2024 2024-06-26T07:10:40.7441363Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:10:40.7442527Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-06-26T07:10:40.7443325Z |-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:10:40.7444195Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-06-26T07:10:40.7445108Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-06-26T07:10:40.7445790Z | | | MIG M. | 2024-06-26T07:10:40.7446600Z |=========================================+========================+======================| 2024-06-26T07:10:40.7569256Z | 0 NVIDIA A10G Off | 00000000:00:1E.0 Off | 0 | 2024-06-26T07:10:40.7570949Z | 0% 32C P0 56W / 300W | 0MiB / 23028MiB | 0% Default | 2024-06-26T07:10:40.7572136Z | | | N/A | 2024-06-26T07:10:40.7572923Z +-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:10:40.7573515Z 2024-06-26T07:10:40.7574169Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:10:40.7574818Z | Processes: | 2024-06-26T07:10:40.7575538Z | GPU GI CI PID Type Process name GPU Memory | 2024-06-26T07:10:40.7576224Z | ID ID Usage | 2024-06-26T07:10:40.7576792Z |=========================================================================================| 2024-06-26T07:10:40.7577460Z | No running processes found | 2024-06-26T07:10:40.7578213Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:10:41.3837224Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-06-26T07:10:42.9477794Z NVIDIA A10G 2024-06-26T07:10:43.4039185Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:10:43.4039859Z + '[' 0 -eq 0 ']' 2024-06-26T07:10:43.4040301Z + echo 'INFO: Ignoring allowed status 0' 2024-06-26T07:10:43.4041010Z + set -e 2024-06-26T07:10:43.4041317Z INFO: Ignoring allowed status 0 2024-06-26T07:10:43.4043060Z == Installing nvidia container toolkit for amzn2 == 2024-06-26T07:10:43.4046181Z + sudo yum install -y yum-utils 2024-06-26T07:10:43.7106934Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:10:45.1918895Z Package yum-utils-1.1.31-46.amzn2.0.1.noarch already installed and latest version 2024-06-26T07:10:45.1919722Z Nothing to do 2024-06-26T07:10:45.3416740Z + [[ amzn2 == \a\m\z\n\2\0\2\3 ]] 2024-06-26T07:10:45.3417644Z + YUM_REPO_URL=https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:10:45.3418713Z + sudo yum-config-manager --add-repo https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:10:45.6420798Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:10:45.6673047Z adding repo from: https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:10:45.6674227Z grabbing file https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T07:10:45.6675182Z repo saved to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T07:10:45.6794150Z + sudo yum install -y nvidia-docker2 2024-06-26T07:10:45.9627852Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:10:47.4150825Z Package nvidia-docker2-2.13.0-1.noarch already installed and latest version 2024-06-26T07:10:47.4151485Z Nothing to do 2024-06-26T07:10:47.5637422Z + sudo systemctl restart docker 2024-06-26T07:11:26.8121721Z Command completed after 1 attempt(s). 2024-06-26T07:11:26.8180075Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:11:26.8180830Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:11:26.8181502Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-06-26T07:11:26.8182144Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:11:26.8190029Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:26.8190517Z env: 2024-06-26T07:11:26.8190935Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:26.8191386Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:26.8191861Z ##[endgroup] 2024-06-26T07:11:27.0369467Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T07:11:27.0518062Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.7/site-packages (5.9.1) 2024-06-26T07:11:27.0616290Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.7/site-packages (11.525.84) 2024-06-26T07:11:27.1520134Z Prepare all required actions 2024-06-26T07:11:27.1520632Z Getting action download info 2024-06-26T07:11:27.2514497Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-06-26T07:11:27.4107220Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-06-26T07:11:27.5355627Z ##[group]Run ./.github/actions/download-build-artifacts 2024-06-26T07:11:27.5356117Z with: 2024-06-26T07:11:27.5356454Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:11:27.5356905Z s3-bucket: gha-artifacts 2024-06-26T07:11:27.5357237Z env: 2024-06-26T07:11:27.5357511Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:27.5357954Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:27.5358429Z ##[endgroup] 2024-06-26T07:11:27.5389746Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:11:27.5390194Z with: 2024-06-26T07:11:27.5390565Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:11:27.5391014Z s3-bucket: gha-artifacts 2024-06-26T07:11:27.5391369Z region: us-east-1 2024-06-26T07:11:27.5391673Z env: 2024-06-26T07:11:27.5391942Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:27.5392390Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:27.5392868Z ##[endgroup] 2024-06-26T07:11:27.9494292Z (node:26399) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:11:27.9495006Z 2024-06-26T07:11:27.9495319Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:11:27.9496035Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:11:27.9497025Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:11:28.0513469Z Found 1 objects with prefix pytorch/pytorch/9673646806/linux-focal-cuda12.1-py3.10-gcc9-sm86/ 2024-06-26T07:11:28.0514555Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:11:35.5492730Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:11:35.5499027Z Artifact download has finished successfully 2024-06-26T07:11:35.5651487Z ##[group]Run unzip -o artifacts.zip 2024-06-26T07:11:35.5651934Z unzip -o artifacts.zip 2024-06-26T07:11:35.5659898Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:35.5660413Z env: 2024-06-26T07:11:35.5660699Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:35.5661155Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:35.5661632Z ##[endgroup] 2024-06-26T07:11:35.5689819Z Archive: artifacts.zip 2024-06-26T07:11:35.5691597Z creating: dist/ 2024-06-26T07:11:37.5433775Z inflating: dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:11:37.5434656Z creating: build/custom_test_artifacts/ 2024-06-26T07:11:37.5435318Z creating: build/custom_test_artifacts/custom-op-build/ 2024-06-26T07:11:37.5436045Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-06-26T07:11:37.5436857Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:11:37.5441373Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:11:37.5442551Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-06-26T07:11:37.5443692Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:11:37.5444656Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:11:37.5445602Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:11:37.5446670Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:11:37.5447993Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:11:37.5448959Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:11:37.5449925Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:11:37.5451045Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:11:37.5452171Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:11:37.5453281Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:11:37.5454383Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:11:37.5455507Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:11:37.5456643Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:11:37.5457648Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:11:37.5458626Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:11:37.5496133Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:11:37.5534418Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:11:37.5536492Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:11:37.5579495Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:11:37.5581568Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:11:37.5583624Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:11:37.5585855Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:11:37.5587913Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:11:37.5589919Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:11:37.5592029Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:11:37.5593986Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:11:37.5595907Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:11:37.5597735Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:11:37.5599491Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:11:37.5601176Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:11:37.5603422Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:11:37.5605133Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:11:37.5606832Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:11:37.5652490Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:11:37.5710799Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:11:37.5712524Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:11:37.5713958Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:11:37.5715205Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:11:37.5716487Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-06-26T07:11:37.5717805Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-06-26T07:11:37.5719290Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-06-26T07:11:37.5720957Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-06-26T07:11:37.5722768Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-06-26T07:11:37.5724200Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-06-26T07:11:37.5725695Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-06-26T07:11:37.5727184Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-06-26T07:11:37.5728717Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-06-26T07:11:37.5730196Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-06-26T07:11:37.5731709Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-06-26T07:11:37.5740855Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-06-26T07:11:37.5868841Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-06-26T07:11:37.5870287Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-06-26T07:11:37.5871854Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-06-26T07:11:37.5873601Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-06-26T07:11:37.5875284Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-06-26T07:11:37.5876809Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-06-26T07:11:37.5878424Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-06-26T07:11:37.5880001Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-06-26T07:11:37.5881668Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-06-26T07:11:37.5883743Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-06-26T07:11:37.5884883Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-06-26T07:11:37.5895532Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-06-26T07:11:37.5974596Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-06-26T07:11:37.5975894Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:11:37.5977061Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:11:37.5978131Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-06-26T07:11:37.5979113Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-06-26T07:11:37.5980111Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-06-26T07:11:37.5980993Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-06-26T07:11:37.5981825Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-06-26T07:11:37.5982595Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-06-26T07:11:37.5983384Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-06-26T07:11:37.6088563Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-06-26T07:11:37.6148395Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-06-26T07:11:37.6149284Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-06-26T07:11:37.6149978Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-06-26T07:11:37.6150771Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:11:37.6155609Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:11:37.6156635Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-06-26T07:11:37.6157540Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:11:37.6158754Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:11:37.6159985Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:11:37.6161056Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:11:37.6162617Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:11:37.6164281Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:11:37.6165276Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:11:37.6166404Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:11:37.6167531Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:11:37.6168635Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:11:37.6169730Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:11:37.6170856Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:11:37.6171970Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:11:37.6172974Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:11:37.6173945Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:11:37.6207935Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:11:37.6244593Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:11:37.6246198Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:11:37.6288403Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:11:37.6289772Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:11:37.6291363Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:11:37.6292789Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:11:37.6294138Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:11:37.6295500Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:11:37.6296844Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:11:37.6298181Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:11:37.6299495Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:11:37.6300787Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:11:37.6301971Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:11:37.6303148Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:11:37.6304447Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:11:37.6305603Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:11:37.6306775Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:11:37.6354730Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:11:37.6412456Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:11:37.6413611Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:11:37.6414571Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:11:37.6415398Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:11:37.6416270Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-06-26T07:11:37.6417185Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-06-26T07:11:37.6418226Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-06-26T07:11:37.6419377Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-06-26T07:11:37.6420488Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-06-26T07:11:37.6421528Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-06-26T07:11:37.6422600Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-06-26T07:11:37.6423674Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-06-26T07:11:37.6424939Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-06-26T07:11:37.6426003Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-06-26T07:11:37.6445753Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-06-26T07:11:37.6447001Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-06-26T07:11:37.6498444Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-06-26T07:11:37.6499615Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:11:37.6500664Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:11:37.6501604Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-06-26T07:11:37.6502480Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-06-26T07:11:37.6503361Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-06-26T07:11:37.6504284Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-06-26T07:11:37.6505110Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-06-26T07:11:37.6505874Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-06-26T07:11:37.6506659Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-06-26T07:11:37.6551756Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-06-26T07:11:37.6552502Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-06-26T07:11:37.6553239Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-06-26T07:11:37.6554095Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:11:37.6559003Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:11:37.6559974Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-06-26T07:11:37.6560946Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:11:37.6561959Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:11:37.6563080Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:11:37.6564205Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:11:37.6565325Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:11:37.6566342Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:11:37.6567357Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:11:37.6568528Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:11:37.6569703Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:11:37.6570866Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:11:37.6572052Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:11:37.6573233Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:11:37.6574403Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:11:37.6575465Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:11:37.6576618Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:11:37.6610368Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:11:37.6648051Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:11:37.6649616Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:11:37.6692032Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:11:37.6693471Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:11:37.6694923Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:11:37.6696405Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:11:37.6697824Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:11:37.6712820Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:11:37.6714245Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:11:37.6715648Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:11:37.6717022Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:11:37.6718329Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:11:37.6719571Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:11:37.6720816Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:11:37.6722070Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:11:37.6723420Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:11:37.6724669Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:11:37.6776796Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:11:37.6834797Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:11:37.6836015Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:11:37.6837040Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:11:37.6837933Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:11:37.6838864Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-06-26T07:11:37.6839849Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-06-26T07:11:37.6841018Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-06-26T07:11:37.6842365Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-06-26T07:11:37.6843668Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-06-26T07:11:37.6844777Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-06-26T07:11:37.6845922Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-06-26T07:11:37.6847142Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-06-26T07:11:37.6848303Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-06-26T07:11:37.6849454Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-06-26T07:11:37.6850628Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-06-26T07:11:37.6851872Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-06-26T07:11:37.6968437Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-06-26T07:11:37.6970796Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-06-26T07:11:37.6971971Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-06-26T07:11:37.6973272Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-06-26T07:11:37.6974510Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-06-26T07:11:37.6975687Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-06-26T07:11:37.6976898Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-06-26T07:11:37.6978120Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-06-26T07:11:37.6979350Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-06-26T07:11:37.6980570Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-06-26T07:11:37.6981779Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-06-26T07:11:37.6991806Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-06-26T07:11:37.7045661Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-06-26T07:11:37.7048371Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:11:37.7050449Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:11:37.7051472Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-06-26T07:11:37.7052434Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-06-26T07:11:37.7053380Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-06-26T07:11:37.7054335Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-06-26T07:11:37.7055243Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-06-26T07:11:37.7056086Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-06-26T07:11:37.7056949Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-06-26T07:11:37.7150303Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-06-26T07:11:37.7191841Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-06-26T07:11:37.7192514Z creating: build/lib/ 2024-06-26T07:11:37.7277150Z inflating: build/lib/libprotobuf-lite.a 2024-06-26T07:11:37.7715258Z inflating: build/lib/libprotobuf.a 2024-06-26T07:11:37.7725406Z inflating: build/lib/libpthreadpool.a 2024-06-26T07:11:37.7733278Z inflating: build/lib/libcpuinfo.a 2024-06-26T07:11:37.7741095Z inflating: build/lib/libcpuinfo_internals.a 2024-06-26T07:11:37.7741776Z inflating: build/lib/libclog.a 2024-06-26T07:11:37.7759395Z inflating: build/lib/libnnpack.a 2024-06-26T07:11:37.7761535Z inflating: build/lib/libnnpack_reference_layers.a 2024-06-26T07:11:37.7824123Z inflating: build/lib/libgtest.a 2024-06-26T07:11:37.7895995Z inflating: build/lib/libbenchmark.a 2024-06-26T07:11:37.7957744Z inflating: build/lib/libasmjit.a 2024-06-26T07:11:37.7965263Z inflating: build/lib/libittnotify.a 2024-06-26T07:11:37.7991983Z inflating: build/lib/libtensorpipe_uv.a 2024-06-26T07:11:37.8114090Z inflating: build/lib/libgloo.a 2024-06-26T07:11:37.8135281Z inflating: build/lib/libfmt.a 2024-06-26T07:11:37.8224203Z inflating: build/lib/libc10.so 2024-06-26T07:11:37.8225896Z inflating: build/lib/libcaffe2_nvrtc.so 2024-06-26T07:11:37.8226617Z inflating: build/lib/libfoxi_loader.a 2024-06-26T07:11:37.8228638Z inflating: build/lib/libtorch_global_deps.so 2024-06-26T07:11:37.8247694Z inflating: build/lib/libpytorch_qnnpack.a 2024-06-26T07:11:37.8265320Z inflating: build/lib/libgmock.a 2024-06-26T07:11:37.8265974Z inflating: build/lib/libgtest_main.a 2024-06-26T07:11:37.8266660Z inflating: build/lib/libbenchmark_main.a 2024-06-26T07:11:37.8752195Z inflating: build/lib/libprotoc.a 2024-06-26T07:11:37.9123088Z inflating: build/lib/libgloo_cuda.a 2024-06-26T07:11:38.8834074Z inflating: build/lib/libdnnl.a 2024-06-26T07:11:38.9383124Z inflating: build/lib/libtensorpipe.a 2024-06-26T07:11:38.9436152Z inflating: build/lib/libc10_cuda.so 2024-06-26T07:11:38.9436813Z inflating: build/lib/libgmock_main.a 2024-06-26T07:11:39.0638616Z inflating: build/lib/libfbgemm.a 2024-06-26T07:11:39.0884023Z inflating: build/lib/libtensorpipe_cuda.a 2024-06-26T07:11:39.1381527Z inflating: build/lib/libkineto.a 2024-06-26T07:11:39.1559581Z inflating: build/lib/libXNNPACK.a 2024-06-26T07:11:39.1600517Z inflating: build/lib/libonnx_proto.a 2024-06-26T07:11:39.2267214Z inflating: build/lib/libonnx.a 2024-06-26T07:11:41.5521487Z inflating: build/lib/libtorch_cpu.so 2024-06-26T07:11:41.5525776Z inflating: build/lib/libunbox_lib.a 2024-06-26T07:11:41.5529754Z inflating: build/lib/libshm.so 2024-06-26T07:11:43.5361130Z inflating: build/lib/libtorch_cuda.so 2024-06-26T07:11:43.5361933Z inflating: build/lib/libtorch.so 2024-06-26T07:11:44.3298063Z inflating: build/lib/libtorch_cuda_linalg.so 2024-06-26T07:11:44.3300046Z inflating: build/lib/libc10d_cuda_test.so 2024-06-26T07:11:44.5194499Z inflating: build/lib/libtorch_python.so 2024-06-26T07:11:44.5260536Z inflating: build/lib/libtorchbind_test.so 2024-06-26T07:11:44.5280634Z inflating: build/lib/libjitbackend_test.so 2024-06-26T07:11:44.5305518Z inflating: build/lib/libbackend_with_compiler.so 2024-06-26T07:11:44.5339082Z inflating: build/lib/libnnapi_backend.so 2024-06-26T07:11:44.5339681Z creating: build/bin/ 2024-06-26T07:11:44.5388413Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-06-26T07:11:44.5438289Z inflating: build/bin/c10_DeviceGuard_test 2024-06-26T07:11:44.5487435Z inflating: build/bin/c10_Device_test 2024-06-26T07:11:44.5544853Z inflating: build/bin/c10_DispatchKeySet_test 2024-06-26T07:11:44.5596964Z inflating: build/bin/c10_Scalar_test 2024-06-26T07:11:44.5645082Z inflating: build/bin/c10_StreamGuard_test 2024-06-26T07:11:44.5697564Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-06-26T07:11:44.5747499Z inflating: build/bin/c10_SymInt_test 2024-06-26T07:11:44.5801106Z inflating: build/bin/c10_InlineStreamGuard_test 2024-06-26T07:11:44.5855725Z inflating: build/bin/c10_SizesAndStrides_test 2024-06-26T07:11:44.5907129Z inflating: build/bin/c10_Bitset_test 2024-06-26T07:11:44.5976630Z inflating: build/bin/c10_cow_test 2024-06-26T07:11:44.6023879Z inflating: build/bin/c10_ConstexprCrc_test 2024-06-26T07:11:44.6073091Z inflating: build/bin/c10_DeadlockDetection_test 2024-06-26T07:11:44.6121706Z inflating: build/bin/c10_Half_test 2024-06-26T07:11:44.6176441Z inflating: build/bin/c10_LeftRight_test 2024-06-26T07:11:44.6229916Z inflating: build/bin/c10_Metaprogramming_test 2024-06-26T07:11:44.6278563Z inflating: build/bin/c10_Synchronized_test 2024-06-26T07:11:44.6332612Z inflating: build/bin/c10_ThreadLocal_test 2024-06-26T07:11:44.6382748Z inflating: build/bin/c10_TypeIndex_test 2024-06-26T07:11:44.6432769Z inflating: build/bin/c10_TypeList_test 2024-06-26T07:11:44.6480078Z inflating: build/bin/c10_TypeTraits_test 2024-06-26T07:11:44.6530871Z inflating: build/bin/c10_accumulate_test 2024-06-26T07:11:44.6584278Z inflating: build/bin/c10_bfloat16_test 2024-06-26T07:11:44.6633269Z inflating: build/bin/c10_bit_cast_test 2024-06-26T07:11:44.6688140Z inflating: build/bin/c10_complex_math_test 2024-06-26T07:11:44.6742177Z inflating: build/bin/c10_complex_test 2024-06-26T07:11:44.6793694Z inflating: build/bin/c10_exception_test 2024-06-26T07:11:44.6842535Z inflating: build/bin/c10_flags_test 2024-06-26T07:11:44.6891895Z inflating: build/bin/c10_generic_math_test 2024-06-26T07:11:44.7050318Z inflating: build/bin/c10_intrusive_ptr_test 2024-06-26T07:11:44.7099780Z inflating: build/bin/c10_irange_test 2024-06-26T07:11:44.7151908Z inflating: build/bin/c10_lazy_test 2024-06-26T07:11:44.7207462Z inflating: build/bin/c10_logging_test 2024-06-26T07:11:44.7279938Z inflating: build/bin/c10_optional_test 2024-06-26T07:11:44.7340428Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-06-26T07:11:44.7392681Z inflating: build/bin/c10_registry_test 2024-06-26T07:11:44.7538388Z inflating: build/bin/c10_small_vector_test 2024-06-26T07:11:44.7589215Z inflating: build/bin/c10_ssize_test 2024-06-26T07:11:44.7639755Z inflating: build/bin/c10_string_util_test 2024-06-26T07:11:44.7696700Z inflating: build/bin/c10_string_view_test 2024-06-26T07:11:44.7745760Z inflating: build/bin/c10_tempfile_test 2024-06-26T07:11:44.7792916Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-06-26T07:11:44.7847773Z inflating: build/bin/c10_typeid_test 2024-06-26T07:11:44.8280788Z inflating: build/bin/protoc-3.13.0.0 2024-06-26T07:11:44.8714132Z inflating: build/bin/protoc 2024-06-26T07:11:44.8766070Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-06-26T07:11:44.8817889Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-06-26T07:11:44.8869715Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-06-26T07:11:44.8920442Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-06-26T07:11:44.8972231Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-06-26T07:11:44.9024265Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-06-26T07:11:44.9071981Z inflating: build/bin/c10_cuda_CUDATest 2024-06-26T07:11:44.9123560Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-06-26T07:11:44.9441755Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-06-26T07:11:44.9774773Z inflating: build/bin/vec_test_all_types_AVX512 2024-06-26T07:11:45.0120733Z inflating: build/bin/vec_test_all_types_AVX2 2024-06-26T07:11:45.0171987Z inflating: build/bin/FileStoreTest 2024-06-26T07:11:45.0226001Z inflating: build/bin/TCPStoreTest 2024-06-26T07:11:45.0277754Z inflating: build/bin/HashStoreTest 2024-06-26T07:11:45.0291008Z inflating: build/bin/ProcessGroupMPITest 2024-06-26T07:11:45.0343305Z inflating: build/bin/test_edge_op_registration 2024-06-26T07:11:45.0347557Z inflating: build/bin/torch_shm_manager 2024-06-26T07:11:45.0350368Z inflating: build/bin/example_allreduce 2024-06-26T07:11:45.0403856Z inflating: build/bin/test_dist_autograd 2024-06-26T07:11:45.0472000Z inflating: build/bin/test_cpp_rpc 2024-06-26T07:11:45.0473962Z inflating: build/bin/parallel_benchmark 2024-06-26T07:11:45.0539091Z inflating: build/bin/test_mobile_nnc 2024-06-26T07:11:45.0548034Z inflating: build/bin/aot_model_compiler_test 2024-06-26T07:11:45.0881717Z inflating: build/bin/test_lazy 2024-06-26T07:11:45.2026442Z inflating: build/bin/test_api 2024-06-26T07:11:45.2097415Z inflating: build/bin/Dict_test 2024-06-26T07:11:45.2148174Z inflating: build/bin/Dimname_test 2024-06-26T07:11:45.2211032Z inflating: build/bin/MaybeOwned_test 2024-06-26T07:11:45.2267114Z inflating: build/bin/NamedTensor_test 2024-06-26T07:11:45.2324256Z inflating: build/bin/apply_utils_test 2024-06-26T07:11:45.2381551Z inflating: build/bin/atest 2024-06-26T07:11:45.2442833Z inflating: build/bin/basic 2024-06-26T07:11:45.2496359Z inflating: build/bin/broadcast_test 2024-06-26T07:11:45.2545766Z inflating: build/bin/cpu_allocator_test 2024-06-26T07:11:45.2602512Z inflating: build/bin/cpu_generator_test 2024-06-26T07:11:45.2654066Z inflating: build/bin/cpu_profiling_allocator_test 2024-06-26T07:11:45.2745305Z inflating: build/bin/cpu_rng_test 2024-06-26T07:11:45.2793660Z inflating: build/bin/dispatch_key_set_test 2024-06-26T07:11:45.2842530Z inflating: build/bin/dlconvertor_test 2024-06-26T07:11:45.2899219Z inflating: build/bin/extension_backend_test 2024-06-26T07:11:45.2952936Z inflating: build/bin/half_test 2024-06-26T07:11:45.3046128Z inflating: build/bin/ivalue_test 2024-06-26T07:11:45.3095225Z inflating: build/bin/lazy_tensor_test 2024-06-26T07:11:45.3148363Z inflating: build/bin/math_kernel_test 2024-06-26T07:11:45.3201399Z inflating: build/bin/memory_format_test 2024-06-26T07:11:45.3253379Z inflating: build/bin/memory_overlapping_test 2024-06-26T07:11:45.3305240Z inflating: build/bin/mobile_memory_cleanup 2024-06-26T07:11:45.3360081Z inflating: build/bin/native_test 2024-06-26T07:11:45.3409588Z inflating: build/bin/operator_name_test 2024-06-26T07:11:45.3459898Z inflating: build/bin/operators_test 2024-06-26T07:11:45.3510914Z inflating: build/bin/packedtensoraccessor_test 2024-06-26T07:11:45.3576617Z inflating: build/bin/pow_test 2024-06-26T07:11:45.3632732Z inflating: build/bin/quantized_test 2024-06-26T07:11:45.3681810Z inflating: build/bin/reduce_ops_test 2024-06-26T07:11:45.3732100Z inflating: build/bin/reportMemoryUsage_test 2024-06-26T07:11:45.3787415Z inflating: build/bin/scalar_tensor_test 2024-06-26T07:11:45.3844984Z inflating: build/bin/scalar_test 2024-06-26T07:11:45.3895670Z inflating: build/bin/StorageUtils_test 2024-06-26T07:11:45.3947160Z inflating: build/bin/stride_properties_test 2024-06-26T07:11:45.4023756Z inflating: build/bin/tensor_iterator_test 2024-06-26T07:11:45.4076836Z inflating: build/bin/test_parallel 2024-06-26T07:11:45.4080000Z inflating: build/bin/thread_init_test 2024-06-26T07:11:45.4134828Z inflating: build/bin/type_ptr_test 2024-06-26T07:11:45.4193233Z inflating: build/bin/type_test 2024-06-26T07:11:45.4244327Z inflating: build/bin/undefined_tensor_test 2024-06-26T07:11:45.4245736Z inflating: build/bin/verify_api_visibility 2024-06-26T07:11:45.4313610Z inflating: build/bin/legacy_vmap_test 2024-06-26T07:11:45.4364244Z inflating: build/bin/weakref_test 2024-06-26T07:11:45.4414411Z inflating: build/bin/wrapdim_test 2024-06-26T07:11:45.4464963Z inflating: build/bin/xla_tensor_test 2024-06-26T07:11:45.4523631Z inflating: build/bin/IListRef_test 2024-06-26T07:11:45.4626907Z inflating: build/bin/List_test 2024-06-26T07:11:45.4691762Z inflating: build/bin/KernelFunction_test 2024-06-26T07:11:45.4810515Z inflating: build/bin/kernel_function_legacy_test 2024-06-26T07:11:45.4904482Z inflating: build/bin/kernel_function_test 2024-06-26T07:11:45.5029313Z inflating: build/bin/kernel_lambda_legacy_test 2024-06-26T07:11:45.5130191Z inflating: build/bin/kernel_lambda_test 2024-06-26T07:11:45.5189923Z inflating: build/bin/kernel_stackbased_test 2024-06-26T07:11:45.5283467Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-06-26T07:11:45.5333560Z inflating: build/bin/CppSignature_test 2024-06-26T07:11:45.5388069Z inflating: build/bin/backend_fallback_test 2024-06-26T07:11:45.5435876Z inflating: build/bin/op_allowlist_test 2024-06-26T07:11:45.5728637Z inflating: build/bin/op_registration_test 2024-06-26T07:11:45.5790645Z inflating: build/bin/inline_container_test 2024-06-26T07:11:45.5842585Z inflating: build/bin/cuda_apply_test 2024-06-26T07:11:45.5892233Z inflating: build/bin/cuda_allocator_test 2024-06-26T07:11:45.5949338Z inflating: build/bin/cuda_atomic_ops_test 2024-06-26T07:11:45.6002079Z inflating: build/bin/cuda_caching_host_allocator_test 2024-06-26T07:11:45.6069890Z inflating: build/bin/cuda_complex_math_test 2024-06-26T07:11:45.6127226Z inflating: build/bin/cuda_complex_test 2024-06-26T07:11:45.6175108Z inflating: build/bin/cuda_device_test 2024-06-26T07:11:45.6230796Z inflating: build/bin/cuda_cub_test 2024-06-26T07:11:45.6279864Z inflating: build/bin/cuda_dlconvertor_test 2024-06-26T07:11:45.6342441Z inflating: build/bin/cuda_distributions_test 2024-06-26T07:11:45.6397518Z inflating: build/bin/cuda_generator_test 2024-06-26T07:11:45.6445655Z inflating: build/bin/cuda_half_test 2024-06-26T07:11:45.6495127Z inflating: build/bin/cuda_integer_divider_test 2024-06-26T07:11:45.6542642Z inflating: build/bin/cuda_optional_test 2024-06-26T07:11:45.6592794Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-06-26T07:11:45.6643739Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-06-26T07:11:45.6691935Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-06-26T07:11:45.6751053Z inflating: build/bin/cuda_stream_test 2024-06-26T07:11:45.6798450Z inflating: build/bin/cuda_cudnn_test 2024-06-26T07:11:45.6849064Z inflating: build/bin/cuda_vectorized_test 2024-06-26T07:11:45.6863407Z inflating: build/bin/tutorial_tensorexpr 2024-06-26T07:11:45.6927648Z inflating: build/bin/ProcessGroupGlooTest 2024-06-26T07:11:45.6983004Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-06-26T07:11:45.7044425Z inflating: build/bin/ProcessGroupNCCLTest 2024-06-26T07:11:45.7104343Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-06-26T07:11:45.7910054Z inflating: build/bin/test_tensorexpr 2024-06-26T07:11:45.8469097Z inflating: build/bin/test_jit 2024-06-26T07:11:45.8469952Z creating: .additional_ci_files/ 2024-06-26T07:11:45.8522610Z inflating: .additional_ci_files/test-times.json 2024-06-26T07:11:45.8734116Z inflating: .additional_ci_files/test-class-times.json 2024-06-26T07:11:45.8760471Z ##[group]Run rm artifacts.zip 2024-06-26T07:11:45.8760864Z rm artifacts.zip 2024-06-26T07:11:45.8768734Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:45.8769227Z env: 2024-06-26T07:11:45.8769522Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:45.8769961Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:45.8770435Z ##[endgroup] 2024-06-26T07:11:45.9471239Z ##[group]Run df -H 2024-06-26T07:11:45.9471568Z df -H 2024-06-26T07:11:45.9479237Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:45.9479731Z env: 2024-06-26T07:11:45.9480010Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:45.9480597Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:45.9481074Z ##[endgroup] 2024-06-26T07:11:45.9508269Z Filesystem Size Used Avail Use% Mounted on 2024-06-26T07:11:45.9508882Z devtmpfs 34G 0 34G 0% /dev 2024-06-26T07:11:45.9509464Z tmpfs 34G 0 34G 0% /dev/shm 2024-06-26T07:11:45.9510029Z tmpfs 34G 512k 34G 1% /run 2024-06-26T07:11:45.9510499Z tmpfs 34G 0 34G 0% /sys/fs/cgroup 2024-06-26T07:11:45.9510959Z /dev/nvme0n1p1 162G 43G 119G 27% / 2024-06-26T07:11:45.9511574Z tmpfs 6.7G 0 6.7G 0% /run/user/0 2024-06-26T07:11:45.9548740Z Prepare all required actions 2024-06-26T07:11:45.9549172Z Getting action download info 2024-06-26T07:11:46.0992803Z ##[group]Run ./.github/actions/download-td-artifacts 2024-06-26T07:11:46.0993257Z with: 2024-06-26T07:11:46.0993512Z env: 2024-06-26T07:11:46.0993784Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:46.0994229Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:46.0994717Z ##[endgroup] 2024-06-26T07:11:46.1025155Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:11:46.1025594Z with: 2024-06-26T07:11:46.1025859Z name: td_results 2024-06-26T07:11:46.1026178Z s3-bucket: gha-artifacts 2024-06-26T07:11:46.1026521Z region: us-east-1 2024-06-26T07:11:46.1026816Z env: 2024-06-26T07:11:46.1027084Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:46.1027529Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:46.1028004Z ##[endgroup] 2024-06-26T07:11:46.5078355Z (node:26421) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:11:46.5079031Z 2024-06-26T07:11:46.5079289Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:11:46.5080004Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:11:46.5080887Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:11:46.5696376Z Found 0 objects with prefix pytorch/pytorch/9673646806/td_results/ 2024-06-26T07:11:46.5703298Z Artifact download has finished successfully 2024-06-26T07:11:46.5844932Z ##[group]Run mkdir -p .additional_ci_files 2024-06-26T07:11:46.5845407Z mkdir -p .additional_ci_files 2024-06-26T07:11:46.5845959Z mv td_results.json .additional_ci_files/td_results.json 2024-06-26T07:11:46.5853750Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:46.5854236Z env: 2024-06-26T07:11:46.5854516Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:46.5854974Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:46.5855447Z ##[endgroup] 2024-06-26T07:11:46.5891028Z mv: cannot stat 'td_results.json': No such file or directory 2024-06-26T07:11:46.5901163Z ##[error]Process completed with exit code 1. 2024-06-26T07:11:46.5948507Z ##[group]Run .github/scripts/parse_ref.py 2024-06-26T07:11:46.5949035Z .github/scripts/parse_ref.py 2024-06-26T07:11:46.5956741Z shell: /usr/bin/bash -e {0} 2024-06-26T07:11:46.5957079Z env: 2024-06-26T07:11:46.5957359Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:46.5957812Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:46.5958285Z ##[endgroup] 2024-06-26T07:11:46.6183797Z Prepare all required actions 2024-06-26T07:11:46.6219952Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-06-26T07:11:46.6220401Z with: 2024-06-26T07:11:46.6220948Z github-token: *** 2024-06-26T07:11:46.6221258Z env: 2024-06-26T07:11:46.6221531Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:46.6221985Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:46.6222466Z ##[endgroup] 2024-06-26T07:11:46.6237340Z ##[group]Run set -eux 2024-06-26T07:11:46.6237670Z set -eux 2024-06-26T07:11:46.6238242Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-06-26T07:11:46.6246111Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:46.6246716Z env: 2024-06-26T07:11:46.6246994Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:46.6247439Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:46.6248067Z GITHUB_TOKEN: *** 2024-06-26T07:11:46.6248373Z ##[endgroup] 2024-06-26T07:11:46.6267678Z + python3 .github/scripts/get_workflow_job_id.py 9673646806 i-02ccdeb779ec2a6a3 2024-06-26T07:11:47.2385645Z setting job-id=26688916604 2024-06-26T07:11:47.2386735Z setting job-name=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:11:47.2579965Z Prepare all required actions 2024-06-26T07:11:47.2580404Z Getting action download info 2024-06-26T07:11:47.3578750Z ##[group]Run ./.github/actions/filter-test-configs 2024-06-26T07:11:47.3579193Z with: 2024-06-26T07:11:47.3579661Z github-token: *** 2024-06-26T07:11:47.3586299Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-06-26T07:11:47.3593533Z job-name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:11:47.3594367Z env: 2024-06-26T07:11:47.3594635Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:47.3595085Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:47.3595558Z ##[endgroup] 2024-06-26T07:11:47.3634764Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:11:47.3635304Z with: 2024-06-26T07:11:47.3635569Z shell: bash 2024-06-26T07:11:47.3635871Z timeout_minutes: 10 2024-06-26T07:11:47.3636199Z max_attempts: 5 2024-06-26T07:11:47.3636504Z retry_wait_seconds: 30 2024-06-26T07:11:47.3637588Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:11:47.3638732Z polling_interval_seconds: 1 2024-06-26T07:11:47.3639108Z warning_on_retry: true 2024-06-26T07:11:47.3639445Z continue_on_error: false 2024-06-26T07:11:47.3639783Z env: 2024-06-26T07:11:47.3640058Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:47.3640495Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:47.3641115Z GITHUB_TOKEN: *** 2024-06-26T07:11:47.3641423Z ##[endgroup] 2024-06-26T07:11:47.4128524Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:11:47.6279562Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T07:11:47.6432759Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.7/site-packages (2.27.1) 2024-06-26T07:11:47.6578570Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.7/site-packages (6.0.1) 2024-06-26T07:11:47.6587609Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2024.6.2) 2024-06-26T07:11:47.6596182Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2.0.12) 2024-06-26T07:11:47.6617380Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (3.7) 2024-06-26T07:11:47.6629010Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (1.26.19) 2024-06-26T07:11:48.4123680Z Command completed after 1 attempt(s). 2024-06-26T07:11:48.4162881Z ##[group]Run set -x 2024-06-26T07:11:48.4163286Z set -x 2024-06-26T07:11:48.4163644Z  2024-06-26T07:11:48.4164253Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:11:48.4164919Z # in runner workspace 2024-06-26T07:11:48.4165447Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-06-26T07:11:48.4173199Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:48.4173742Z env: 2024-06-26T07:11:48.4174037Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:48.4174513Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:48.4175033Z ##[endgroup] 2024-06-26T07:11:48.4195957Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-06-26T07:11:48.4410403Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:11:48.4410948Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:11:48.4411424Z echo "Job name: ${JOB_NAME}" 2024-06-26T07:11:48.4411825Z  2024-06-26T07:11:48.4412358Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:11:48.4413030Z # in runner workspace 2024-06-26T07:11:48.4413595Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-06-26T07:11:48.4414220Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-06-26T07:11:48.4414671Z  --job-name "${JOB_NAME}" \ 2024-06-26T07:11:48.4421208Z  --test-matrix "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-06-26T07:11:48.4428249Z  --selected-test-configs "" \ 2024-06-26T07:11:48.4428710Z  --pr-number "${PR_NUMBER}" \ 2024-06-26T07:11:48.4429129Z  --tag "${TAG}" \ 2024-06-26T07:11:48.4429522Z  --event-name "${EVENT_NAME}" \ 2024-06-26T07:11:48.4429960Z  --schedule "${SCHEDULE}" \ 2024-06-26T07:11:48.4430487Z  --branch "${HEAD_BRANCH}" 2024-06-26T07:11:48.4438306Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:48.4438804Z env: 2024-06-26T07:11:48.4439076Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:48.4439526Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:48.4440188Z GITHUB_TOKEN: *** 2024-06-26T07:11:48.4440971Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:11:48.4441820Z PR_NUMBER: 2024-06-26T07:11:48.4442129Z TAG: ciflow/inductor/129470 2024-06-26T07:11:48.4442707Z EVENT_NAME: push 2024-06-26T07:11:48.4443016Z SCHEDULE: 2024-06-26T07:11:48.4443296Z HEAD_BRANCH: 2024-06-26T07:11:48.4443590Z ##[endgroup] 2024-06-26T07:11:48.4462856Z Workflow: inductor-periodic 2024-06-26T07:11:48.4463844Z Job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:11:48.6759007Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-06-26T07:11:48.8127954Z ##[group]Run echo "Filtered matrix:" 2024-06-26T07:11:48.8128403Z echo "Filtered matrix:" 2024-06-26T07:11:48.8134852Z echo "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" 2024-06-26T07:11:48.8141578Z  2024-06-26T07:11:48.8141854Z echo 2024-06-26T07:11:48.8142233Z echo "Is the current job unstable? False" 2024-06-26T07:11:48.8142687Z  2024-06-26T07:11:48.8142947Z echo 2024-06-26T07:11:48.8143300Z echo "Is keep-going label set? False" 2024-06-26T07:11:48.8143740Z  2024-06-26T07:11:48.8144136Z echo 2024-06-26T07:11:48.8144447Z echo "Renabled issues? " 2024-06-26T07:11:48.8152093Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:48.8152576Z env: 2024-06-26T07:11:48.8152964Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:48.8153408Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:48.8153878Z ##[endgroup] 2024-06-26T07:11:48.8174031Z Filtered matrix: 2024-06-26T07:11:48.8182348Z {include: [{config: dynamo_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}]} 2024-06-26T07:11:48.8189018Z 2024-06-26T07:11:48.8189168Z Is the current job unstable? False 2024-06-26T07:11:48.8189451Z 2024-06-26T07:11:48.8189725Z Is keep-going label set? False 2024-06-26T07:11:48.8189979Z 2024-06-26T07:11:48.8190112Z Renabled issues? 2024-06-26T07:11:48.8244755Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:11:48.8245505Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:11:48.8253232Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:48.8253713Z env: 2024-06-26T07:11:48.8254016Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:48.8254479Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:48.8254954Z JOB_TIMEOUT: 240 2024-06-26T07:11:48.8255248Z ##[endgroup] 2024-06-26T07:11:48.8325811Z ##[group]Run set -x 2024-06-26T07:11:48.8326191Z set -x 2024-06-26T07:11:48.8326483Z  2024-06-26T07:11:48.8326829Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-06-26T07:11:48.8327357Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-06-26T07:11:48.8327904Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-06-26T07:11:48.8328399Z  TEST_COMMAND=.ci/onnx/test.sh 2024-06-26T07:11:48.8328799Z else 2024-06-26T07:11:48.8329123Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:11:48.8329548Z fi 2024-06-26T07:11:48.8329822Z  2024-06-26T07:11:48.8330280Z # detached container should get cleaned up by teardown_ec2_linux 2024-06-26T07:11:48.8331014Z # TODO: Stop building test binaries as part of the build phase 2024-06-26T07:11:48.8331673Z # Used for GPU_FLAG since that doesn't play nice 2024-06-26T07:11:48.8332241Z # shellcheck disable=SC2086,SC2090 2024-06-26T07:11:48.8332686Z container_name=$(docker run \ 2024-06-26T07:11:48.8333098Z  ${GPU_FLAG:-} \ 2024-06-26T07:11:48.8333461Z  -e BUILD_ENVIRONMENT \ 2024-06-26T07:11:48.8333853Z  -e PR_NUMBER \ 2024-06-26T07:11:48.8334199Z  -e GITHUB_ACTIONS \ 2024-06-26T07:11:48.8334583Z  -e GITHUB_REPOSITORY \ 2024-06-26T07:11:48.8334976Z  -e GITHUB_WORKFLOW \ 2024-06-26T07:11:48.8335353Z  -e GITHUB_JOB \ 2024-06-26T07:11:48.8335702Z  -e GITHUB_RUN_ID \ 2024-06-26T07:11:48.8336180Z  -e GITHUB_RUN_NUMBER \ 2024-06-26T07:11:48.8336576Z  -e GITHUB_RUN_ATTEMPT \ 2024-06-26T07:11:48.8336955Z  -e JOB_ID \ 2024-06-26T07:11:48.8337286Z  -e JOB_NAME \ 2024-06-26T07:11:48.8337625Z  -e BASE_SHA \ 2024-06-26T07:11:48.8337958Z  -e BRANCH \ 2024-06-26T07:11:48.8338281Z  -e SHA1 \ 2024-06-26T07:11:48.8338619Z  -e AWS_DEFAULT_REGION \ 2024-06-26T07:11:48.8339013Z  -e IN_WHEEL_TEST \ 2024-06-26T07:11:48.8339385Z  -e SHARD_NUMBER \ 2024-06-26T07:11:48.8339740Z  -e TEST_CONFIG \ 2024-06-26T07:11:48.8340102Z  -e NUM_TEST_SHARDS \ 2024-06-26T07:11:48.8340486Z  -e REENABLED_ISSUES \ 2024-06-26T07:11:48.8340887Z  -e CONTINUE_THROUGH_ERROR \ 2024-06-26T07:11:48.8341301Z  -e VERBOSE_TEST_LOGS \ 2024-06-26T07:11:48.8341695Z  -e NO_TEST_TIMEOUT \ 2024-06-26T07:11:48.8342067Z  -e NO_TD \ 2024-06-26T07:11:48.8342405Z  -e TD_DISTRIBUTED \ 2024-06-26T07:11:48.8342777Z  -e PR_LABELS \ 2024-06-26T07:11:48.8343173Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-06-26T07:11:48.8343622Z  -e SCCACHE_BUCKET \ 2024-06-26T07:11:48.8344102Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-06-26T07:11:48.8344529Z  -e XLA_CUDA \ 2024-06-26T07:11:48.8344925Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-06-26T07:11:48.8345427Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-06-26T07:11:48.8345930Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-06-26T07:11:48.8346432Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-06-26T07:11:48.8346886Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-06-26T07:11:48.8347297Z  -e DASHBOARD_TAG \ 2024-06-26T07:11:48.8347756Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-06-26T07:11:48.8348296Z  --security-opt seccomp=unconfined \ 2024-06-26T07:11:48.8348758Z  --cap-add=SYS_PTRACE \ 2024-06-26T07:11:48.8349141Z  --ipc=host \ 2024-06-26T07:11:48.8349486Z  --shm-size="${SHM_SIZE}" \ 2024-06-26T07:11:48.8349872Z  --tty \ 2024-06-26T07:11:48.8350180Z  --detach \ 2024-06-26T07:11:48.8350527Z  --name="${container_name}" \ 2024-06-26T07:11:48.8350926Z  --user jenkins \ 2024-06-26T07:11:48.8351468Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-06-26T07:11:48.8352021Z  -w /var/lib/jenkins/workspace \ 2024-06-26T07:11:48.8352450Z  "${DOCKER_IMAGE}" 2024-06-26T07:11:48.8352784Z ) 2024-06-26T07:11:48.8353182Z # Propagate download.pytorch.org IP to container 2024-06-26T07:11:48.8354076Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-06-26T07:11:48.8355016Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-06-26T07:11:48.8355897Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-06-26T07:11:48.8363408Z shell: /usr/bin/bash -e {0} 2024-06-26T07:11:48.8363755Z env: 2024-06-26T07:11:48.8364028Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:48.8364464Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:48.8365061Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:11:48.8365548Z PR_NUMBER: 2024-06-26T07:11:48.8365864Z GITHUB_REPOSITORY: pytorch/pytorch 2024-06-26T07:11:48.8366278Z GITHUB_WORKFLOW: inductor-periodic 2024-06-26T07:11:48.8366673Z GITHUB_JOB: test 2024-06-26T07:11:48.8366980Z GITHUB_RUN_ID: 9673646806 2024-06-26T07:11:48.8367330Z GITHUB_RUN_NUMBER: 45567 2024-06-26T07:11:48.8367667Z GITHUB_RUN_ATTEMPT: 1 2024-06-26T07:11:48.8367992Z JOB_ID: 26688916604 2024-06-26T07:11:48.8368769Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:11:48.8369681Z BRANCH: 2024-06-26T07:11:48.8370006Z SHA1: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:11:48.8370500Z BASE_SHA: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:11:48.8370958Z TEST_CONFIG: aot_eager_timm 2024-06-26T07:11:48.8371312Z SHARD_NUMBER: 1 2024-06-26T07:11:48.8371605Z NUM_TEST_SHARDS: 2 2024-06-26T07:11:48.8371919Z REENABLED_ISSUES: 2024-06-26T07:11:48.8372253Z CONTINUE_THROUGH_ERROR: False 2024-06-26T07:11:48.8372628Z VERBOSE_TEST_LOGS: False 2024-06-26T07:11:48.8372975Z NO_TEST_TIMEOUT: False 2024-06-26T07:11:48.8373302Z NO_TD: False 2024-06-26T07:11:48.8373596Z TD_DISTRIBUTED: False 2024-06-26T07:11:48.8374008Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-06-26T07:11:48.8374553Z SCCACHE_S3_KEY_PREFIX: inductor-periodic 2024-06-26T07:11:48.8374968Z SHM_SIZE: 2g 2024-06-26T07:11:48.8376021Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:11:48.8377156Z XLA_CUDA: 2024-06-26T07:11:48.8377625Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:11:48.8378220Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-06-26T07:11:48.8378643Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-06-26T07:11:48.8379037Z DASHBOARD_TAG: 2024-06-26T07:11:48.8379540Z HUGGING_FACE_HUB_TOKEN: *** 2024-06-26T07:11:48.8379911Z ##[endgroup] 2024-06-26T07:11:48.8400439Z + [[ aot_eager_timm == \m\u\l\t\i\g\p\u ]] 2024-06-26T07:11:48.8401118Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]] 2024-06-26T07:11:48.8401625Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:11:48.8407699Z +++ nproc --ignore=2 2024-06-26T07:11:48.8420461Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_9673646806 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:12:00.0720466Z + container_name=9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:12:00.0722038Z + docker exec -i 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 sudo bash -c '/bin/cat >> /etc/hosts' 2024-06-26T07:12:00.0724942Z + grep download.pytorch.org /etc/hosts 2024-06-26T07:12:00.1445869Z + echo DOCKER_CONTAINER_ID=9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:12:00.1448446Z ++ echo dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:12:00.1449989Z + docker exec -t 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 sh -c 'pip install dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-06-26T07:12:00.5249382Z Processing ./dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+gitb8c4c54) 2024-06-26T07:12:00.8367252Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.13.1) 2024-06-26T07:12:00.8369887Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (4.12.2) 2024-06-26T07:12:00.8371867Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.12.1) 2024-06-26T07:12:00.8374531Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.8.8) 2024-06-26T07:12:00.8377383Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.1.4) 2024-06-26T07:12:00.8380474Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2024.2.0) 2024-06-26T07:12:00.8406198Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.3.0) 2024-06-26T07:12:00.8465933Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.21.2) 2024-06-26T07:12:00.8896291Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.1.5) 2024-06-26T07:12:00.9054562Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.3.0) 2024-06-26T07:12:02.2160263Z Installing collected packages: torch 2024-06-26T07:12:10.9292799Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-06-26T07:12:10.9294154Z timm 0.9.7 requires torchvision, which is not installed. 2024-06-26T07:12:10.9294818Z Successfully installed torch-2.5.0a0+gitb8c4c54 2024-06-26T07:12:11.0053280Z ++ dirname .ci/pytorch/test.sh 2024-06-26T07:12:11.0059054Z + source .ci/pytorch/common.sh 2024-06-26T07:12:11.0060861Z +++ dirname .ci/pytorch/common.sh 2024-06-26T07:12:11.0065657Z ++ source .ci/pytorch/common_utils.sh 2024-06-26T07:12:11.0067651Z +++ declare -f -t trap_add 2024-06-26T07:12:11.0072345Z ++ set -ex 2024-06-26T07:12:11.0072993Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-06-26T07:12:11.0073543Z ++ BUILD_TEST_LIBTORCH=0 2024-06-26T07:12:11.0074155Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]] 2024-06-26T07:12:11.0075422Z ++ stat -c %u /var/lib/jenkins/workspace 2024-06-26T07:12:11.0083939Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-06-26T07:12:11.0084373Z + trap_add cleanup_workspace EXIT 2024-06-26T07:12:11.0084778Z + trap_add_cmd=cleanup_workspace 2024-06-26T07:12:11.0085150Z + shift 2024-06-26T07:12:11.0088712Z + for trap_add_name in "$@" 2024-06-26T07:12:11.0089273Z +++ trap -p EXIT 2024-06-26T07:12:11.0090882Z ++ eval 'extract_trap_cmd ' 2024-06-26T07:12:11.0091363Z +++ extract_trap_cmd 2024-06-26T07:12:11.0092147Z +++ printf '%s\n' '' 2024-06-26T07:12:11.0092702Z ++ printf '%s\n' cleanup_workspace 2024-06-26T07:12:11.0093195Z + trap -- ' 2024-06-26T07:12:11.0093518Z cleanup_workspace' EXIT 2024-06-26T07:12:11.0093970Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-06-26T07:12:11.5451980Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-06-26T07:12:11.5924137Z + echo 'Environment variables:' 2024-06-26T07:12:11.5924583Z Environment variables: 2024-06-26T07:12:11.5924904Z + env 2024-06-26T07:12:11.5929358Z INSTALLED_DB=yes 2024-06-26T07:12:11.5929774Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-06-26T07:12:11.5930177Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:12:11.5930562Z NV_NVML_DEV_VERSION=12.1.105-1 2024-06-26T07:12:11.5931468Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:12:11.5932148Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:12:11.5932715Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-06-26T07:12:11.5933257Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:12:11.5933872Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:12:11.5934433Z HOSTNAME=9f605c1586ec 2024-06-26T07:12:11.5935337Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_7ed15f99-0d2c-4d8d-9481-c167bf6fd31e 2024-06-26T07:12:11.5936198Z GITHUB_ACTION=__self 2024-06-26T07:12:11.5936557Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:12:11.5940475Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-06-26T07:12:11.5944763Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-06-26T07:12:11.5945335Z NV_NVTX_VERSION=12.1.105-1 2024-06-26T07:12:11.5945709Z GITHUB_RUN_NUMBER=45567 2024-06-26T07:12:11.5946094Z TEST_CONFIG=aot_eager_timm 2024-06-26T07:12:11.5946556Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:12:11.5947062Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:12:11.5947583Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-06-26T07:12:11.5948026Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-06-26T07:12:11.5948513Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-06-26T07:12:11.5948944Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:12:11.5949507Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:12:11.5950020Z GITHUB_REF_TYPE=tag 2024-06-26T07:12:11.5950383Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:12:11.5950767Z NCCL_VERSION=2.17.1-1 2024-06-26T07:12:11.5951219Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:11.5951842Z XLA_CUDA= 2024-06-26T07:12:11.5952370Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:12:11.5954438Z *** 2024-06-26T07:12:11.5954774Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:12:11.5955183Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:12:11.5955602Z GITHUB_ACTIONS=true 2024-06-26T07:12:11.5955933Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:11.5956506Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-06-26T07:12:11.5957107Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-06-26T07:12:11.5957572Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:11.5958142Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:12:11.5958639Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:11.5959651Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:12:11.5960447Z UCC_HOME=/usr 2024-06-26T07:12:11.5960809Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-06-26T07:12:11.5961232Z VERBOSE_TEST_LOGS=False 2024-06-26T07:12:11.5961631Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:12:11.5962115Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-06-26T07:12:11.5962804Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:12:11.5963355Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-06-26T07:12:11.5963736Z SHARD_NUMBER=1 2024-06-26T07:12:11.5964102Z GITHUB_REF_PROTECTED=false 2024-06-26T07:12:11.5964458Z HOME=/var/lib/jenkins 2024-06-26T07:12:11.5964829Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:12:11.5965321Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:12:11.5965800Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:12:11.5966569Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-06-26T07:12:11.5966982Z CUDA_VERSION=12.1.1 2024-06-26T07:12:11.5967390Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-06-26T07:12:11.5967824Z NUM_TEST_SHARDS=2 2024-06-26T07:12:11.5968124Z UCX_HOME=/usr 2024-06-26T07:12:11.5968668Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-06-26T07:12:11.5969753Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_7ed15f99-0d2c-4d8d-9481-c167bf6fd31e 2024-06-26T07:12:11.5971098Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:12:11.5972417Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_7ed15f99-0d2c-4d8d-9481-c167bf6fd31e 2024-06-26T07:12:11.5973573Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:12:11.5974216Z GITHUB_EVENT_NAME=push 2024-06-26T07:12:11.5974541Z DASHBOARD_TAG= 2024-06-26T07:12:11.5974840Z GITHUB_RUN_ID=9673646806 2024-06-26T07:12:11.5975285Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-06-26T07:12:11.5975809Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-06-26T07:12:11.5976779Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_7ed15f99-0d2c-4d8d-9481-c167bf6fd31e 2024-06-26T07:12:11.5977671Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:12:11.5978073Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-06-26T07:12:11.5978440Z PR_NUMBER= 2024-06-26T07:12:11.5978731Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:12:11.5979062Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:12:11.5979491Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:12:11.5980018Z TERM=xterm 2024-06-26T07:12:11.5980360Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-06-26T07:12:11.5980768Z INSTALLED_VISION=yes 2024-06-26T07:12:11.5981072Z BRANCH= 2024-06-26T07:12:11.5981354Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:12:11.5981755Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:12:11.5982171Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:12:11.5982895Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:12:11.5983631Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:12:11.5984218Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:12:11.5984800Z REENABLED_ISSUES= 2024-06-26T07:12:11.5985099Z SHLVL=1 2024-06-26T07:12:11.5985357Z MAX_JOBS=14 2024-06-26T07:12:11.5985674Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-06-26T07:12:11.5986014Z NVARCH=x86_64 2024-06-26T07:12:11.5986307Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:12:11.5986757Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:11.5987279Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:12:11.5987747Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-06-26T07:12:11.5988390Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:12:11.5988964Z GITHUB_JOB=test 2024-06-26T07:12:11.5989361Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-06-26T07:12:11.5989900Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:12:11.5990403Z NO_TEST_TIMEOUT=False 2024-06-26T07:12:11.5990725Z TD_DISTRIBUTED=False 2024-06-26T07:12:11.5991087Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-06-26T07:12:11.5991562Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:12:11.5991982Z NV_NVPROF_VERSION=12.1.105-1 2024-06-26T07:12:11.5992341Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:12:11.5992691Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:12:11.5993059Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:12:11.5994150Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:12:11.5995186Z GITHUB_BASE_REF= 2024-06-26T07:12:11.5995500Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:12:11.5995872Z CI=true 2024-06-26T07:12:11.5996261Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:12:11.5996670Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:12:11.5997038Z JOB_ID=26688916604 2024-06-26T07:12:11.5997344Z INSTALLED_PROTOBUF=yes 2024-06-26T07:12:11.5997664Z GITHUB_HEAD_REF= 2024-06-26T07:12:11.5997951Z GITHUB_ACTION_REF= 2024-06-26T07:12:11.5998372Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:12:11.5998881Z GITHUB_WORKFLOW=inductor-periodic 2024-06-26T07:12:11.5999284Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:12:11.6000151Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_7ed15f99-0d2c-4d8d-9481-c167bf6fd31e 2024-06-26T07:12:11.6000946Z NO_TD=False 2024-06-26T07:12:11.6001243Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:12:11.6001615Z _=/usr/bin/env 2024-06-26T07:12:11.6002076Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-06-26T07:12:11.6073587Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-06-26T07:12:11.6074597Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-06-26T07:12:11.6075523Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-06-26T07:12:11.6076408Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-06-26T07:12:11.6077050Z + BUILD_DIR=build 2024-06-26T07:12:11.6077442Z + BUILD_RENAMED_DIR=build_renamed 2024-06-26T07:12:11.6077842Z + BUILD_BIN_DIR=build/bin 2024-06-26T07:12:11.6078235Z + SHARD_NUMBER=1 2024-06-26T07:12:11.6078546Z + NUM_TEST_SHARDS=2 2024-06-26T07:12:11.6078859Z + export VALGRIND=ON 2024-06-26T07:12:11.6079237Z + VALGRIND=ON 2024-06-26T07:12:11.6079720Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]] 2024-06-26T07:12:11.6080275Z + [[ 0 == \1 ]] 2024-06-26T07:12:11.6080569Z + [[ False == \1 ]] 2024-06-26T07:12:11.6081106Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]] 2024-06-26T07:12:11.6081689Z ++ realpath build/custom_test_artifacts 2024-06-26T07:12:11.6084281Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-06-26T07:12:11.6085011Z + [[ -n '' ]] 2024-06-26T07:12:11.6085421Z + echo 'Environment variables' 2024-06-26T07:12:11.6085806Z Environment variables 2024-06-26T07:12:11.6086187Z + env 2024-06-26T07:12:11.6089860Z INSTALLED_DB=yes 2024-06-26T07:12:11.6090255Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-06-26T07:12:11.6090761Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:12:11.6091358Z NV_NVML_DEV_VERSION=12.1.105-1 2024-06-26T07:12:11.6092050Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:12:11.6092617Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:12:11.6093192Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-06-26T07:12:11.6093978Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:12:11.6094741Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:12:11.6095535Z HOSTNAME=9f605c1586ec 2024-06-26T07:12:11.6096590Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_7ed15f99-0d2c-4d8d-9481-c167bf6fd31e 2024-06-26T07:12:11.6097659Z GITHUB_ACTION=__self 2024-06-26T07:12:11.6098084Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:12:11.6101811Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-06-26T07:12:11.6105624Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-06-26T07:12:11.6106299Z NV_NVTX_VERSION=12.1.105-1 2024-06-26T07:12:11.6106649Z GITHUB_RUN_NUMBER=45567 2024-06-26T07:12:11.6106998Z TEST_CONFIG=aot_eager_timm 2024-06-26T07:12:11.6107371Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:12:11.6107838Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:12:11.6108289Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-06-26T07:12:11.6108729Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-06-26T07:12:11.6109149Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-06-26T07:12:11.6109578Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:12:11.6110070Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:12:11.6110529Z GITHUB_REF_TYPE=tag 2024-06-26T07:12:11.6110859Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:12:11.6111226Z NCCL_VERSION=2.17.1-1 2024-06-26T07:12:11.6111611Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:11.6112050Z XLA_CUDA= 2024-06-26T07:12:11.6112499Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:12:11.6112885Z *** 2024-06-26T07:12:11.6113174Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:12:11.6113571Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:12:11.6113935Z GITHUB_ACTIONS=true 2024-06-26T07:12:11.6114262Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:11.6114744Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-06-26T07:12:11.6115268Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-06-26T07:12:11.6115736Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:11.6116226Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:12:11.6116702Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:11.6117602Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:12:11.6118356Z UCC_HOME=/usr 2024-06-26T07:12:11.6118683Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-06-26T07:12:11.6119089Z VERBOSE_TEST_LOGS=False 2024-06-26T07:12:11.6119436Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:12:11.6119878Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-06-26T07:12:11.6120378Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:12:11.6120838Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-06-26T07:12:11.6121218Z SHARD_NUMBER=1 2024-06-26T07:12:11.6121523Z GITHUB_REF_PROTECTED=false 2024-06-26T07:12:11.6121876Z HOME=/var/lib/jenkins 2024-06-26T07:12:11.6122461Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:12:11.6123105Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:12:11.6123586Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:12:11.6124137Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-06-26T07:12:11.6124555Z CUDA_VERSION=12.1.1 2024-06-26T07:12:11.6124964Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-06-26T07:12:11.6125404Z NUM_TEST_SHARDS=2 2024-06-26T07:12:11.6125711Z UCX_HOME=/usr 2024-06-26T07:12:11.6126221Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-06-26T07:12:11.6127270Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_7ed15f99-0d2c-4d8d-9481-c167bf6fd31e 2024-06-26T07:12:11.6128622Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:12:11.6129946Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_7ed15f99-0d2c-4d8d-9481-c167bf6fd31e 2024-06-26T07:12:11.6131045Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:12:11.6131748Z GITHUB_EVENT_NAME=push 2024-06-26T07:12:11.6132075Z DASHBOARD_TAG= 2024-06-26T07:12:11.6132384Z GITHUB_RUN_ID=9673646806 2024-06-26T07:12:11.6132836Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-06-26T07:12:11.6152286Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-06-26T07:12:11.6153315Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_7ed15f99-0d2c-4d8d-9481-c167bf6fd31e 2024-06-26T07:12:11.6154204Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:12:11.6154600Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-06-26T07:12:11.6155100Z PR_NUMBER= 2024-06-26T07:12:11.6155373Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:12:11.6155686Z VALGRIND=ON 2024-06-26T07:12:11.6155987Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:12:11.6156421Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:12:11.6156867Z TERM=xterm 2024-06-26T07:12:11.6157190Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-06-26T07:12:11.6157600Z INSTALLED_VISION=yes 2024-06-26T07:12:11.6157907Z BRANCH= 2024-06-26T07:12:11.6158185Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:12:11.6158582Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:12:11.6159002Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:12:11.6159734Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:12:11.6160480Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:12:11.6160958Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:12:11.6161412Z REENABLED_ISSUES= 2024-06-26T07:12:11.6161705Z SHLVL=1 2024-06-26T07:12:11.6161960Z MAX_JOBS=14 2024-06-26T07:12:11.6162410Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-06-26T07:12:11.6162753Z NVARCH=x86_64 2024-06-26T07:12:11.6163044Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:12:11.6163496Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:11.6164018Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:12:11.6164483Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-06-26T07:12:11.6165127Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:12:11.6165692Z GITHUB_JOB=test 2024-06-26T07:12:11.6166082Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-06-26T07:12:11.6166621Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:12:11.6167123Z NO_TEST_TIMEOUT=False 2024-06-26T07:12:11.6167449Z TD_DISTRIBUTED=False 2024-06-26T07:12:11.6167816Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-06-26T07:12:11.6168242Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:12:11.6168658Z NV_NVPROF_VERSION=12.1.105-1 2024-06-26T07:12:11.6169026Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:12:11.6169368Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:12:11.6169723Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:12:11.6170735Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:12:11.6171833Z GITHUB_BASE_REF= 2024-06-26T07:12:11.6172247Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:12:11.6172638Z CI=true 2024-06-26T07:12:11.6172958Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:12:11.6173373Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:12:11.6173740Z JOB_ID=26688916604 2024-06-26T07:12:11.6174059Z INSTALLED_PROTOBUF=yes 2024-06-26T07:12:11.6174393Z GITHUB_HEAD_REF= 2024-06-26T07:12:11.6174698Z GITHUB_ACTION_REF= 2024-06-26T07:12:11.6175117Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:12:11.6175634Z GITHUB_WORKFLOW=inductor-periodic 2024-06-26T07:12:11.6176037Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:12:11.6176927Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_7ed15f99-0d2c-4d8d-9481-c167bf6fd31e 2024-06-26T07:12:11.6177721Z NO_TD=False 2024-06-26T07:12:11.6178025Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:12:11.6178385Z _=/usr/bin/env 2024-06-26T07:12:11.6178710Z + echo 'Testing pytorch' 2024-06-26T07:12:11.6179052Z Testing pytorch 2024-06-26T07:12:11.6179380Z + export LANG=C.UTF-8 2024-06-26T07:12:11.6179717Z + LANG=C.UTF-8 2024-06-26T07:12:11.6180003Z + PR_NUMBER= 2024-06-26T07:12:11.6180323Z + [[ aot_eager_timm == \d\e\f\a\u\l\t ]] 2024-06-26T07:12:11.6180790Z + [[ aot_eager_timm == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-06-26T07:12:11.6181250Z + [[ aot_eager_timm == \s\l\o\w ]] 2024-06-26T07:12:11.6181833Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-06-26T07:12:11.6182519Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]] 2024-06-26T07:12:11.6183064Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:12:11.6183625Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:12:11.6184127Z + [[ aot_eager_timm == *crossref* ]] 2024-06-26T07:12:11.6184675Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-06-26T07:12:11.6185303Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]] 2024-06-26T07:12:11.6185929Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:12:11.6186484Z + pip_install --user ninja==1.10.2 2024-06-26T07:12:11.6187013Z + pip install --progress-bar off --user ninja==1.10.2 2024-06-26T07:12:11.9748418Z Collecting ninja==1.10.2 2024-06-26T07:12:11.9910800Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-06-26T07:12:12.0054100Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-06-26T07:12:13.2927094Z Installing collected packages: ninja 2024-06-26T07:12:13.2993134Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-06-26T07:12:13.2994408Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-06-26T07:12:13.3046141Z Successfully installed ninja-1.10.2 2024-06-26T07:12:13.3881265Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:12:13.3883779Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:12:13.3885391Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-06-26T07:12:13.3885953Z + install_tlparse 2024-06-26T07:12:13.3886347Z + pip_install --user tlparse==0.3.7 2024-06-26T07:12:13.3886964Z + pip install --progress-bar off --user tlparse==0.3.7 2024-06-26T07:12:13.7390459Z Collecting tlparse==0.3.7 2024-06-26T07:12:13.7564302Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-06-26T07:12:13.7705785Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-06-26T07:12:15.0983353Z Installing collected packages: tlparse 2024-06-26T07:12:15.1365551Z Successfully installed tlparse-0.3.7 2024-06-26T07:12:15.2167760Z ++ python -m site --user-base 2024-06-26T07:12:15.2319218Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:12:15.2321343Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]] 2024-06-26T07:12:15.2322531Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]] 2024-06-26T07:12:15.2323510Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:12:15.2324769Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-06-26T07:12:15.2325927Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass 2024-06-26T07:12:15.2326614Z + cd test 2024-06-26T07:12:15.2327135Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-06-26T07:12:16.6941224Z + [[ aot_eager_timm == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-06-26T07:12:16.6941842Z + [[ aot_eager_timm == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-06-26T07:12:16.6942626Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:12:16.6943184Z + pushd test 2024-06-26T07:12:16.6943508Z ~/workspace/test ~/workspace 2024-06-26T07:12:16.6944101Z ++ python -c 'import torch; print(torch.version.cuda)' 2024-06-26T07:12:18.1662395Z + CUDA_VERSION=12.1 2024-06-26T07:12:18.1662923Z + '[' 12.1 == 12.4 ']' 2024-06-26T07:12:18.1663272Z + ISCUDA124= 2024-06-26T07:12:18.1663571Z + popd 2024-06-26T07:12:18.1663931Z ~/workspace 2024-06-26T07:12:18.1667314Z + DYNAMO_BENCHMARK_FLAGS=() 2024-06-26T07:12:18.1668051Z + [[ aot_eager_timm == *dynamo_eager* ]] 2024-06-26T07:12:18.1668496Z + [[ aot_eager_timm == *aot_eager* ]] 2024-06-26T07:12:18.1669037Z + DYNAMO_BENCHMARK_FLAGS+=(--backend aot_eager) 2024-06-26T07:12:18.1669501Z + [[ aot_eager_timm == *dynamic* ]] 2024-06-26T07:12:18.1669930Z + [[ aot_eager_timm == *cpu_inductor* ]] 2024-06-26T07:12:18.1670422Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-06-26T07:12:18.1695461Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-06-26T07:12:18.1696162Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-06-26T07:12:18.1696797Z + cd test 2024-06-26T07:12:18.1697237Z + python -c 'import torch; print(torch.__config__.show())' 2024-06-26T07:12:19.5207949Z PyTorch built with: 2024-06-26T07:12:19.5208619Z - GCC 9.4 2024-06-26T07:12:19.5209066Z - C++ Version: 201703 2024-06-26T07:12:19.5209993Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:12:19.5211081Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:12:19.5211735Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:12:19.5212235Z - LAPACK is enabled (usually provided by MKL) 2024-06-26T07:12:19.5212707Z - NNPACK is enabled 2024-06-26T07:12:19.5213086Z - CPU capability usage: AVX2 2024-06-26T07:12:19.5213485Z - CUDA Runtime 12.1 2024-06-26T07:12:19.5214007Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-06-26T07:12:19.5214601Z - CuDNN 90.1 (built against CUDA 12.4) 2024-06-26T07:12:19.5215042Z - Magma 2.6.1 2024-06-26T07:12:19.5222181Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-06-26T07:12:19.5228812Z 2024-06-26T07:12:19.7947388Z + cd test 2024-06-26T07:12:19.7948078Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-06-26T07:12:21.0077332Z ATen/Parallel: 2024-06-26T07:12:21.0077878Z at::get_num_threads() : 8 2024-06-26T07:12:21.0078377Z at::get_num_interop_threads() : 16 2024-06-26T07:12:21.0078930Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:12:21.0079457Z omp_get_max_threads() : 8 2024-06-26T07:12:21.0080658Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:12:21.0081481Z mkl_get_max_threads() : 8 2024-06-26T07:12:21.0082098Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:12:21.0083005Z std::thread::hardware_concurrency() : 16 2024-06-26T07:12:21.0083431Z Environment variables: 2024-06-26T07:12:21.0083780Z OMP_NUM_THREADS : [not set] 2024-06-26T07:12:21.0084157Z MKL_NUM_THREADS : [not set] 2024-06-26T07:12:21.0084578Z ATen parallel backend: OpenMP 2024-06-26T07:12:21.0084831Z 2024-06-26T07:12:21.2490420Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-06-26T07:12:21.2491547Z + [[ aot_eager_timm == *backward* ]] 2024-06-26T07:12:21.2492098Z + [[ aot_eager_timm == *xla* ]] 2024-06-26T07:12:21.2492617Z + [[ aot_eager_timm == *executorch* ]] 2024-06-26T07:12:21.2493179Z + [[ aot_eager_timm == \j\i\t\_\l\e\g\a\c\y ]] 2024-06-26T07:12:21.2493972Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-06-26T07:12:21.2494680Z + [[ aot_eager_timm == distributed ]] 2024-06-26T07:12:21.2495275Z + [[ aot_eager_timm == *inductor_distributed* ]] 2024-06-26T07:12:21.2495965Z + [[ aot_eager_timm == *inductor-halide* ]] 2024-06-26T07:12:21.2496686Z + [[ aot_eager_timm == *inductor-micro-benchmark* ]] 2024-06-26T07:12:21.2497376Z + [[ aot_eager_timm == *huggingface* ]] 2024-06-26T07:12:21.2497948Z + [[ aot_eager_timm == *timm* ]] 2024-06-26T07:12:21.2498470Z + install_torchvision 2024-06-26T07:12:21.2498909Z + local orig_preload 2024-06-26T07:12:21.2499336Z + local commit 2024-06-26T07:12:21.2499771Z ++ get_pinned_commit vision 2024-06-26T07:12:21.2500329Z ++ cat .github/ci_commit_pins/vision.txt 2024-06-26T07:12:21.2505846Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:12:21.2506434Z + orig_preload= 2024-06-26T07:12:21.2506886Z + '[' -n '' ']' 2024-06-26T07:12:21.2507812Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:12:21.2509224Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:12:21.5639585Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:12:21.5643346Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-l31y6fx6 2024-06-26T07:12:21.5660304Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-l31y6fx6 2024-06-26T07:12:22.9719542Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-06-26T07:12:22.9732864Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:12:24.2003422Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:12:24.4213076Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:12:26.7069072Z Preparing metadata (setup.py) ... [?25l- \ done 2024-06-26T07:12:26.7117190Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2) 2024-06-26T07:12:26.7120271Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+gitb8c4c54) 2024-06-26T07:12:26.7125943Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-06-26T07:12:26.7328332Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-06-26T07:12:26.7334909Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-06-26T07:12:26.7337742Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.12.1) 2024-06-26T07:12:26.7340713Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-06-26T07:12:26.7342924Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-06-26T07:12:26.7345946Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.2.0) 2024-06-26T07:12:26.7773134Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-06-26T07:12:26.7928989Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-06-26T07:12:26.8004157Z Building wheels for collected packages: torchvision 2024-06-26T07:13:40.1703182Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / done 2024-06-26T07:13:40.1729536Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2112977 sha256=da788fb323ac9ec0ddd37cdbc31c42e50ba8ea6b007a4dbac4bc53ea01423111 2024-06-26T07:13:40.1731678Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-06-26T07:13:40.1765988Z Successfully built torchvision 2024-06-26T07:13:41.3840881Z Installing collected packages: torchvision 2024-06-26T07:13:41.7651992Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-06-26T07:13:41.8745738Z + '[' -n '' ']' 2024-06-26T07:13:41.8748608Z + id=0 2024-06-26T07:13:41.8750213Z + test_dynamo_benchmark timm_models 0 2024-06-26T07:13:41.8751050Z ++ pwd 2024-06-26T07:13:41.8751731Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:13:41.8752258Z + local suite=timm_models 2024-06-26T07:13:41.8752605Z + shift 2024-06-26T07:13:41.8752878Z + local shard_id=0 2024-06-26T07:13:41.8754206Z + shift 2024-06-26T07:13:41.8754613Z + [[ aot_eager_timm == *perf_compare* ]] 2024-06-26T07:13:41.8755134Z + [[ aot_eager_timm == *perf* ]] 2024-06-26T07:13:41.8755552Z + [[ aot_eager_timm == *cpu_inductor* ]] 2024-06-26T07:13:41.8756081Z + [[ aot_eager_timm == *aot_inductor* ]] 2024-06-26T07:13:41.8756845Z + test_single_dynamo_benchmark inference timm_models 0 --inference --bfloat16 2024-06-26T07:13:41.8757436Z ++ pwd 2024-06-26T07:13:41.8757931Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:13:41.8758650Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:13:41.8764794Z + local name=inference 2024-06-26T07:13:41.8765218Z + shift 2024-06-26T07:13:41.8765502Z + local suite=timm_models 2024-06-26T07:13:41.8765907Z + shift 2024-06-26T07:13:41.8766279Z + local shard_id=0 2024-06-26T07:13:41.8766686Z + shift 2024-06-26T07:13:41.8767066Z + partition_flags=() 2024-06-26T07:13:41.8767510Z + local partition_flags 2024-06-26T07:13:41.8803203Z + [[ -n 2 ]] 2024-06-26T07:13:41.8803638Z + [[ -n 0 ]] 2024-06-26T07:13:41.8804312Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-06-26T07:13:41.8804981Z + [[ aot_eager_timm == *perf_compare* ]] 2024-06-26T07:13:41.8805412Z + [[ aot_eager_timm == *perf* ]] 2024-06-26T07:13:41.8805814Z + [[ aot_eager_timm == *aot_inductor* ]] 2024-06-26T07:13:41.8807386Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --backend aot_eager --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv 2024-06-26T07:13:47.6670483Z 2024-06-26T07:13:48.0345283Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:13:48.0345733Z 2024-06-26T07:13:48.1423740Z model.safetensors: 0% 0.00/95.5M [00:00 will be ignored 2024-06-26T07:22:49.9603115Z pass 2024-06-26T07:22:50.0457247Z TIMING: entire_frame_compile:27.13724 backend_compile:16.78706 2024-06-26T07:22:50.0460127Z STATS: call_* op count: 1028 | FakeTensor.__torch_dispatch__:4504 | FakeTensorMode.__torch_dispatch__:54077 | ProxyTorchDispatchMode.__torch_dispatch__:18627 2024-06-26T07:22:50.0461257Z Dynamo produced 3 graphs covering 1028 ops with 7 graph breaks (5 unique) 2024-06-26T07:22:54.2165236Z 2024-06-26T07:22:54.9878382Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:54.9878839Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:54.9882503Z cuda train botnet26t_256 2024-06-26T07:23:03.8864469Z pass 2024-06-26T07:23:03.9037662Z TIMING: entire_frame_compile:7.41737 backend_compile:3.98832 2024-06-26T07:23:03.9038817Z STATS: call_* op count: 341 | FakeTensor.__torch_dispatch__:262 | FakeTensorMode.__torch_dispatch__:19797 | ProxyTorchDispatchMode.__torch_dispatch__:5527 2024-06-26T07:23:03.9040038Z Dynamo produced 2 graphs covering 341 ops with 6 graph breaks (5 unique) 2024-06-26T07:23:07.6519169Z 2024-06-26T07:23:11.3746266Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:11.3746754Z loading model: 0it [00:03, ?it/s] 2024-06-26T07:23:11.3747192Z cuda train cait_m36_384 2024-06-26T07:23:11.3751916Z Traceback (most recent call last): 2024-06-26T07:23:11.3752899Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2321, in validate_model 2024-06-26T07:23:11.3753661Z self.model_iter_fn(model, example_inputs) 2024-06-26T07:23:11.3754784Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 371, in forward_and_backward_pass 2024-06-26T07:23:11.3755798Z pred = mod(*cloned_inputs) 2024-06-26T07:23:11.3756960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T07:23:11.3758062Z return self._call_impl(*args, **kwargs) 2024-06-26T07:23:11.3758976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:23:11.3759798Z return forward_call(*args, **kwargs) 2024-06-26T07:23:11.3760617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 361, in forward 2024-06-26T07:23:11.3761417Z x = self.forward_features(x) 2024-06-26T07:23:11.3762745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 346, in forward_features 2024-06-26T07:23:11.3763633Z x = self.blocks(x) 2024-06-26T07:23:11.3764806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T07:23:11.3765936Z return self._call_impl(*args, **kwargs) 2024-06-26T07:23:11.3767309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:23:11.3768671Z return forward_call(*args, **kwargs) 2024-06-26T07:23:11.3769608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/container.py", line 249, in forward 2024-06-26T07:23:11.3770470Z input = module(input) 2024-06-26T07:23:11.3771352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T07:23:11.3772210Z return self._call_impl(*args, **kwargs) 2024-06-26T07:23:11.3773137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:23:11.3773947Z return forward_call(*args, **kwargs) 2024-06-26T07:23:11.3774761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 197, in forward 2024-06-26T07:23:11.3775613Z x = x + self.drop_path(self.gamma_1 * self.attn(self.norm1(x))) 2024-06-26T07:23:11.3776715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T07:23:11.3777580Z return self._call_impl(*args, **kwargs) 2024-06-26T07:23:11.3778482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:23:11.3779299Z return forward_call(*args, **kwargs) 2024-06-26T07:23:11.3780107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 151, in forward 2024-06-26T07:23:11.3781033Z x = (attn @ v).transpose(1, 2).reshape(B, N, C) 2024-06-26T07:23:11.3784311Z torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 21.98 GiB of which 16.44 MiB is free. Process 31836 has 21.95 GiB memory in use. Of the allocated memory 21.21 GiB is allocated by PyTorch, and 430.66 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) 2024-06-26T07:23:11.3786937Z 2024-06-26T07:23:11.3787231Z The above exception was the direct cause of the following exception: 2024-06-26T07:23:11.3787674Z 2024-06-26T07:23:11.3787816Z Traceback (most recent call last): 2024-06-26T07:23:11.3788456Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4229, in run 2024-06-26T07:23:11.3789103Z ) = runner.load_model( 2024-06-26T07:23:11.3789743Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 303, in load_model 2024-06-26T07:23:11.3790486Z self.validate_model(model, example_inputs) 2024-06-26T07:23:11.3791222Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2323, in validate_model 2024-06-26T07:23:11.3791966Z raise RuntimeError("Eager run failed") from e 2024-06-26T07:23:11.3792425Z RuntimeError: Eager run failed 2024-06-26T07:23:11.3792678Z 2024-06-26T07:23:11.3792791Z eager_fail_to_run 2024-06-26T07:23:14.8838793Z 2024-06-26T07:23:15.5816518Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:15.5817228Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:15.5817891Z cuda train coat_lite_mini 2024-06-26T07:23:25.7694318Z pass 2024-06-26T07:23:25.7884823Z TIMING: entire_frame_compile:8.8048 backend_compile:6.00299 2024-06-26T07:23:25.7885942Z STATS: call_* op count: 480 | FakeTensor.__torch_dispatch__:967 | FakeTensorMode.__torch_dispatch__:28693 | ProxyTorchDispatchMode.__torch_dispatch__:10627 2024-06-26T07:23:25.7887057Z Dynamo produced 2 graphs covering 480 ops with 6 graph breaks (5 unique) 2024-06-26T07:23:29.6316913Z 2024-06-26T07:23:31.0752224Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:31.0753114Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:23:31.0753663Z cuda train convit_base 2024-06-26T07:23:31.1071418Z WARNING:common:fp64 golden ref were not generated for convit_base. Setting accuracy check to cosine 2024-06-26T07:23:52.0428405Z pass 2024-06-26T07:23:52.0903055Z TIMING: entire_frame_compile:19.97149 backend_compile:14.0111 2024-06-26T07:23:52.0904179Z STATS: call_* op count: 1216 | FakeTensor.__torch_dispatch__:1617 | FakeTensorMode.__torch_dispatch__:69291 | ProxyTorchDispatchMode.__torch_dispatch__:28022 2024-06-26T07:23:52.0907473Z Dynamo produced 3 graphs covering 1216 ops with 7 graph breaks (5 unique) 2024-06-26T07:23:55.8480583Z 2024-06-26T07:23:56.5591557Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:56.5592197Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:56.5592666Z cuda train convmixer_768_32 2024-06-26T07:24:11.0812879Z pass 2024-06-26T07:24:11.1401371Z TIMING: entire_frame_compile:8.40382 backend_compile:6.02362 2024-06-26T07:24:11.1403020Z STATS: call_* op count: 235 | FakeTensor.__torch_dispatch__:1603 | FakeTensorMode.__torch_dispatch__:35503 | ProxyTorchDispatchMode.__torch_dispatch__:9010 2024-06-26T07:24:11.1404126Z Dynamo produced 2 graphs covering 235 ops with 5 graph breaks (4 unique) 2024-06-26T07:24:14.8559763Z 2024-06-26T07:24:16.3784138Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:16.3785162Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:24:16.3786156Z cuda train convnext_base 2024-06-26T07:24:33.0851484Z W0626 07:24:33.084000 140515260904064 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:25:08.2609391Z pass 2024-06-26T07:25:08.3442918Z TIMING: entire_frame_compile:46.62241 backend_compile:27.98857 2024-06-26T07:25:08.3444639Z STATS: call_* op count: 1621 | FakeTensor.__torch_dispatch__:8040 | FakeTensorMode.__torch_dispatch__:87733 | ProxyTorchDispatchMode.__torch_dispatch__:29746 2024-06-26T07:25:08.3445775Z Dynamo produced 3 graphs covering 1621 ops with 7 graph breaks (5 unique) 2024-06-26T07:25:12.9207654Z 2024-06-26T07:25:13.5474103Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:13.5476582Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:13.5477048Z cuda train crossvit_9_240 2024-06-26T07:25:26.1066919Z W0626 07:25:26.105000 140502158484096 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:25:52.3534352Z pass 2024-06-26T07:25:52.3711515Z TIMING: entire_frame_compile:36.81989 backend_compile:23.15865 2024-06-26T07:25:52.3713488Z STATS: call_* op count: 1403 | FakeTensor.__torch_dispatch__:6968 | FakeTensorMode.__torch_dispatch__:80184 | ProxyTorchDispatchMode.__torch_dispatch__:27501 2024-06-26T07:25:52.3714636Z Dynamo produced 3 graphs covering 1403 ops with 7 graph breaks (5 unique) 2024-06-26T07:25:56.7951041Z 2024-06-26T07:25:57.7812064Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:57.7812541Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:57.7812978Z cuda train cspdarknet53 2024-06-26T07:26:13.4205446Z W0626 07:26:13.419000 140241360470656 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:26:36.4855312Z pass 2024-06-26T07:26:36.5148631Z TIMING: entire_frame_compile:35.44788 backend_compile:19.27888 2024-06-26T07:26:36.5152962Z STATS: call_* op count: 1070 | FakeTensor.__torch_dispatch__:4444 | FakeTensorMode.__torch_dispatch__:62448 | ProxyTorchDispatchMode.__torch_dispatch__:17131 2024-06-26T07:26:36.5154550Z Dynamo produced 3 graphs covering 1070 ops with 7 graph breaks (5 unique) 2024-06-26T07:26:40.7876978Z 2024-06-26T07:26:42.1618898Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:42.1619495Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:26:42.1622331Z cuda train deit_base_distilled_patch16_224 2024-06-26T07:26:52.5016807Z W0626 07:26:52.500000 140427075031680 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:27:07.6866363Z pass 2024-06-26T07:27:07.7782557Z TIMING: entire_frame_compile:20.61838 backend_compile:12.75793 2024-06-26T07:27:07.7784474Z STATS: call_* op count: 841 | FakeTensor.__torch_dispatch__:4082 | FakeTensorMode.__torch_dispatch__:43343 | ProxyTorchDispatchMode.__torch_dispatch__:14631 2024-06-26T07:27:07.7785605Z Dynamo produced 3 graphs covering 841 ops with 7 graph breaks (5 unique) 2024-06-26T07:27:11.8048577Z 2024-06-26T07:27:12.9486974Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:27:12.9487433Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:27:12.9487890Z cuda train dla102 2024-06-26T07:27:29.1251239Z W0626 07:27:29.124000 139881779917440 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:28:04.6908317Z pass 2024-06-26T07:28:04.7201808Z TIMING: entire_frame_compile:47.92444 backend_compile:29.04989 2024-06-26T07:28:04.7204105Z STATS: call_* op count: 1389 | FakeTensor.__torch_dispatch__:8125 | FakeTensorMode.__torch_dispatch__:98447 | ProxyTorchDispatchMode.__torch_dispatch__:26824 2024-06-26T07:28:04.7205242Z Dynamo produced 3 graphs covering 1389 ops with 7 graph breaks (5 unique) 2024-06-26T07:28:09.3272039Z 2024-06-26T07:28:10.8134770Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:28:10.8135278Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:28:10.8139607Z cuda train dm_nfnet_f0 2024-06-26T07:28:27.8047452Z pass 2024-06-26T07:28:27.8555400Z TIMING: entire_frame_compile:13.83811 backend_compile:7.10638 2024-06-26T07:28:27.8557979Z STATS: call_* op count: 620 | FakeTensorMode.__torch_dispatch__:31690 | FakeTensor.__torch_dispatch__:334 | ProxyTorchDispatchMode.__torch_dispatch__:10519 2024-06-26T07:28:27.8560168Z Dynamo produced 2 graphs covering 620 ops with 6 graph breaks (5 unique) 2024-06-26T07:28:31.7005662Z 2024-06-26T07:28:33.0320134Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:28:33.0320646Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:28:33.0321143Z cuda train dpn107 2024-06-26T07:29:03.9071067Z pass 2024-06-26T07:29:03.9828055Z TIMING: entire_frame_compile:23.94994 backend_compile:13.40486 2024-06-26T07:29:03.9830465Z STATS: call_* op count: 746 | FakeTensor.__torch_dispatch__:820 | FakeTensorMode.__torch_dispatch__:61879 | ProxyTorchDispatchMode.__torch_dispatch__:16915 2024-06-26T07:29:03.9831593Z Dynamo produced 2 graphs covering 746 ops with 6 graph breaks (5 unique) 2024-06-26T07:29:08.1603489Z 2024-06-26T07:29:08.9154633Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:29:18.4588594Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:29:18.4589300Z cuda train eca_botnext26ts_256 2024-06-26T07:29:18.4591118Z W0626 07:29:18.458000 139857891336832 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:29:30.6373851Z pass 2024-06-26T07:29:30.6527045Z TIMING: entire_frame_compile:20.17116 backend_compile:11.6924 2024-06-26T07:29:30.6530859Z STATS: call_* op count: 707 | FakeTensor.__torch_dispatch__:2470 | FakeTensorMode.__torch_dispatch__:37465 | ProxyTorchDispatchMode.__torch_dispatch__:11488 2024-06-26T07:29:30.6532081Z Dynamo produced 3 graphs covering 707 ops with 7 graph breaks (5 unique) 2024-06-26T07:29:34.6127788Z 2024-06-26T07:29:35.3831971Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:29:35.3833846Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:29:35.3834309Z cuda train eca_halonext26ts 2024-06-26T07:29:45.2512058Z W0626 07:29:45.250000 139686015722112 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:29:57.8750414Z pass 2024-06-26T07:29:57.8888276Z TIMING: entire_frame_compile:20.90124 backend_compile:11.89872 2024-06-26T07:29:57.8889708Z STATS: call_* op count: 740 | FakeTensor.__torch_dispatch__:2541 | FakeTensorMode.__torch_dispatch__:38705 | ProxyTorchDispatchMode.__torch_dispatch__:11921 2024-06-26T07:29:57.8890830Z Dynamo produced 3 graphs covering 740 ops with 7 graph breaks (5 unique) 2024-06-26T07:30:01.8439007Z 2024-06-26T07:30:02.4929865Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:30:02.4930620Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:30:02.4931094Z cuda train ese_vovnet19b_dw 2024-06-26T07:30:08.5901382Z W0626 07:30:08.589000 139817348883072 torch/_logging/_internal.py:1040] [7/0] Profiler function will be ignored 2024-06-26T07:30:19.9933988Z pass 2024-06-26T07:30:20.0008252Z TIMING: entire_frame_compile:16.51938 backend_compile:8.97637 2024-06-26T07:30:20.0009308Z STATS: call_* op count: 459 | FakeTensor.__torch_dispatch__:2244 | FakeTensorMode.__torch_dispatch__:27656 | ProxyTorchDispatchMode.__torch_dispatch__:7887 2024-06-26T07:30:20.0010409Z Dynamo produced 4 graphs covering 459 ops with 7 graph breaks (5 unique) 2024-06-26T07:30:23.9246989Z 2024-06-26T07:30:24.6447502Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:30:24.6447966Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:30:24.6450912Z cuda train fbnetc_100 2024-06-26T07:30:36.5412075Z W0626 07:30:36.540000 139879259083392 torch/_logging/_internal.py:1040] [7/0] Profiler function will be ignored 2024-06-26T07:30:58.7182908Z pass 2024-06-26T07:30:58.7252812Z TIMING: entire_frame_compile:32.78413 backend_compile:18.08153 2024-06-26T07:30:58.7254791Z STATS: call_* op count: 993 | FakeTensor.__torch_dispatch__:4430 | FakeTensorMode.__torch_dispatch__:59216 | ProxyTorchDispatchMode.__torch_dispatch__:16037 2024-06-26T07:30:58.7256204Z Dynamo produced 4 graphs covering 993 ops with 7 graph breaks (5 unique) 2024-06-26T07:31:03.0986625Z 2024-06-26T07:31:03.9902399Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:31:03.9902873Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:31:03.9903304Z cuda train fbnetv3_b 2024-06-26T07:31:22.6606126Z pass 2024-06-26T07:31:22.6735236Z TIMING: entire_frame_compile:17.15293 backend_compile:9.37716 2024-06-26T07:31:22.6736299Z STATS: call_* op count: 610 | FakeTensor.__torch_dispatch__:1116 | FakeTensorMode.__torch_dispatch__:46899 | ProxyTorchDispatchMode.__torch_dispatch__:11990 2024-06-26T07:31:22.6737403Z Dynamo produced 2 graphs covering 610 ops with 6 graph breaks (5 unique) 2024-06-26T07:31:26.6423994Z 2024-06-26T07:31:27.6448018Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:31:27.6448472Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:31:27.6448904Z cuda train gernet_l 2024-06-26T07:31:40.3327465Z pass 2024-06-26T07:31:40.3548015Z TIMING: entire_frame_compile:10.60264 backend_compile:5.3082 2024-06-26T07:31:40.3550462Z STATS: call_* op count: 410 | FakeTensor.__torch_dispatch__:410 | FakeTensorMode.__torch_dispatch__:26799 | ProxyTorchDispatchMode.__torch_dispatch__:6598 2024-06-26T07:31:40.3551601Z Dynamo produced 2 graphs covering 410 ops with 6 graph breaks (5 unique) 2024-06-26T07:31:44.1803098Z 2024-06-26T07:31:44.9517432Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:31:44.9517901Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:31:44.9518339Z cuda train ghostnet_100 2024-06-26T07:31:58.4435576Z pass 2024-06-26T07:31:58.4498857Z TIMING: entire_frame_compile:12.312 backend_compile:8.52158 2024-06-26T07:31:58.4499911Z STATS: call_* op count: 360 | FakeTensor.__torch_dispatch__:1764 | FakeTensorMode.__torch_dispatch__:45995 | ProxyTorchDispatchMode.__torch_dispatch__:11967 2024-06-26T07:31:58.4501034Z Dynamo produced 2 graphs covering 360 ops with 6 graph breaks (5 unique) 2024-06-26T07:32:02.3236524Z 2024-06-26T07:32:03.3855472Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:32:03.3856065Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:32:03.3856619Z cuda train gluon_inception_v3 2024-06-26T07:32:23.3489932Z W0626 07:32:23.348000 139728692945536 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:32:55.0973611Z pass 2024-06-26T07:32:55.1205330Z TIMING: entire_frame_compile:48.35074 backend_compile:27.04817 2024-06-26T07:32:55.1208142Z STATS: call_* op count: 1376 | FakeTensor.__torch_dispatch__:6261 | FakeTensorMode.__torch_dispatch__:86959 | ProxyTorchDispatchMode.__torch_dispatch__:24031 2024-06-26T07:32:55.1209340Z Dynamo produced 3 graphs covering 1376 ops with 7 graph breaks (5 unique) 2024-06-26T07:32:59.7913139Z 2024-06-26T07:33:00.6511117Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:33:00.6511598Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:33:00.6512027Z cuda train gmixer_24_224 2024-06-26T07:33:14.0870136Z pass 2024-06-26T07:33:14.1043958Z TIMING: entire_frame_compile:11.03986 backend_compile:7.70831 2024-06-26T07:33:14.1045167Z STATS: call_* op count: 684 | FakeTensor.__torch_dispatch__:1991 | FakeTensorMode.__torch_dispatch__:41974 | ProxyTorchDispatchMode.__torch_dispatch__:16426 2024-06-26T07:33:14.1046600Z Dynamo produced 2 graphs covering 684 ops with 6 graph breaks (5 unique) 2024-06-26T07:33:17.8642759Z 2024-06-26T07:33:18.5752061Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:33:18.5752552Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:33:18.5752994Z cuda train gmlp_s16_224 2024-06-26T07:33:31.0695360Z W0626 07:33:31.068000 139704392340096 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:33:59.9752000Z pass 2024-06-26T07:33:59.9998107Z TIMING: entire_frame_compile:38.65858 backend_compile:24.48354 2024-06-26T07:33:59.9999428Z STATS: call_* op count: 1458 | FakeTensor.__torch_dispatch__:7847 | FakeTensorMode.__torch_dispatch__:84123 | ProxyTorchDispatchMode.__torch_dispatch__:28113 2024-06-26T07:34:00.0005542Z Dynamo produced 3 graphs covering 1458 ops with 7 graph breaks (5 unique) 2024-06-26T07:34:04.4281925Z 2024-06-26T07:34:06.0266201Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:34:06.0266759Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:34:06.0267204Z cuda train hrnet_w18 2024-06-26T07:34:51.4860856Z pass 2024-06-26T07:34:51.5080062Z TIMING: entire_frame_compile:40.14121 backend_compile:27.79679 2024-06-26T07:34:51.5082199Z STATS: call_* op count: 1394 | FakeTensor.__torch_dispatch__:5985 | FakeTensorMode.__torch_dispatch__:161604 | ProxyTorchDispatchMode.__torch_dispatch__:39732 2024-06-26T07:34:51.5083590Z Dynamo produced 2 graphs covering 1394 ops with 5 graph breaks (4 unique) 2024-06-26T07:34:55.9614875Z 2024-06-26T07:34:57.0404144Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:34:57.0404617Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:34:57.0405047Z cuda train inception_v3 2024-06-26T07:35:16.8735927Z pass 2024-06-26T07:35:16.8965423Z TIMING: entire_frame_compile:17.26675 backend_compile:9.15566 2024-06-26T07:35:16.8966521Z STATS: call_* op count: 506 | FakeTensor.__torch_dispatch__:665 | FakeTensorMode.__torch_dispatch__:44862 | ProxyTorchDispatchMode.__torch_dispatch__:11234 2024-06-26T07:35:16.8967627Z Dynamo produced 2 graphs covering 506 ops with 6 graph breaks (5 unique) 2024-06-26T07:35:20.9344055Z 2024-06-26T07:35:22.2739796Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:35:22.2740467Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:35:22.2741004Z cuda train jx_nest_base 2024-06-26T07:35:43.7039590Z W0626 07:35:43.702000 139883472413312 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:36:13.7863478Z pass 2024-06-26T07:36:13.8449173Z TIMING: entire_frame_compile:45.33232 backend_compile:28.11336 2024-06-26T07:36:13.8451080Z STATS: call_* op count: 1700 | FakeTensor.__torch_dispatch__:7156 | FakeTensorMode.__torch_dispatch__:95406 | ProxyTorchDispatchMode.__torch_dispatch__:33243 2024-06-26T07:36:13.8452215Z Dynamo produced 3 graphs covering 1700 ops with 7 graph breaks (5 unique) 2024-06-26T07:36:18.4494294Z 2024-06-26T07:36:19.0097881Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:36:19.0098763Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:36:19.0099859Z cuda train lcnet_050 2024-06-26T07:36:24.6343232Z E0626 07:36:24.633000 140180953285248 torch/_dynamo/utils.py:1470] RMSE (res-fp64): 0.00769, (ref-fp64): 0.00017 and shape=torch.Size([32]). res.dtype: torch.float32, multiplier: 3.000000, tol: 0.010000 2024-06-26T07:36:24.6344828Z E0626 07:36:24.633000 140180953285248 torch/_dynamo/utils.py:1384] Accuracy failed for key name blocks.1.0.bn2.weight.grad 2024-06-26T07:36:24.6357643Z fail_accuracy 2024-06-26T07:36:24.6378086Z TIMING: entire_frame_compile:5.17451 backend_compile:2.79185 2024-06-26T07:36:24.6379115Z STATS: call_* op count: 168 | FakeTensor.__torch_dispatch__:305 | FakeTensorMode.__torch_dispatch__:13926 | ProxyTorchDispatchMode.__torch_dispatch__:3329 2024-06-26T07:36:24.6380212Z Dynamo produced 2 graphs covering 168 ops with 6 graph breaks (5 unique) 2024-06-26T07:36:28.3997845Z 2024-06-26T07:36:29.2022309Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:36:29.2023295Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:36:29.2024470Z cuda train levit_128 2024-06-26T07:36:43.3727171Z W0626 07:36:43.371000 139833154654848 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:37:03.9924509Z pass 2024-06-26T07:37:03.9971291Z TIMING: entire_frame_compile:33.28334 backend_compile:20.79436 2024-06-26T07:37:03.9973876Z STATS: call_* op count: 1225 | FakeTensor.__torch_dispatch__:5301 | FakeTensorMode.__torch_dispatch__:77191 | ProxyTorchDispatchMode.__torch_dispatch__:23437 2024-06-26T07:37:03.9975359Z Dynamo produced 3 graphs covering 1225 ops with 7 graph breaks (5 unique) 2024-06-26T07:37:08.4525322Z 2024-06-26T07:37:09.5109659Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:37:09.5110376Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:37:09.5110852Z cuda train mixer_b16_224 2024-06-26T07:37:18.1957693Z W0626 07:37:18.195000 140549172109952 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:37:32.6533305Z pass 2024-06-26T07:37:32.7206655Z TIMING: entire_frame_compile:19.34879 backend_compile:12.20314 2024-06-26T07:37:32.7208760Z STATS: call_* op count: 720 | FakeTensor.__torch_dispatch__:3928 | FakeTensorMode.__torch_dispatch__:40865 | ProxyTorchDispatchMode.__torch_dispatch__:13749 2024-06-26T07:37:32.7210429Z Dynamo produced 3 graphs covering 720 ops with 7 graph breaks (5 unique) 2024-06-26T07:37:33.8356133Z accuracy pass_rate=93.33% 2024-06-26T07:37:33.8359319Z calls_captured gmean=0.00x mean=833.533x 2024-06-26T07:37:33.8362836Z unique_graphs gmean=0.00x mean=2.533x 2024-06-26T07:37:33.8366511Z graph_breaks gmean=0.00x mean=6.267x 2024-06-26T07:37:33.8369989Z unique_graph_breaks gmean=0.00x mean=4.767x 2024-06-26T07:37:33.8373070Z autograd_captures gmean=0.00x mean=0.000x 2024-06-26T07:37:33.8376477Z autograd_compiles gmean=0.00x mean=0.000x 2024-06-26T07:37:33.8379803Z cudagraph_skips gmean=0.00x mean=0.000x 2024-06-26T07:37:34.4212275Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy//aot_eager_timm_training.csv 2024-06-26T07:37:34.6571338Z adv_inception_v3 PASS 2024-06-26T07:37:34.6575311Z beit_base_patch16_224 PASS 2024-06-26T07:37:34.6580672Z botnet26t_256 PASS 2024-06-26T07:37:34.6586041Z cait_m36_384 XFAIL 2024-06-26T07:37:34.6592372Z coat_lite_mini PASS 2024-06-26T07:37:34.6597087Z convit_base PASS 2024-06-26T07:37:34.6602724Z convmixer_768_32 PASS 2024-06-26T07:37:34.6608027Z convnext_base PASS 2024-06-26T07:37:34.6613517Z crossvit_9_240 PASS 2024-06-26T07:37:34.6618444Z cspdarknet53 PASS 2024-06-26T07:37:34.6623632Z deit_base_distilled_patch16_224 PASS 2024-06-26T07:37:34.6629002Z dla102 PASS 2024-06-26T07:37:34.6634188Z dm_nfnet_f0 PASS 2024-06-26T07:37:34.6639432Z dpn107 PASS 2024-06-26T07:37:34.6644862Z eca_botnext26ts_256 PASS 2024-06-26T07:37:34.6650061Z eca_halonext26ts PASS 2024-06-26T07:37:34.6655259Z ese_vovnet19b_dw PASS 2024-06-26T07:37:34.6660485Z fbnetc_100 PASS 2024-06-26T07:37:34.6665771Z fbnetv3_b PASS 2024-06-26T07:37:34.6671043Z gernet_l PASS 2024-06-26T07:37:34.6676245Z ghostnet_100 PASS 2024-06-26T07:37:34.6681438Z gluon_inception_v3 PASS 2024-06-26T07:37:34.6686945Z gmixer_24_224 PASS 2024-06-26T07:37:34.6692106Z gmlp_s16_224 PASS 2024-06-26T07:37:34.6697259Z hrnet_w18 PASS 2024-06-26T07:37:34.6702383Z inception_v3 PASS 2024-06-26T07:37:34.6707978Z jx_nest_base PASS 2024-06-26T07:37:34.6713187Z lcnet_050 XFAIL 2024-06-26T07:37:34.6718328Z levit_128 PASS 2024-06-26T07:37:34.6723651Z mixer_b16_224 PASS 2024-06-26T07:37:34.7099135Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy//aot_eager_timm_training.csv 2024-06-26T07:37:34.9633536Z adv_inception_v3 PASS 2024-06-26T07:37:34.9637796Z beit_base_patch16_224 PASS 2024-06-26T07:37:34.9642431Z botnet26t_256 PASS 2024-06-26T07:37:34.9646974Z cait_m36_384 PASS 2024-06-26T07:37:34.9651324Z coat_lite_mini PASS 2024-06-26T07:37:34.9655749Z convit_base PASS 2024-06-26T07:37:34.9660159Z convmixer_768_32 PASS 2024-06-26T07:37:34.9664445Z convnext_base PASS 2024-06-26T07:37:34.9669221Z crossvit_9_240 PASS 2024-06-26T07:37:34.9673407Z cspdarknet53 PASS 2024-06-26T07:37:34.9677891Z deit_base_distilled_patch16_224 PASS 2024-06-26T07:37:34.9682118Z dla102 PASS 2024-06-26T07:37:34.9686971Z dm_nfnet_f0 PASS 2024-06-26T07:37:34.9691330Z dpn107 PASS 2024-06-26T07:37:34.9695660Z eca_botnext26ts_256 PASS 2024-06-26T07:37:34.9700108Z eca_halonext26ts PASS 2024-06-26T07:37:34.9704476Z ese_vovnet19b_dw PASS 2024-06-26T07:37:34.9709072Z fbnetc_100 PASS 2024-06-26T07:37:34.9713292Z fbnetv3_b PASS 2024-06-26T07:37:34.9717591Z gernet_l PASS 2024-06-26T07:37:34.9722007Z ghostnet_100 PASS 2024-06-26T07:37:34.9726366Z gluon_inception_v3 PASS 2024-06-26T07:37:34.9730890Z gmixer_24_224 PASS 2024-06-26T07:37:34.9735173Z gmlp_s16_224 PASS 2024-06-26T07:37:34.9739581Z hrnet_w18 PASS 2024-06-26T07:37:34.9743750Z inception_v3 PASS 2024-06-26T07:37:34.9748155Z jx_nest_base PASS 2024-06-26T07:37:34.9752423Z lcnet_050 PASS 2024-06-26T07:37:34.9756741Z levit_128 PASS 2024-06-26T07:37:34.9761040Z mixer_b16_224 PASS 2024-06-26T07:37:35.0131012Z + cleanup_workspace 2024-06-26T07:37:35.0131933Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-06-26T07:37:35.0133261Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-06-26T07:37:35.0134263Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-06-26T07:37:35.0135000Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-06-26T07:37:35.0135797Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-06-26T07:37:35.0136664Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-06-26T07:37:35.0137347Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-06-26T07:37:35.5640695Z ##[group]Run cat test/**/*_toprint.log || true 2024-06-26T07:37:35.5641219Z cat test/**/*_toprint.log || true 2024-06-26T07:37:35.5649082Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:37:35.5649571Z env: 2024-06-26T07:37:35.5649846Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:35.5650294Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:35.5651008Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:35.5651632Z ##[endgroup] 2024-06-26T07:37:35.5713779Z cat: test/**/*_toprint.log: No such file or directory 2024-06-26T07:37:35.5741923Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-06-26T07:37:35.5742382Z kill "$MONITOR_SCRIPT_PID" 2024-06-26T07:37:35.5750121Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:37:35.5750627Z env: 2024-06-26T07:37:35.5750897Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:35.5751342Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:35.5752178Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:35.5752817Z MONITOR_SCRIPT_PID: 26369 2024-06-26T07:37:35.5753156Z ##[endgroup] 2024-06-26T07:37:35.5934197Z Prepare all required actions 2024-06-26T07:37:35.5934635Z Getting action download info 2024-06-26T07:37:35.7761753Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-06-26T07:37:35.9275483Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-06-26T07:37:35.9275943Z with: 2024-06-26T07:37:35.9276437Z file-suffix: test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688916604 2024-06-26T07:37:35.9277059Z s3-bucket: gha-artifacts 2024-06-26T07:37:35.9277387Z env: 2024-06-26T07:37:35.9277664Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:35.9278111Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:35.9278860Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:35.9279507Z ##[endgroup] 2024-06-26T07:37:35.9303811Z ##[group]Run # Remove any previous test jsons if they exist 2024-06-26T07:37:35.9304550Z # Remove any previous test jsons if they exist 2024-06-26T07:37:35.9305055Z rm -f test-jsons-*.zip 2024-06-26T07:37:35.9305569Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-06-26T07:37:35.9313382Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:37:35.9313871Z env: 2024-06-26T07:37:35.9314152Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:35.9314588Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:35.9315304Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:35.9316139Z FILE_SUFFIX: test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688916604 2024-06-26T07:37:35.9316731Z ##[endgroup] 2024-06-26T07:37:35.9485550Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-06-26T07:37:35.9514535Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-06-26T07:37:35.9515212Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-06-26T07:37:35.9521162Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-06-26T07:37:35.9547416Z ##[group]Run # Remove any previous test reports if they exist 2024-06-26T07:37:35.9548069Z # Remove any previous test reports if they exist 2024-06-26T07:37:35.9548592Z rm -f test-reports-*.zip 2024-06-26T07:37:35.9549178Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-06-26T07:37:35.9556861Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:37:35.9557346Z env: 2024-06-26T07:37:35.9557624Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:35.9558070Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:35.9558805Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:35.9559692Z FILE_SUFFIX: test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688916604 2024-06-26T07:37:35.9560292Z ##[endgroup] 2024-06-26T07:37:35.9712822Z adding: test/test-reports/inference_timm_models.csv (deflated 69%) 2024-06-26T07:37:35.9713622Z adding: test/test-reports/training_timm_models.csv (deflated 66%) 2024-06-26T07:37:35.9717281Z adding: test/test-reports/training_timm_model_graph_breaks.csv (deflated 97%) 2024-06-26T07:37:35.9718204Z adding: test/test-reports/training_timm_model_graph_break_deduped.csv (deflated 73%) 2024-06-26T07:37:35.9743144Z ##[group]Run # Remove any previous usage logs if they exist 2024-06-26T07:37:35.9743824Z # Remove any previous usage logs if they exist 2024-06-26T07:37:35.9744444Z rm -f logs-*.zip 2024-06-26T07:37:35.9745094Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-06-26T07:37:35.9745865Z # so check to see if the file exists first 2024-06-26T07:37:35.9746513Z if [ -f 'usage_log.txt' ]; then 2024-06-26T07:37:35.9747039Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-06-26T07:37:35.9747517Z fi 2024-06-26T07:37:35.9747969Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-06-26T07:37:35.9748542Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-06-26T07:37:35.9749080Z fi 2024-06-26T07:37:35.9756252Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:37:35.9756739Z env: 2024-06-26T07:37:35.9757012Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:35.9757471Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:35.9758184Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:35.9759016Z FILE_SUFFIX: test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688916604 2024-06-26T07:37:35.9759602Z ##[endgroup] 2024-06-26T07:37:35.9804104Z adding: usage_log.txt (deflated 92%) 2024-06-26T07:37:35.9868998Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-06-26T07:37:35.9869698Z # Remove any previous debugging artifacts if they exist 2024-06-26T07:37:35.9870251Z rm -f debug-*.zip 2024-06-26T07:37:35.9870636Z if [ -d 'test/debug' ]; then 2024-06-26T07:37:35.9871129Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-06-26T07:37:35.9871600Z fi 2024-06-26T07:37:35.9878214Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:37:35.9878704Z env: 2024-06-26T07:37:35.9878978Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:35.9879425Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:35.9880147Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:35.9880986Z FILE_SUFFIX: test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688916604 2024-06-26T07:37:35.9881583Z ##[endgroup] 2024-06-26T07:37:35.9955296Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T07:37:35.9955744Z with: 2024-06-26T07:37:35.9956030Z s3-bucket: gha-artifacts 2024-06-26T07:37:35.9956453Z s3-prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:37:35.9956936Z retention-days: 14 2024-06-26T07:37:35.9957266Z if-no-files-found: warn 2024-06-26T07:37:35.9957623Z path: test-jsons-*.zip 2024-06-26T07:37:35.9957964Z name: artifact 2024-06-26T07:37:35.9958260Z region: us-east-1 2024-06-26T07:37:35.9958561Z env: 2024-06-26T07:37:35.9958837Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:35.9959320Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:35.9960043Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:35.9960662Z ##[endgroup] 2024-06-26T07:37:36.2978326Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T07:37:36.2978950Z With the provided path, there will be 1 file uploaded 2024-06-26T07:37:36.2979588Z Uploading to s3 prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:37:36.3004929Z Starting upload of test-jsons-test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688916604.zip 2024-06-26T07:37:36.7429179Z Finished upload of test-jsons-test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688916604.zip 2024-06-26T07:37:36.7568478Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T07:37:36.7568925Z with: 2024-06-26T07:37:36.7569210Z s3-bucket: gha-artifacts 2024-06-26T07:37:36.7569631Z s3-prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:37:36.7570088Z retention-days: 14 2024-06-26T07:37:36.7570426Z if-no-files-found: error 2024-06-26T07:37:36.7570792Z path: test-reports-*.zip 2024-06-26T07:37:36.7571146Z name: artifact 2024-06-26T07:37:36.7571445Z region: us-east-1 2024-06-26T07:37:36.7571744Z env: 2024-06-26T07:37:36.7572018Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:36.7572457Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:36.7573175Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:36.7573932Z ##[endgroup] 2024-06-26T07:37:37.0665264Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T07:37:37.0667002Z With the provided path, there will be 1 file uploaded 2024-06-26T07:37:37.0668223Z Uploading to s3 prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:37:37.0691115Z Starting upload of test-reports-test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688916604.zip 2024-06-26T07:37:37.1664578Z Finished upload of test-reports-test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688916604.zip 2024-06-26T07:37:37.1791983Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T07:37:37.1792415Z with: 2024-06-26T07:37:37.1792695Z s3-bucket: gha-artifacts 2024-06-26T07:37:37.1793113Z s3-prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:37:37.1793579Z retention-days: 14 2024-06-26T07:37:37.1793906Z if-no-files-found: ignore 2024-06-26T07:37:37.1794260Z path: logs-*.zip 2024-06-26T07:37:37.1794566Z name: artifact 2024-06-26T07:37:37.1794875Z region: us-east-1 2024-06-26T07:37:37.1795172Z env: 2024-06-26T07:37:37.1795446Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:37.1795904Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:37.1796613Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:37.1797235Z ##[endgroup] 2024-06-26T07:37:37.4867060Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T07:37:37.4867657Z With the provided path, there will be 1 file uploaded 2024-06-26T07:37:37.4868272Z Uploading to s3 prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:37:37.4893653Z Starting upload of logs-test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688916604.zip 2024-06-26T07:37:37.6079877Z Finished upload of logs-test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688916604.zip 2024-06-26T07:37:37.6208939Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T07:37:37.6209379Z with: 2024-06-26T07:37:37.6209683Z s3-bucket: gha-artifacts 2024-06-26T07:37:37.6210108Z s3-prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:37:37.6210579Z retention-days: 14 2024-06-26T07:37:37.6210926Z if-no-files-found: ignore 2024-06-26T07:37:37.6211284Z path: debug-*.zip 2024-06-26T07:37:37.6211600Z name: artifact 2024-06-26T07:37:37.6211897Z region: us-east-1 2024-06-26T07:37:37.6212204Z env: 2024-06-26T07:37:37.6212561Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:37.6213078Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:37.6213797Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:37.6214427Z ##[endgroup] 2024-06-26T07:37:37.9169285Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-06-26T07:37:37.9438807Z ##[group]Run # shellcheck disable=SC2156 2024-06-26T07:37:37.9439393Z # shellcheck disable=SC2156 2024-06-26T07:37:37.9440183Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-06-26T07:37:37.9448487Z shell: /usr/bin/bash -e {0} 2024-06-26T07:37:37.9448836Z env: 2024-06-26T07:37:37.9449112Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:37.9449564Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:37.9450269Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:37.9450895Z ##[endgroup] 2024-06-26T07:37:38.1059461Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-06-26T07:37:38.1060025Z with: 2024-06-26T07:37:38.1060282Z env: 2024-06-26T07:37:38.1060552Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:38.1060991Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:38.1061703Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:38.1062321Z ##[endgroup] 2024-06-26T07:37:38.1079769Z ##[group]Run set -eou pipefail 2024-06-26T07:37:38.1080308Z set -eou pipefail 2024-06-26T07:37:38.1080654Z  2024-06-26T07:37:38.1081153Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-06-26T07:37:38.1081773Z for _ in $(seq 1440); do 2024-06-26T07:37:38.1082243Z  # Break if no ssh session exists anymore 2024-06-26T07:37:38.1082938Z  if [ "$(who)" = "" ]; then 2024-06-26T07:37:38.1083342Z  break 2024-06-26T07:37:38.1083678Z  fi 2024-06-26T07:37:38.1083975Z  echo "." 2024-06-26T07:37:38.1084301Z  sleep 5 2024-06-26T07:37:38.1084615Z done 2024-06-26T07:37:38.1092081Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:37:38.1092635Z env: 2024-06-26T07:37:38.1092952Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:38.1093427Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:38.1094156Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:38.1094781Z ##[endgroup] 2024-06-26T07:37:38.1115174Z Holding runner for 2 hours until all ssh sessions have logged out 2024-06-26T07:37:38.1148869Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:37:38.1149674Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:37:38.1150265Z # shellcheck disable=SC2046 2024-06-26T07:37:38.1150735Z docker stop $(docker ps -q) || true 2024-06-26T07:37:38.1151209Z # Prune all of the docker images 2024-06-26T07:37:38.1151660Z docker system prune -af 2024-06-26T07:37:38.1158343Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:37:38.1158841Z env: 2024-06-26T07:37:38.1159112Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:37:38.1159569Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:37:38.1160293Z DOCKER_CONTAINER_ID: 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:38.1160931Z ##[endgroup] 2024-06-26T07:37:38.4433097Z 9f605c1586ec 2024-06-26T07:37:39.1753702Z Deleted Containers: 2024-06-26T07:37:39.1754323Z 9f605c1586ec38f4e71898753672d0b26903d8de0bc11327f4dfc28aaeab1fb9 2024-06-26T07:37:39.1754802Z 2024-06-26T07:37:44.1223152Z Deleted Images: 2024-06-26T07:37:44.1224950Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:37:44.1227296Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb 2024-06-26T07:37:44.1229224Z deleted: sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd 2024-06-26T07:37:44.1230330Z deleted: sha256:754ead7012f0b7691885a9660f02e80be6cee8c5665486fb995c1fb06adb6e92 2024-06-26T07:37:44.1231344Z deleted: sha256:fbe4496d8830e11be86d8f222522e7956f32f85bfc06b05ea53961f34a58af74 2024-06-26T07:37:44.1232175Z deleted: sha256:1c83814aa1a1a7e0cc998e3479bdab8e8766b975560cbf00690acf39ed899a80 2024-06-26T07:37:44.1233006Z deleted: sha256:e9ea8e61942c7f8ae4aab97b8d102b01d6bb4b6cf2a605808cd34130a17d8a0f 2024-06-26T07:37:44.1233843Z deleted: sha256:acc5fc32588cd95e082a6ce35e48d0f303ae8f96e3ab65c4e375b85e1f3c4a92 2024-06-26T07:37:44.1234658Z deleted: sha256:a93e53400904d29e4185d3be942ea34f19572ea438962788ae4e9b3d9bf4c498 2024-06-26T07:37:44.1235636Z deleted: sha256:c97092b88b5e4c77c68bbc4b51afab64e38719af1ff58013ec49a696cdd40a6e 2024-06-26T07:37:44.1236490Z deleted: sha256:b132889c90d67ffa53655846603e3fcc2372c23f04dbdbc54690a39d304eb89b 2024-06-26T07:37:44.1237292Z deleted: sha256:915d54081ed607710624abe3f2490114c745858831f4fc13396bd3b4b9a121bc 2024-06-26T07:37:44.1238090Z deleted: sha256:bfa8d446c498c87f535aef102695564de1b22c755e6c63944ae2f5073a2b31ec 2024-06-26T07:37:44.1238896Z deleted: sha256:e71460fb363c29e70bbe357d680aaa7ae694f094ef2f2713609158ad90611744 2024-06-26T07:37:44.1239790Z deleted: sha256:84079bb1b4ee1d0119ae3d3e701de460a1324912297c675425c9a36d30fcdb0f 2024-06-26T07:37:44.1240599Z deleted: sha256:d86ac3ca148d4758cb03b493fdd49109d8f1a2c7c85c4b755493c3c4529553e5 2024-06-26T07:37:44.1241427Z deleted: sha256:532b3ab97c44a673a0d5f62ee46fa6a83248a84de8877754a612355be56ed0a3 2024-06-26T07:37:44.1242605Z deleted: sha256:617e2a35f25ae24500c88c0c9b8475f0687fbcbed3d200013b4c99c6d8941aff 2024-06-26T07:37:44.1243436Z deleted: sha256:b88120bcd5a26a660c09c5df13176e0ef098fb2c8cdb57d74ab0796a3d1194d6 2024-06-26T07:37:44.1244358Z deleted: sha256:c016b2e7764735243205fb76fa9a273bb3dd4bf05532288e3a613ce022131ac3 2024-06-26T07:37:44.1245492Z deleted: sha256:bcaca462c390deddd2995de2d245a7495e7d2e1e94f0edad30a573c83d0588fd 2024-06-26T07:37:44.1246522Z deleted: sha256:9e260eba003d5c25d7b410e5c4c3093845d9e46611896840901c689d7fbf8ee6 2024-06-26T07:37:44.1247328Z deleted: sha256:3e62b86f8b911a9b59c62d55f063e71683ae6a01c22418db5d27930712adaad6 2024-06-26T07:37:44.1248126Z deleted: sha256:905920ccb1d88449b8dde686c34a4b2a8b6f59e78ac44916b189ea3593b9fbbb 2024-06-26T07:37:44.1248933Z deleted: sha256:961201237e03acd5061406238dd7a3c1343ac888d7583212cd0ab5ae1b4f0540 2024-06-26T07:37:44.1249743Z deleted: sha256:70047a2aac62159fa28fb4dbd4567ec2ffbab68fe046967e09acb0eb3e5cb9cb 2024-06-26T07:37:44.1250570Z deleted: sha256:1ec680ad781436ae65fa987f5fa1ed032ecd5c82a6e35eea6707de501f62c0f4 2024-06-26T07:37:44.1251428Z deleted: sha256:ef859623a50e85abf6e6c1dbd6188aa765a66cc7b2bd618a1cbe4825b7b2bd91 2024-06-26T07:37:44.1252276Z deleted: sha256:404ade5cacff59773b93ab438bfa64cd3a171d07f6f168bdc4a5d4bfa0b43526 2024-06-26T07:37:44.1253111Z deleted: sha256:5b27de6ee3d9f2f18217bf18dec725f7b4954c0e48c85d35752634c22d62209b 2024-06-26T07:37:44.1253933Z deleted: sha256:498e39c575b3fa069acaac7e46b91b4e54ae3a05f2ada5c4347091f346f25d4c 2024-06-26T07:37:44.1254727Z deleted: sha256:1405919d2694d65669b1f4c842282021afe89208ce86570e96464903733da2d0 2024-06-26T07:37:44.1255524Z deleted: sha256:d5b41e5d70896c0472fc3e6da0516a17d89cd6635dc75010ecddc31ce9690fad 2024-06-26T07:37:44.1256343Z deleted: sha256:18e9379b62b9d0bb40293d28a652eafaee66a499f66f0776bf14e37a513101b7 2024-06-26T07:37:44.1257431Z deleted: sha256:a781f80ed00ff2926a699fd01a309bb0b4fb7d5655eb652c205bb9c7f811035f 2024-06-26T07:37:44.1258249Z deleted: sha256:c0430973b7cf759075330ee29c79bfd1312c91e6033d9dd883b4b1c29743a171 2024-06-26T07:37:44.1259065Z deleted: sha256:e733b3d66efb9342e5a283376c0e1ddffa47a42fecc26e61ecaf0a3472885d97 2024-06-26T07:37:44.1259889Z deleted: sha256:8929c0b2047f74b84c82c39eff9077a92ae1a828cde00c9ab78fb3e0ed5e0283 2024-06-26T07:37:44.1260693Z deleted: sha256:f5b56155fb58a0a60fe6391d3581cf91610ed69d4f0a321257c19944da296a52 2024-06-26T07:37:44.1261502Z deleted: sha256:b54d2cb355238d1aa91e58df93064447b80f53706d1ca62823acc834f5ac54fe 2024-06-26T07:37:44.1262299Z deleted: sha256:e7311285af9eeddb51aa5629aa98386e89f1456ff32843b6aee6443247edf66c 2024-06-26T07:37:44.1263115Z deleted: sha256:67657ceec347da8dc8085d5731633efeac115ba677fa4b7c5966c4260f7abab0 2024-06-26T07:37:44.1264013Z deleted: sha256:b496e50409dbf414930fc9b4437aa0aa4ce27bc7840e050ae459aaffcf9aca86 2024-06-26T07:37:44.1264824Z deleted: sha256:8f5276170cc4a60d36ad7b964a502306b29546254cedcbcb101ea9123497d2c0 2024-06-26T07:37:44.1265622Z deleted: sha256:53d524d1564745e96a7e515462171b73184d9f1ff5454b32fcfbbca39d361467 2024-06-26T07:37:44.1266443Z deleted: sha256:2acbadb00d69b6aca575f04baba4ea26ae2b5c2d85f09945004ad93a78a8eb75 2024-06-26T07:37:44.1267414Z deleted: sha256:02665efeff878aeb603c1edb2e21cb700c171c025d64f5aae1b2b20b12f3964a 2024-06-26T07:37:44.1268251Z deleted: sha256:19bd23e9a51dfe883a8b8d2ff6d66ff1b3476b8912856406bb79b81762aa6d4b 2024-06-26T07:37:44.1269058Z deleted: sha256:73aa8637bf9299d6298dabd858480c274a62117388f2ca1d5e9eb15c6d1e4155 2024-06-26T07:37:44.1269867Z deleted: sha256:5f8968406305f348b1caebc0e035187d6e9965ace7b9d5f2f21114d37c9fe2e1 2024-06-26T07:37:44.1270678Z deleted: sha256:d827104914db0c503cf5c81f13e2456bf4efce42d4cf1f8d5d70838207278bee 2024-06-26T07:37:44.1271519Z deleted: sha256:867b219724f2f1aaf56f42e3e9f03a5d64bdca8a6a72407efd10328aeea5d510 2024-06-26T07:37:44.1272449Z deleted: sha256:b9ff2a7cceb9237b26bcb17728edf8a558eec5a76f63626e84f80541dfc0f1d4 2024-06-26T07:37:44.1273270Z deleted: sha256:be73f509106819007ce2b0f5a1301ac56ae583cb6c1fb69bee93efa0840368f1 2024-06-26T07:37:44.1274084Z deleted: sha256:2a11c07c915c3fdbb4e73096dea73ea950c970d50fe1a5764060bd62b7b7cf2c 2024-06-26T07:37:44.1274902Z deleted: sha256:5e9989622e2abc1c72f8184343507b45d75e4ff971edb68b96cbcd75ae484ff5 2024-06-26T07:37:44.1275725Z deleted: sha256:05ce642bd9cf5a9205bff107df28b1d6cef7d5f6ab3ea944d28c032ec61270df 2024-06-26T07:37:44.1276530Z deleted: sha256:017702617f6c07b73f2088678f4710b794cfe897a213a5cf122a0e2519eae0ac 2024-06-26T07:37:44.1277333Z deleted: sha256:44b3b37bb1ca39dc5cf356de41a23a5c12f605a1992813e408087fb2b0ad6d1c 2024-06-26T07:37:44.1278144Z deleted: sha256:8c387982e9f27db357a1d149dc149cabc0bd04668ad6956387a5072682dada67 2024-06-26T07:37:44.1278958Z deleted: sha256:10d415dca4e3fbb88a747b40317f3c6333c041ba8f73855f67fb9fe981fae5ee 2024-06-26T07:37:44.1279772Z deleted: sha256:6c65469bf0f53dcd625faefe363b8f3136c08388a6cf64d9158e8576e4ee89b1 2024-06-26T07:37:44.1280594Z deleted: sha256:027db5d8b76deeefbd8c34b485ab7ff14ed2a3e6800425c1ce0d77c2e9fdd4ab 2024-06-26T07:37:44.1281415Z deleted: sha256:24941234ad28cf5c8174d7cf18326c10b4e17f2c2137672ea0322df461597dd7 2024-06-26T07:37:44.1282407Z deleted: sha256:240ea2eebfa8519d6c2d88f4c99e66ce4e038d000f93ce8d03726c01d81180ef 2024-06-26T07:37:44.1283306Z deleted: sha256:09e823cb391a69f6a195e3ca8a41624b5f8b34eee14c6eb36277de814a91966e 2024-06-26T07:37:44.1284101Z deleted: sha256:730479d695169a6593bf0ac16b377491dce65175a6d158e012422367d04577fd 2024-06-26T07:37:44.1284901Z deleted: sha256:b0a30f9f7bd5459ff662453b9346a7ba8ea8db1191528345c1953eab4aa2f54e 2024-06-26T07:37:44.1285725Z deleted: sha256:75cefa87effaa3ad361c74642839923101f0f9982f814f3c9e2ed4ad52f2a008 2024-06-26T07:37:44.1286527Z deleted: sha256:6f97db87ba9512e615e871c6982000602982e1e5a5be334a56e7292251e84e00 2024-06-26T07:37:44.1287335Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-06-26T07:37:44.1288145Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-06-26T07:37:44.1288943Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-06-26T07:37:44.1289749Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-06-26T07:37:44.1290560Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-06-26T07:37:44.1291370Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-06-26T07:37:44.1292247Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-06-26T07:37:44.1293074Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-06-26T07:37:44.1293897Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-06-26T07:37:44.1294724Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-06-26T07:37:44.1295531Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-06-26T07:37:44.1296011Z 2024-06-26T07:37:44.1296167Z Total reclaimed space: 36.35GB 2024-06-26T07:37:44.1371543Z Post job cleanup. 2024-06-26T07:37:44.1418613Z Post job cleanup. 2024-06-26T07:37:44.2228047Z [command]/usr/bin/git version 2024-06-26T07:37:44.2270106Z git version 2.40.1 2024-06-26T07:37:44.2309245Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/34e593df-3c07-426a-bd7e-b465fba3b78a' before making global git config changes 2024-06-26T07:37:44.2310578Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T07:37:44.2314891Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:37:44.2348206Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T07:37:44.2377175Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T07:37:44.2600599Z Entering 'android/libs/fbjni' 2024-06-26T07:37:44.2638266Z Entering 'third_party/FP16' 2024-06-26T07:37:44.2675025Z Entering 'third_party/FXdiv' 2024-06-26T07:37:44.2712193Z Entering 'third_party/NNPACK' 2024-06-26T07:37:44.2747698Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:37:44.2784906Z Entering 'third_party/XNNPACK' 2024-06-26T07:37:44.2832201Z Entering 'third_party/benchmark' 2024-06-26T07:37:44.2868897Z Entering 'third_party/cpp-httplib' 2024-06-26T07:37:44.2901348Z Entering 'third_party/cpuinfo' 2024-06-26T07:37:44.2936891Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:37:44.2971511Z Entering 'third_party/cutlass' 2024-06-26T07:37:44.3013711Z Entering 'third_party/eigen' 2024-06-26T07:37:44.3050838Z Entering 'third_party/fbgemm' 2024-06-26T07:37:44.3089209Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:37:44.3124410Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:37:44.3159287Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:37:44.3198180Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:37:44.3233346Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:37:44.3270271Z Entering 'third_party/flatbuffers' 2024-06-26T07:37:44.3307247Z Entering 'third_party/fmt' 2024-06-26T07:37:44.3342888Z Entering 'third_party/foxi' 2024-06-26T07:37:44.3380194Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:37:44.3418183Z Entering 'third_party/gloo' 2024-06-26T07:37:44.3452993Z Entering 'third_party/googletest' 2024-06-26T07:37:44.3486265Z Entering 'third_party/ideep' 2024-06-26T07:37:44.3521817Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:37:44.3564368Z Entering 'third_party/ittapi' 2024-06-26T07:37:44.3600691Z Entering 'third_party/kineto' 2024-06-26T07:37:44.3636947Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:37:44.3673438Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:37:44.3710404Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:37:44.3744503Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:37:44.3777495Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:37:44.3811798Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:37:44.3849455Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:37:44.3886219Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:37:44.3921762Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:37:44.3957299Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:37:44.3993880Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:37:44.4028933Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:37:44.4064805Z Entering 'third_party/mimalloc' 2024-06-26T07:37:44.4100294Z Entering 'third_party/nccl/nccl' 2024-06-26T07:37:44.4137147Z Entering 'third_party/nlohmann' 2024-06-26T07:37:44.4170500Z Entering 'third_party/onnx' 2024-06-26T07:37:44.4217669Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:37:44.4254805Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:37:44.4292822Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:37:44.4327136Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:37:44.4361220Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:37:44.4396378Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:37:44.4431038Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:37:44.4467060Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:37:44.4498938Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:37:44.4529899Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:37:44.4563314Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:37:44.4595004Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:37:44.4631893Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:37:44.4683288Z Entering 'third_party/pocketfft' 2024-06-26T07:37:44.4720198Z Entering 'third_party/protobuf' 2024-06-26T07:37:44.4757544Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:37:44.4791437Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:37:44.4829584Z Entering 'third_party/psimd' 2024-06-26T07:37:44.4863513Z Entering 'third_party/pthreadpool' 2024-06-26T07:37:44.4899065Z Entering 'third_party/pybind11' 2024-06-26T07:37:44.4933529Z Entering 'third_party/python-peachpy' 2024-06-26T07:37:44.4967946Z Entering 'third_party/sleef' 2024-06-26T07:37:44.5000736Z Entering 'third_party/tensorpipe' 2024-06-26T07:37:44.5035464Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:37:44.5071036Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:37:44.5105890Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:37:44.5138538Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:37:44.5172633Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:37:44.5221439Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T07:37:44.5239595Z http.https://github.com/.extraheader 2024-06-26T07:37:44.5248053Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-06-26T07:37:44.5276815Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T07:37:44.5485747Z Entering 'android/libs/fbjni' 2024-06-26T07:37:44.5507237Z http.https://github.com/.extraheader 2024-06-26T07:37:44.5531545Z Entering 'third_party/FP16' 2024-06-26T07:37:44.5553444Z http.https://github.com/.extraheader 2024-06-26T07:37:44.5578775Z Entering 'third_party/FXdiv' 2024-06-26T07:37:44.5599966Z http.https://github.com/.extraheader 2024-06-26T07:37:44.5622346Z Entering 'third_party/NNPACK' 2024-06-26T07:37:44.5643893Z http.https://github.com/.extraheader 2024-06-26T07:37:44.5667946Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:37:44.5690357Z http.https://github.com/.extraheader 2024-06-26T07:37:44.5714672Z Entering 'third_party/XNNPACK' 2024-06-26T07:37:44.5736805Z http.https://github.com/.extraheader 2024-06-26T07:37:44.5771504Z Entering 'third_party/benchmark' 2024-06-26T07:37:44.5793259Z http.https://github.com/.extraheader 2024-06-26T07:37:44.5817199Z Entering 'third_party/cpp-httplib' 2024-06-26T07:37:44.5837091Z http.https://github.com/.extraheader 2024-06-26T07:37:44.5862337Z Entering 'third_party/cpuinfo' 2024-06-26T07:37:44.5884772Z http.https://github.com/.extraheader 2024-06-26T07:37:44.5909042Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:37:44.5931933Z http.https://github.com/.extraheader 2024-06-26T07:37:44.5955496Z Entering 'third_party/cutlass' 2024-06-26T07:37:44.5978640Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6008263Z Entering 'third_party/eigen' 2024-06-26T07:37:44.6029882Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6055353Z Entering 'third_party/fbgemm' 2024-06-26T07:37:44.6076844Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6102963Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:37:44.6125514Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6148349Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:37:44.6171193Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6193241Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:37:44.6214654Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6243060Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:37:44.6264532Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6286222Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:37:44.6307482Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6331591Z Entering 'third_party/flatbuffers' 2024-06-26T07:37:44.6353919Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6380332Z Entering 'third_party/fmt' 2024-06-26T07:37:44.6402848Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6427475Z Entering 'third_party/foxi' 2024-06-26T07:37:44.6449188Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6473868Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:37:44.6495221Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6517334Z Entering 'third_party/gloo' 2024-06-26T07:37:44.6540069Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6564958Z Entering 'third_party/googletest' 2024-06-26T07:37:44.6586012Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6608487Z Entering 'third_party/ideep' 2024-06-26T07:37:44.6631752Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6654311Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:37:44.6675681Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6704803Z Entering 'third_party/ittapi' 2024-06-26T07:37:44.6727182Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6751166Z Entering 'third_party/kineto' 2024-06-26T07:37:44.6773491Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6796985Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:37:44.6819460Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6844059Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:37:44.6865476Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6890766Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:37:44.6911944Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6936300Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:37:44.6958058Z http.https://github.com/.extraheader 2024-06-26T07:37:44.6982154Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:37:44.7002952Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7026401Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:37:44.7047223Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7069948Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:37:44.7090926Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7115239Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:37:44.7136716Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7160760Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:37:44.7182240Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7207984Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:37:44.7229296Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7253996Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:37:44.7275004Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7297968Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:37:44.7317755Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7341633Z Entering 'third_party/mimalloc' 2024-06-26T07:37:44.7363556Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7387978Z Entering 'third_party/nccl/nccl' 2024-06-26T07:37:44.7408818Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7433220Z Entering 'third_party/nlohmann' 2024-06-26T07:37:44.7454135Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7479591Z Entering 'third_party/onnx' 2024-06-26T07:37:44.7502190Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7537236Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:37:44.7559344Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7584078Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:37:44.7605108Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7630217Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:37:44.7653127Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7678158Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:37:44.7700659Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7723489Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:37:44.7744126Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7767062Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:37:44.7788795Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7811244Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:37:44.7832890Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7856689Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:37:44.7878338Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7908240Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:37:44.7928338Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7951488Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:37:44.7974021Z http.https://github.com/.extraheader 2024-06-26T07:37:44.7997321Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:37:44.8019179Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8044377Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:37:44.8064175Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8088680Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:37:44.8110207Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8148504Z Entering 'third_party/pocketfft' 2024-06-26T07:37:44.8169913Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8193192Z Entering 'third_party/protobuf' 2024-06-26T07:37:44.8215400Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8242112Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:37:44.8264324Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8288273Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:37:44.8310522Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8336084Z Entering 'third_party/psimd' 2024-06-26T07:37:44.8356414Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8379218Z Entering 'third_party/pthreadpool' 2024-06-26T07:37:44.8400957Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8424762Z Entering 'third_party/pybind11' 2024-06-26T07:37:44.8446811Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8470475Z Entering 'third_party/python-peachpy' 2024-06-26T07:37:44.8493375Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8516171Z Entering 'third_party/sleef' 2024-06-26T07:37:44.8537801Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8561843Z Entering 'third_party/tensorpipe' 2024-06-26T07:37:44.8585392Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8608561Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:37:44.8630119Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8653636Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:37:44.8675045Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8698092Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:37:44.8719200Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8743133Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:37:44.8764482Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8786350Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:37:44.8807719Z http.https://github.com/.extraheader 2024-06-26T07:37:44.8986171Z A job completed hook has been configured by the self-hosted runner administrator 2024-06-26T07:37:44.9005277Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T07:37:44.9011963Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:37:44.9012475Z ##[endgroup] 2024-06-26T07:37:45.5905993Z Cleaning up orphan processes