2024-06-26T07:04:17.8637529Z Current runner version: '2.317.0' 2024-06-26T07:04:17.8643655Z Runner name: 'i-036744df35e90073e' 2024-06-26T07:04:17.8644511Z Runner group name: 'Default' 2024-06-26T07:04:17.8645404Z Machine name: 'ip-10-0-27-148' 2024-06-26T07:04:17.8650073Z ##[group]GITHUB_TOKEN Permissions 2024-06-26T07:04:17.8652009Z Actions: read 2024-06-26T07:04:17.8652506Z Attestations: read 2024-06-26T07:04:17.8653047Z Checks: read 2024-06-26T07:04:17.8653532Z Contents: read 2024-06-26T07:04:17.8654008Z Deployments: read 2024-06-26T07:04:17.8654520Z Discussions: read 2024-06-26T07:04:17.8655044Z Issues: read 2024-06-26T07:04:17.8655509Z Metadata: read 2024-06-26T07:04:17.8656010Z Packages: read 2024-06-26T07:04:17.8656507Z Pages: read 2024-06-26T07:04:17.8656981Z PullRequests: read 2024-06-26T07:04:17.8657596Z RepositoryProjects: read 2024-06-26T07:04:17.8658164Z SecurityEvents: read 2024-06-26T07:04:17.8658680Z Statuses: read 2024-06-26T07:04:17.8659189Z ##[endgroup] 2024-06-26T07:04:17.8662368Z Secret source: Actions 2024-06-26T07:04:17.8663077Z Prepare workflow directory 2024-06-26T07:04:18.2095873Z Prepare all required actions 2024-06-26T07:04:18.2284240Z Getting action download info 2024-06-26T07:04:18.3764129Z Download action repository 'pytorch/test-infra@main' (SHA:43a2ce341cc31288e9a38b65ce600a7f43021bd5) 2024-06-26T07:04:18.6701432Z Download action repository 'pytorch/pytorch@main' (SHA:c422a9549d2b320b47a9391f6cbcf592b2876c88) 2024-06-26T07:04:21.4019921Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-06-26T07:04:21.5556685Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-06-26T07:04:21.8159184Z Getting action download info 2024-06-26T07:04:21.9193815Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-06-26T07:04:22.1090016Z Getting action download info 2024-06-26T07:04:22.2089791Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-06-26T07:04:22.3618498Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/129470 (b8c4c54d347aa776934c60784e35936878ef18dc) 2024-06-26T07:04:22.3620680Z ##[group] Inputs 2024-06-26T07:04:22.3621170Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:04:22.3628418Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-06-26T07:04:22.3636298Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:22.3637477Z sync-tag: 2024-06-26T07:04:22.3638571Z timeout-minutes: 240 2024-06-26T07:04:22.3638919Z use-gha: 2024-06-26T07:04:22.3639222Z dashboard-tag: 2024-06-26T07:04:22.3639554Z s3-bucket: gha-artifacts 2024-06-26T07:04:22.3639912Z aws-role-to-assume: 2024-06-26T07:04:22.3640269Z ##[endgroup] 2024-06-26T07:04:22.3641503Z Complete job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:04:22.4205824Z A job started hook has been configured by the self-hosted runner administrator 2024-06-26T07:04:22.4344517Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T07:04:22.4354999Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:22.4355593Z ##[endgroup] 2024-06-26T07:04:23.1130459Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-06-26T07:04:23.1131029Z with: 2024-06-26T07:04:23.1131632Z github-secret: *** 2024-06-26T07:04:23.1132575Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-06-26T07:04:23.1133564Z activate-with-label: false 2024-06-26T07:04:23.1133940Z label: with-ssh 2024-06-26T07:04:23.1134282Z remove-existing-keys: true 2024-06-26T07:04:23.1134695Z fail-silently: true 2024-06-26T07:04:23.1135001Z env: 2024-06-26T07:04:23.1135282Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:23.1135632Z ##[endgroup] 2024-06-26T07:04:23.2004942Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-06-26T07:04:23.2006040Z ciflow reference detected, attempting to extract PR number 2024-06-26T07:04:23.4779918Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-06-26T07:04:23.5497853Z No SSH keys found for user pytorch-bot[bot] 2024-06-26T07:04:23.5498616Z Grabbing public ssh keys from https://github.com/leslie-fang-intel.keys 2024-06-26T07:04:23.6127762Z No SSH keys found for user leslie-fang-intel 2024-06-26T07:04:23.6234542Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-06-26T07:04:23.6235229Z with: 2024-06-26T07:04:23.6235579Z submodules: recursive 2024-06-26T07:04:23.6235987Z fetch-depth: 0 2024-06-26T07:04:23.6236328Z env: 2024-06-26T07:04:23.6236655Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:23.6237064Z ##[endgroup] 2024-06-26T07:04:23.6429247Z ##[group]Run retry () { 2024-06-26T07:04:23.6429699Z retry () { 2024-06-26T07:04:23.6458817Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-06-26T07:04:23.6459420Z } 2024-06-26T07:04:23.6459745Z echo "${GITHUB_WORKSPACE}" 2024-06-26T07:04:23.6460193Z if [ -z "${NO_SUDO}" ]; then 2024-06-26T07:04:23.6460718Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T07:04:23.6461195Z else 2024-06-26T07:04:23.6461553Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T07:04:23.6462003Z fi 2024-06-26T07:04:23.6462365Z mkdir "${GITHUB_WORKSPACE}" 2024-06-26T07:04:23.6470474Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:23.6471003Z env: 2024-06-26T07:04:23.6471302Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:23.6471664Z NO_SUDO: 2024-06-26T07:04:23.6471946Z ##[endgroup] 2024-06-26T07:04:23.6494107Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:04:25.7600808Z ##[group]Run malfet/checkout@silent-checkout 2024-06-26T07:04:25.7601273Z with: 2024-06-26T07:04:25.7601603Z ref: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:04:25.7602066Z fetch-depth: 0 2024-06-26T07:04:25.7602392Z submodules: recursive 2024-06-26T07:04:25.7602741Z quiet-checkout: true 2024-06-26T07:04:25.7603218Z repository: pytorch/pytorch 2024-06-26T07:04:25.7603725Z token: *** 2024-06-26T07:04:25.7604026Z ssh-strict: true 2024-06-26T07:04:25.7604355Z persist-credentials: true 2024-06-26T07:04:25.7604711Z clean: true 2024-06-26T07:04:25.7605071Z sparse-checkout-cone-mode: true 2024-06-26T07:04:25.7605470Z lfs: false 2024-06-26T07:04:25.7605779Z set-safe-directory: true 2024-06-26T07:04:25.7606132Z env: 2024-06-26T07:04:25.7606410Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:25.7606755Z ##[endgroup] 2024-06-26T07:04:25.8521517Z Syncing repository: pytorch/pytorch 2024-06-26T07:04:25.8523358Z ##[group]Getting Git version info 2024-06-26T07:04:25.8524148Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T07:04:25.8525010Z [command]/usr/bin/git version 2024-06-26T07:04:25.8525382Z git version 2.40.1 2024-06-26T07:04:25.8526987Z ##[endgroup] 2024-06-26T07:04:25.8538790Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/f4e9628a-b7a1-4312-9d47-1d3f8c72c4a7' before making global git config changes 2024-06-26T07:04:25.8540309Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T07:04:25.8541453Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:04:25.8571067Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T07:04:25.8575092Z ##[group]Initializing the repository 2024-06-26T07:04:25.8578237Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:04:25.8597349Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-06-26T07:04:25.8598822Z hint: is subject to change. To configure the initial branch name to use in all 2024-06-26T07:04:25.8599943Z hint: of your new repositories, which will suppress this warning, call: 2024-06-26T07:04:25.8600514Z hint: 2024-06-26T07:04:25.8601191Z hint: git config --global init.defaultBranch 2024-06-26T07:04:25.8601715Z hint: 2024-06-26T07:04:25.8602269Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-06-26T07:04:25.8603126Z hint: 'development'. The just-created branch can be renamed via this command: 2024-06-26T07:04:25.8603923Z hint: 2024-06-26T07:04:25.8604247Z hint: git branch -m 2024-06-26T07:04:25.8605003Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-06-26T07:04:25.8606860Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-06-26T07:04:25.8632538Z ##[endgroup] 2024-06-26T07:04:25.8633146Z ##[group]Disabling automatic garbage collection 2024-06-26T07:04:25.8636108Z [command]/usr/bin/git config --local gc.auto 0 2024-06-26T07:04:25.8660112Z ##[endgroup] 2024-06-26T07:04:25.8660634Z ##[group]Setting up auth 2024-06-26T07:04:25.8666247Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T07:04:25.8686675Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T07:04:25.8894410Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T07:04:25.8919581Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T07:04:25.9117736Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:04:25.9154228Z ##[endgroup] 2024-06-26T07:04:25.9154849Z ##[group]Fetching the repository 2024-06-26T07:04:25.9161369Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-06-26T07:04:28.2106935Z remote: Enumerating objects: 987619 2024-06-26T07:04:28.2107949Z remote: Enumerating objects: 990209, done. 2024-06-26T07:04:28.2110269Z remote: Counting objects: 0% (1/2590) 2024-06-26T07:04:28.2110997Z remote: Counting objects: 1% (26/2590) 2024-06-26T07:04:28.2113170Z remote: Counting objects: 2% (52/2590) 2024-06-26T07:04:28.2113924Z remote: Counting objects: 3% (78/2590) 2024-06-26T07:04:28.2115720Z remote: Counting objects: 4% (104/2590) 2024-06-26T07:04:28.2116473Z remote: Counting objects: 5% (130/2590) 2024-06-26T07:04:28.2118355Z remote: Counting objects: 6% (156/2590) 2024-06-26T07:04:28.2119153Z remote: Counting objects: 7% (182/2590) 2024-06-26T07:04:28.2120396Z remote: Counting objects: 8% (208/2590) 2024-06-26T07:04:28.2121136Z remote: Counting objects: 9% (234/2590) 2024-06-26T07:04:28.2123645Z remote: Counting objects: 10% (259/2590) 2024-06-26T07:04:28.2124472Z remote: Counting objects: 11% (285/2590) 2024-06-26T07:04:28.2125134Z remote: Counting objects: 12% (311/2590) 2024-06-26T07:04:28.2125717Z remote: Counting objects: 13% (337/2590) 2024-06-26T07:04:28.2126311Z remote: Counting objects: 14% (363/2590) 2024-06-26T07:04:28.2126898Z remote: Counting objects: 15% (389/2590) 2024-06-26T07:04:28.2127487Z remote: Counting objects: 16% (415/2590) 2024-06-26T07:04:28.2128071Z remote: Counting objects: 17% (441/2590) 2024-06-26T07:04:28.2128887Z remote: Counting objects: 18% (467/2590) 2024-06-26T07:04:28.2129535Z remote: Counting objects: 19% (493/2590) 2024-06-26T07:04:28.2132867Z remote: Counting objects: 20% (518/2590) 2024-06-26T07:04:28.2133681Z remote: Counting objects: 21% (544/2590) 2024-06-26T07:04:28.2134274Z remote: Counting objects: 22% (570/2590) 2024-06-26T07:04:28.2134863Z remote: Counting objects: 23% (596/2590) 2024-06-26T07:04:28.2135505Z remote: Counting objects: 24% (622/2590) 2024-06-26T07:04:28.2136103Z remote: Counting objects: 25% (648/2590) 2024-06-26T07:04:28.2136698Z remote: Counting objects: 26% (674/2590) 2024-06-26T07:04:28.2137358Z remote: Counting objects: 27% (700/2590) 2024-06-26T07:04:28.2138098Z remote: Counting objects: 28% (726/2590) 2024-06-26T07:04:28.2138699Z remote: Counting objects: 29% (752/2590) 2024-06-26T07:04:28.2141978Z remote: Counting objects: 30% (777/2590) 2024-06-26T07:04:28.2142579Z remote: Counting objects: 31% (803/2590) 2024-06-26T07:04:28.2143156Z remote: Counting objects: 32% (829/2590) 2024-06-26T07:04:28.2150492Z remote: Counting objects: 33% (855/2590) 2024-06-26T07:04:28.2151305Z remote: Counting objects: 34% (881/2590) 2024-06-26T07:04:28.2151956Z remote: Counting objects: 35% (907/2590) 2024-06-26T07:04:28.2152544Z remote: Counting objects: 36% (933/2590) 2024-06-26T07:04:28.2153144Z remote: Counting objects: 37% (959/2590) 2024-06-26T07:04:28.2153760Z remote: Counting objects: 38% (985/2590) 2024-06-26T07:04:28.2154363Z remote: Counting objects: 39% (1011/2590) 2024-06-26T07:04:28.2155030Z remote: Counting objects: 40% (1036/2590) 2024-06-26T07:04:28.2155647Z remote: Counting objects: 41% (1062/2590) 2024-06-26T07:04:28.2156265Z remote: Counting objects: 42% (1088/2590) 2024-06-26T07:04:28.2156929Z remote: Counting objects: 43% (1114/2590) 2024-06-26T07:04:28.2157737Z remote: Counting objects: 44% (1140/2590) 2024-06-26T07:04:28.2158615Z remote: Counting objects: 45% (1166/2590) 2024-06-26T07:04:28.2159303Z remote: Counting objects: 46% (1192/2590) 2024-06-26T07:04:28.2160121Z remote: Counting objects: 47% (1218/2590) 2024-06-26T07:04:28.2160779Z remote: Counting objects: 48% (1244/2590) 2024-06-26T07:04:28.2161405Z remote: Counting objects: 49% (1270/2590) 2024-06-26T07:04:28.2162240Z remote: Counting objects: 50% (1295/2590) 2024-06-26T07:04:28.2163000Z remote: Counting objects: 51% (1321/2590) 2024-06-26T07:04:28.2163609Z remote: Counting objects: 52% (1347/2590) 2024-06-26T07:04:28.2164314Z remote: Counting objects: 53% (1373/2590) 2024-06-26T07:04:28.2165185Z remote: Counting objects: 54% (1399/2590) 2024-06-26T07:04:28.2165796Z remote: Counting objects: 55% (1425/2590) 2024-06-26T07:04:28.2166506Z remote: Counting objects: 56% (1451/2590) 2024-06-26T07:04:28.2167403Z remote: Counting objects: 57% (1477/2590) 2024-06-26T07:04:28.2168288Z remote: Counting objects: 58% (1503/2590) 2024-06-26T07:04:28.2169205Z remote: Counting objects: 59% (1529/2590) 2024-06-26T07:04:28.2170104Z remote: Counting objects: 60% (1554/2590) 2024-06-26T07:04:28.2170980Z remote: Counting objects: 61% (1580/2590) 2024-06-26T07:04:28.2171860Z remote: Counting objects: 62% (1606/2590) 2024-06-26T07:04:28.2172785Z remote: Counting objects: 63% (1632/2590) 2024-06-26T07:04:28.2173678Z remote: Counting objects: 64% (1658/2590) 2024-06-26T07:04:28.2174571Z remote: Counting objects: 65% (1684/2590) 2024-06-26T07:04:28.2175455Z remote: Counting objects: 66% (1710/2590) 2024-06-26T07:04:28.2176352Z remote: Counting objects: 67% (1736/2590) 2024-06-26T07:04:28.2177388Z remote: Counting objects: 68% (1762/2590) 2024-06-26T07:04:28.2178293Z remote: Counting objects: 69% (1788/2590) 2024-06-26T07:04:28.2179173Z remote: Counting objects: 70% (1813/2590) 2024-06-26T07:04:28.2180061Z remote: Counting objects: 71% (1839/2590) 2024-06-26T07:04:28.2180950Z remote: Counting objects: 72% (1865/2590) 2024-06-26T07:04:28.2181844Z remote: Counting objects: 73% (1891/2590) 2024-06-26T07:04:28.2182728Z remote: Counting objects: 74% (1917/2590) 2024-06-26T07:04:28.2183629Z remote: Counting objects: 75% (1943/2590) 2024-06-26T07:04:28.2184531Z remote: Counting objects: 76% (1969/2590) 2024-06-26T07:04:28.2185428Z remote: Counting objects: 77% (1995/2590) 2024-06-26T07:04:28.2186317Z remote: Counting objects: 78% (2021/2590) 2024-06-26T07:04:28.2187364Z remote: Counting objects: 79% (2047/2590) 2024-06-26T07:04:28.2188261Z remote: Counting objects: 80% (2072/2590) 2024-06-26T07:04:28.2189165Z remote: Counting objects: 81% (2098/2590) 2024-06-26T07:04:28.2190046Z remote: Counting objects: 82% (2124/2590) 2024-06-26T07:04:28.2190943Z remote: Counting objects: 83% (2150/2590) 2024-06-26T07:04:28.2191838Z remote: Counting objects: 84% (2176/2590) 2024-06-26T07:04:28.2192730Z remote: Counting objects: 85% (2202/2590) 2024-06-26T07:04:28.2193619Z remote: Counting objects: 86% (2228/2590) 2024-06-26T07:04:28.2194510Z remote: Counting objects: 87% (2254/2590) 2024-06-26T07:04:28.2195413Z remote: Counting objects: 88% (2280/2590) 2024-06-26T07:04:28.2196315Z remote: Counting objects: 89% (2306/2590) 2024-06-26T07:04:28.2197198Z remote: Counting objects: 90% (2331/2590) 2024-06-26T07:04:28.2198268Z remote: Counting objects: 91% (2357/2590) 2024-06-26T07:04:28.2199092Z remote: Counting objects: 92% (2383/2590) 2024-06-26T07:04:28.2199691Z remote: Counting objects: 93% (2409/2590) 2024-06-26T07:04:28.2200302Z remote: Counting objects: 94% (2435/2590) 2024-06-26T07:04:28.2200912Z remote: Counting objects: 95% (2461/2590) 2024-06-26T07:04:28.2201521Z remote: Counting objects: 96% (2487/2590) 2024-06-26T07:04:28.2202118Z remote: Counting objects: 97% (2513/2590) 2024-06-26T07:04:28.2202727Z remote: Counting objects: 98% (2539/2590) 2024-06-26T07:04:28.2203330Z remote: Counting objects: 99% (2565/2590) 2024-06-26T07:04:28.2203934Z remote: Counting objects: 100% (2590/2590) 2024-06-26T07:04:28.2204560Z remote: Counting objects: 100% (2590/2590), done. 2024-06-26T07:04:28.2320838Z remote: Compressing objects: 0% (1/1068) 2024-06-26T07:04:28.2746057Z remote: Compressing objects: 1% (11/1068) 2024-06-26T07:04:28.3174469Z remote: Compressing objects: 2% (22/1068) 2024-06-26T07:04:28.3559312Z remote: Compressing objects: 3% (33/1068) 2024-06-26T07:04:28.3778074Z remote: Compressing objects: 4% (43/1068) 2024-06-26T07:04:28.4058295Z remote: Compressing objects: 5% (54/1068) 2024-06-26T07:04:28.4253958Z remote: Compressing objects: 6% (65/1068) 2024-06-26T07:04:28.4652718Z remote: Compressing objects: 7% (75/1068) 2024-06-26T07:04:28.5514602Z remote: Compressing objects: 8% (86/1068) 2024-06-26T07:04:28.6316829Z remote: Compressing objects: 9% (97/1068) 2024-06-26T07:04:28.6568344Z remote: Compressing objects: 10% (107/1068) 2024-06-26T07:04:28.6755177Z remote: Compressing objects: 11% (118/1068) 2024-06-26T07:04:28.6858860Z remote: Compressing objects: 12% (129/1068) 2024-06-26T07:04:28.6859656Z remote: Compressing objects: 13% (139/1068) 2024-06-26T07:04:28.6986219Z remote: Compressing objects: 14% (150/1068) 2024-06-26T07:04:28.6987469Z remote: Compressing objects: 15% (161/1068) 2024-06-26T07:04:28.7053414Z remote: Compressing objects: 16% (171/1068) 2024-06-26T07:04:28.7054242Z remote: Compressing objects: 17% (182/1068) 2024-06-26T07:04:28.7054896Z remote: Compressing objects: 18% (193/1068) 2024-06-26T07:04:28.7070726Z remote: Compressing objects: 19% (203/1068) 2024-06-26T07:04:28.7091707Z remote: Compressing objects: 20% (214/1068) 2024-06-26T07:04:28.7107823Z remote: Compressing objects: 21% (225/1068) 2024-06-26T07:04:28.7123434Z remote: Compressing objects: 22% (235/1068) 2024-06-26T07:04:28.7149545Z remote: Compressing objects: 23% (246/1068) 2024-06-26T07:04:28.7158391Z remote: Compressing objects: 24% (257/1068) 2024-06-26T07:04:28.7167467Z remote: Compressing objects: 25% (267/1068) 2024-06-26T07:04:28.7175694Z remote: Compressing objects: 26% (278/1068) 2024-06-26T07:04:28.7180889Z remote: Compressing objects: 27% (289/1068) 2024-06-26T07:04:28.7188662Z remote: Compressing objects: 28% (300/1068) 2024-06-26T07:04:28.7202678Z remote: Compressing objects: 29% (310/1068) 2024-06-26T07:04:28.7207322Z remote: Compressing objects: 30% (321/1068) 2024-06-26T07:04:28.7209937Z remote: Compressing objects: 31% (332/1068) 2024-06-26T07:04:28.7212954Z remote: Compressing objects: 32% (342/1068) 2024-06-26T07:04:28.7225274Z remote: Compressing objects: 33% (353/1068) 2024-06-26T07:04:28.7235993Z remote: Compressing objects: 34% (364/1068) 2024-06-26T07:04:28.7251428Z remote: Compressing objects: 35% (374/1068) 2024-06-26T07:04:28.7270036Z remote: Compressing objects: 36% (385/1068) 2024-06-26T07:04:28.7285181Z remote: Compressing objects: 37% (396/1068) 2024-06-26T07:04:28.7301486Z remote: Compressing objects: 38% (406/1068) 2024-06-26T07:04:28.7317598Z remote: Compressing objects: 39% (417/1068) 2024-06-26T07:04:28.7333728Z remote: Compressing objects: 40% (428/1068) 2024-06-26T07:04:28.7352892Z remote: Compressing objects: 41% (438/1068) 2024-06-26T07:04:28.7367412Z remote: Compressing objects: 42% (449/1068) 2024-06-26T07:04:28.7380962Z remote: Compressing objects: 43% (460/1068) 2024-06-26T07:04:28.7385500Z remote: Compressing objects: 44% (470/1068) 2024-06-26T07:04:28.7391636Z remote: Compressing objects: 45% (481/1068) 2024-06-26T07:04:28.7398509Z remote: Compressing objects: 46% (492/1068) 2024-06-26T07:04:28.7415375Z remote: Compressing objects: 47% (502/1068) 2024-06-26T07:04:28.7427421Z remote: Compressing objects: 48% (513/1068) 2024-06-26T07:04:28.7431680Z remote: Compressing objects: 49% (524/1068) 2024-06-26T07:04:28.7439207Z remote: Compressing objects: 50% (534/1068) 2024-06-26T07:04:28.7448880Z remote: Compressing objects: 51% (545/1068) 2024-06-26T07:04:28.7461444Z remote: Compressing objects: 52% (556/1068) 2024-06-26T07:04:28.7478973Z remote: Compressing objects: 53% (567/1068) 2024-06-26T07:04:28.7493539Z remote: Compressing objects: 54% (577/1068) 2024-06-26T07:04:28.7503129Z remote: Compressing objects: 55% (588/1068) 2024-06-26T07:04:28.7517044Z remote: Compressing objects: 56% (599/1068) 2024-06-26T07:04:28.7530163Z remote: Compressing objects: 57% (609/1068) 2024-06-26T07:04:28.7541089Z remote: Compressing objects: 58% (620/1068) 2024-06-26T07:04:28.7549029Z remote: Compressing objects: 59% (631/1068) 2024-06-26T07:04:28.7560199Z remote: Compressing objects: 60% (641/1068) 2024-06-26T07:04:28.7579507Z remote: Compressing objects: 61% (652/1068) 2024-06-26T07:04:28.7589228Z remote: Compressing objects: 62% (663/1068) 2024-06-26T07:04:28.7591842Z remote: Compressing objects: 63% (673/1068) 2024-06-26T07:04:28.7598385Z remote: Compressing objects: 64% (684/1068) 2024-06-26T07:04:28.7606072Z remote: Compressing objects: 65% (695/1068) 2024-06-26T07:04:28.7608672Z remote: Compressing objects: 66% (705/1068) 2024-06-26T07:04:28.7619875Z remote: Compressing objects: 67% (716/1068) 2024-06-26T07:04:28.7622430Z remote: Compressing objects: 68% (727/1068) 2024-06-26T07:04:28.7625270Z remote: Compressing objects: 69% (737/1068) 2024-06-26T07:04:28.7628581Z remote: Compressing objects: 70% (748/1068) 2024-06-26T07:04:28.7631824Z remote: Compressing objects: 71% (759/1068) 2024-06-26T07:04:28.7634822Z remote: Compressing objects: 72% (769/1068) 2024-06-26T07:04:28.7637403Z remote: Compressing objects: 73% (780/1068) 2024-06-26T07:04:28.7638326Z remote: Compressing objects: 74% (791/1068) 2024-06-26T07:04:28.7638992Z remote: Compressing objects: 75% (801/1068) 2024-06-26T07:04:28.7639652Z remote: Compressing objects: 76% (812/1068) 2024-06-26T07:04:28.7640403Z remote: Compressing objects: 77% (823/1068) 2024-06-26T07:04:28.7641050Z remote: Compressing objects: 78% (834/1068) 2024-06-26T07:04:28.7643092Z remote: Compressing objects: 79% (844/1068) 2024-06-26T07:04:28.7643751Z remote: Compressing objects: 80% (855/1068) 2024-06-26T07:04:28.7645581Z remote: Compressing objects: 81% (866/1068) 2024-06-26T07:04:28.7648827Z remote: Compressing objects: 82% (876/1068) 2024-06-26T07:04:28.7651376Z remote: Compressing objects: 83% (887/1068) 2024-06-26T07:04:28.7652019Z remote: Compressing objects: 84% (898/1068) 2024-06-26T07:04:28.7653974Z remote: Compressing objects: 85% (908/1068) 2024-06-26T07:04:28.7654617Z remote: Compressing objects: 86% (919/1068) 2024-06-26T07:04:28.7656760Z remote: Compressing objects: 87% (930/1068) 2024-06-26T07:04:28.7659453Z remote: Compressing objects: 88% (940/1068) 2024-06-26T07:04:28.7662102Z remote: Compressing objects: 89% (951/1068) 2024-06-26T07:04:28.7662769Z remote: Compressing objects: 90% (962/1068) 2024-06-26T07:04:28.7663415Z remote: Compressing objects: 91% (972/1068) 2024-06-26T07:04:28.7665279Z remote: Compressing objects: 92% (983/1068) 2024-06-26T07:04:28.7667874Z remote: Compressing objects: 93% (994/1068) 2024-06-26T07:04:28.7668643Z remote: Compressing objects: 94% (1004/1068) 2024-06-26T07:04:28.7669291Z remote: Compressing objects: 95% (1015/1068) 2024-06-26T07:04:28.7669923Z remote: Compressing objects: 96% (1026/1068) 2024-06-26T07:04:28.7670562Z remote: Compressing objects: 97% (1036/1068) 2024-06-26T07:04:28.7673259Z remote: Compressing objects: 98% (1047/1068) 2024-06-26T07:04:28.7673991Z remote: Compressing objects: 99% (1058/1068) 2024-06-26T07:04:28.7674748Z remote: Compressing objects: 100% (1068/1068) 2024-06-26T07:04:28.7675417Z remote: Compressing objects: 100% (1068/1068), done. 2024-06-26T07:04:49.0422447Z remote: Total 990209 (delta 2055), reused 1861 (delta 1521), pack-reused 987619 2024-06-26T07:05:13.0586493Z [command]/usr/bin/git rev-parse --verify --quiet b8c4c54d347aa776934c60784e35936878ef18dc^{object} 2024-06-26T07:05:13.0608885Z b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:05:13.0612860Z ##[endgroup] 2024-06-26T07:05:13.0613412Z ##[group]Determining the checkout info 2024-06-26T07:05:13.0615528Z ##[endgroup] 2024-06-26T07:05:13.0616094Z ##[group]Checking out the ref 2024-06-26T07:05:13.0618050Z [command]/usr/bin/git checkout --quiet --force b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:05:14.2825656Z ##[endgroup] 2024-06-26T07:05:14.2826378Z ##[group]Setting up auth for fetching submodules 2024-06-26T07:05:14.2831133Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:05:14.2872483Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-06-26T07:05:14.2892522Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-06-26T07:05:14.2910636Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-06-26T07:05:14.2928648Z ##[endgroup] 2024-06-26T07:05:14.2929184Z ##[group]Fetching submodules 2024-06-26T07:05:14.2932286Z [command]/usr/bin/git submodule sync --recursive 2024-06-26T07:05:14.3139314Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-06-26T07:05:14.3343985Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-06-26T07:05:14.3345406Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-06-26T07:05:14.3346833Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-06-26T07:05:14.3348364Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-06-26T07:05:14.3350092Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-06-26T07:05:14.3352495Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-06-26T07:05:14.3354956Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-06-26T07:05:14.3357941Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-06-26T07:05:14.3361297Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-06-26T07:05:14.3364770Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-06-26T07:05:14.3367648Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-06-26T07:05:14.3371126Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-06-26T07:05:14.3374439Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-06-26T07:05:14.3378183Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-06-26T07:05:14.3381612Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-06-26T07:05:14.3385245Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-06-26T07:05:14.3389259Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:05:14.3392789Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-06-26T07:05:14.3396791Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-06-26T07:05:14.3401117Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-06-26T07:05:14.3405096Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-06-26T07:05:14.3409220Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-06-26T07:05:14.3413535Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-06-26T07:05:14.3417780Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-06-26T07:05:14.3422167Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-06-26T07:05:14.3426374Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-06-26T07:05:14.3431222Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-06-26T07:05:14.3435499Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-06-26T07:05:14.3440486Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-06-26T07:05:14.3445118Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-06-26T07:05:14.3450034Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-06-26T07:05:14.3454842Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-06-26T07:05:14.3459758Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-06-26T07:05:14.3464469Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-06-26T07:05:14.3469438Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-06-26T07:05:14.3490058Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-06-26T07:05:14.6305128Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-06-26T07:05:14.8358472Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-06-26T07:05:15.0110540Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-06-26T07:05:15.2469969Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-06-26T07:05:17.2576462Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-06-26T07:05:27.1948875Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-06-26T07:05:27.5447061Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-06-26T07:05:28.0485647Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-06-26T07:05:28.6625469Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-06-26T07:05:29.8827039Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-06-26T07:05:31.5811899Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-06-26T07:05:37.4891093Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-06-26T07:05:38.7693755Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-06-26T07:05:40.4012161Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-06-26T07:05:41.6223411Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-06-26T07:05:41.8088679Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-06-26T07:05:42.2245962Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-06-26T07:05:42.5165014Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-06-26T07:05:43.4834155Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-06-26T07:05:43.8405218Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-06-26T07:05:44.1062130Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-06-26T07:05:45.6442280Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-06-26T07:05:46.3886237Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-06-26T07:05:47.0399392Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-06-26T07:05:52.6180627Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-06-26T07:05:54.9884285Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-06-26T07:06:00.6458438Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-06-26T07:06:00.8518983Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-06-26T07:06:08.9904275Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-06-26T07:06:09.1414946Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-06-26T07:06:09.3259790Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-06-26T07:06:10.2851016Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-06-26T07:06:10.5597007Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-06-26T07:06:11.1793337Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-06-26T07:06:11.5803454Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-06-26T07:06:11.5889374Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-06-26T07:06:11.5953483Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-06-26T07:06:11.6135201Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-06-26T07:06:11.6422614Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-06-26T07:06:12.4112896Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-06-26T07:06:12.4291159Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-06-26T07:06:12.4649549Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-06-26T07:06:12.5488051Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-06-26T07:06:12.5753421Z Submodule path 'third_party/cudnn_frontend': checked out 'aa3abd4bc689d6412979c7f55f9cd132848c9c6a' 2024-06-26T07:06:12.9969727Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-06-26T07:06:13.2083333Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-06-26T07:06:13.2709908Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-06-26T07:06:13.2721155Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:06:13.2724812Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:06:13.2727122Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:06:13.2729446Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:06:13.2731684Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:06:13.2747024Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-06-26T07:06:14.2795649Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-06-26T07:06:14.8653479Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-06-26T07:06:16.5758832Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-06-26T07:06:17.5330362Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-06-26T07:06:17.8432720Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-06-26T07:06:17.9270808Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-06-26T07:06:18.2664303Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-06-26T07:06:18.3213852Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-06-26T07:06:18.3302348Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-06-26T07:06:18.4229285Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-06-26T07:06:18.4517181Z Submodule path 'third_party/fmt': checked out 'e69e5f977d458f2650bb346dadf2ad30c5320281' 2024-06-26T07:06:18.4579288Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-06-26T07:06:18.4880416Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-06-26T07:06:18.5072361Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-06-26T07:06:18.5423851Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:06:18.5526773Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-06-26T07:06:18.5537877Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-06-26T07:06:18.5553492Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-06-26T07:06:30.5739857Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-06-26T07:06:30.5875556Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-06-26T07:06:30.6648830Z Submodule path 'third_party/kineto': checked out '8681ff11e1fa54da39023076c5c43eddd87b7a8a' 2024-06-26T07:06:30.6660319Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:06:30.6662593Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:06:30.6664391Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:06:30.6681471Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-06-26T07:06:31.1772457Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-06-26T07:06:32.3880194Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-06-26T07:06:33.4052188Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-06-26T07:06:33.4062349Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:06:33.4064205Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:06:33.4065948Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:06:33.4067713Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:06:33.4069565Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:06:33.4071816Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:06:33.4074091Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:06:33.4076147Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:06:33.4094798Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-06-26T07:06:34.4199227Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-06-26T07:06:34.7794775Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-06-26T07:06:36.0186624Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-06-26T07:06:36.2879450Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-06-26T07:06:36.7687669Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-06-26T07:06:37.7573535Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-06-26T07:06:43.4568550Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-06-26T07:06:43.8288197Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-06-26T07:06:43.8425795Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-06-26T07:06:43.8723572Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-06-26T07:06:43.8817426Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-06-26T07:06:43.8828121Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:06:43.8845460Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-06-26T07:06:44.1845290Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-06-26T07:06:44.1980809Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-06-26T07:06:44.2319580Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-06-26T07:06:44.3129591Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-06-26T07:06:44.3250710Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-06-26T07:06:44.3545323Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out 'a33701196adfad74917046096bf5a2aa0ab0bb50' 2024-06-26T07:06:44.4040185Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-06-26T07:06:44.4343420Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-06-26T07:06:44.4534051Z Submodule path 'third_party/nccl/nccl': checked out '48bb7fec7953112ff37499a272317f6663f8f600' 2024-06-26T07:06:44.5390341Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-06-26T07:06:44.8103842Z Submodule path 'third_party/onnx': checked out '990217f043af7222348ca8f0301e17fa7b841781' 2024-06-26T07:06:44.8127032Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-06-26T07:06:44.8128850Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-06-26T07:06:44.8144538Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-06-26T07:06:45.2187945Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-06-26T07:06:46.2982828Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-06-26T07:06:46.3261218Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-06-26T07:06:46.3792660Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-06-26T07:06:46.3804698Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:06:46.3811762Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:06:46.3813548Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:06:46.3815664Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:06:46.3817717Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:06:46.3819823Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:06:46.3821790Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:06:46.3823473Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:06:46.3835486Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-06-26T07:06:46.8011763Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-06-26T07:06:47.8154457Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-06-26T07:06:48.1277806Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-06-26T07:06:54.0784970Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-06-26T07:06:54.3597870Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-06-26T07:06:54.5681647Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-06-26T07:06:54.8663382Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-06-26T07:07:02.4388210Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-06-26T07:07:02.4719600Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-06-26T07:07:02.4846076Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-06-26T07:07:02.5674865Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-06-26T07:07:02.5768130Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-06-26T07:07:02.5873194Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-06-26T07:07:02.5985386Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-06-26T07:07:02.5996612Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:02.5999137Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:02.6015662Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-06-26T07:07:04.4349368Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-06-26T07:07:05.6068406Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-06-26T07:07:05.6432113Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:07:05.9978691Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-06-26T07:07:06.0058372Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-06-26T07:07:06.2236082Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-06-26T07:07:06.2253514Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:06.2255342Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:06.2272731Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-06-26T07:07:06.6755149Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-06-26T07:07:07.6339820Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-06-26T07:07:07.6961189Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-06-26T07:07:07.7021820Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-06-26T07:07:07.7109237Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-06-26T07:07:07.7389702Z Submodule path 'third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b' 2024-06-26T07:07:07.7614251Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-06-26T07:07:07.7942203Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-06-26T07:07:07.8142478Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-06-26T07:07:07.8153740Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:07.8155724Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:07.8158463Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:07.8161362Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:07.8178687Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-06-26T07:07:08.7971882Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-06-26T07:07:08.9868751Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-06-26T07:07:10.0167910Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-06-26T07:07:11.1499614Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-06-26T07:07:11.1611124Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-06-26T07:07:11.2169596Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-06-26T07:07:11.2397366Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-06-26T07:07:11.2408197Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:11.2425558Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-06-26T07:07:11.4226641Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-06-26T07:07:11.4248546Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-06-26T07:07:11.4453137Z Entering 'android/libs/fbjni' 2024-06-26T07:07:11.4480968Z Entering 'third_party/FP16' 2024-06-26T07:07:11.4507522Z Entering 'third_party/FXdiv' 2024-06-26T07:07:11.4534451Z Entering 'third_party/NNPACK' 2024-06-26T07:07:11.4563616Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:11.4589434Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:11.4626432Z Entering 'third_party/benchmark' 2024-06-26T07:07:11.4656052Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:11.4681910Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:11.4711055Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:11.4736321Z Entering 'third_party/cutlass' 2024-06-26T07:07:11.4768629Z Entering 'third_party/eigen' 2024-06-26T07:07:11.4799620Z Entering 'third_party/fbgemm' 2024-06-26T07:07:11.4828017Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:11.4854288Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:11.4882759Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:11.4914459Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:11.4943464Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:11.4971309Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:11.5002876Z Entering 'third_party/fmt' 2024-06-26T07:07:11.5031846Z Entering 'third_party/foxi' 2024-06-26T07:07:11.5058945Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:11.5084676Z Entering 'third_party/gloo' 2024-06-26T07:07:11.5114418Z Entering 'third_party/googletest' 2024-06-26T07:07:11.5142991Z Entering 'third_party/ideep' 2024-06-26T07:07:11.5170736Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:11.5204850Z Entering 'third_party/ittapi' 2024-06-26T07:07:11.5231826Z Entering 'third_party/kineto' 2024-06-26T07:07:11.5258784Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:11.5286171Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:11.5315583Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:11.5343490Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:11.5371287Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:11.5398961Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:11.5427896Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:11.5456242Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:11.5483497Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:11.5510993Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:11.5539584Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:11.5566437Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:11.5593866Z Entering 'third_party/mimalloc' 2024-06-26T07:07:11.5623119Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:11.5648512Z Entering 'third_party/nlohmann' 2024-06-26T07:07:11.5675410Z Entering 'third_party/onnx' 2024-06-26T07:07:11.5715375Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:11.5744488Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:11.5774056Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:11.5804252Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:11.5831918Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:11.5860785Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:11.5888637Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:11.5915841Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:11.5945862Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:11.5972428Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:11.5998189Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:11.6026347Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:11.6054235Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:11.6099558Z Entering 'third_party/pocketfft' 2024-06-26T07:07:11.6126045Z Entering 'third_party/protobuf' 2024-06-26T07:07:11.6154592Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:11.6183227Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:11.6212526Z Entering 'third_party/psimd' 2024-06-26T07:07:11.6239508Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:11.6268274Z Entering 'third_party/pybind11' 2024-06-26T07:07:11.6295164Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:11.6323319Z Entering 'third_party/sleef' 2024-06-26T07:07:11.6355226Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:11.6383789Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:11.6408955Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:11.6436482Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:11.6462741Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:11.6490462Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:11.6529648Z ##[endgroup] 2024-06-26T07:07:11.6531987Z ##[group]Persisting credentials for submodules 2024-06-26T07:07:11.6534638Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-06-26T07:07:11.6737787Z Entering 'android/libs/fbjni' 2024-06-26T07:07:11.6772695Z Entering 'third_party/FP16' 2024-06-26T07:07:11.6809798Z Entering 'third_party/FXdiv' 2024-06-26T07:07:11.6845036Z Entering 'third_party/NNPACK' 2024-06-26T07:07:11.6880217Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:11.6914868Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:11.6961097Z Entering 'third_party/benchmark' 2024-06-26T07:07:11.6996477Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:11.7031430Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:11.7067557Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:11.7103467Z Entering 'third_party/cutlass' 2024-06-26T07:07:11.7145540Z Entering 'third_party/eigen' 2024-06-26T07:07:11.7180355Z Entering 'third_party/fbgemm' 2024-06-26T07:07:11.7216025Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:11.7250917Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:11.7283346Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:11.7322210Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:11.7355202Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:11.7391329Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:11.7431145Z Entering 'third_party/fmt' 2024-06-26T07:07:11.7467864Z Entering 'third_party/foxi' 2024-06-26T07:07:11.7503263Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:11.7539297Z Entering 'third_party/gloo' 2024-06-26T07:07:11.7573322Z Entering 'third_party/googletest' 2024-06-26T07:07:11.7609280Z Entering 'third_party/ideep' 2024-06-26T07:07:11.7642570Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:11.7680504Z Entering 'third_party/ittapi' 2024-06-26T07:07:11.7716348Z Entering 'third_party/kineto' 2024-06-26T07:07:11.7751090Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:11.7785325Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:11.7819459Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:11.7852453Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:11.7883814Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:11.7914347Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:11.7951185Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:11.7986325Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:11.8023089Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:11.8058788Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:11.8093991Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:11.8129513Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:11.8165330Z Entering 'third_party/mimalloc' 2024-06-26T07:07:11.8200311Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:11.8235156Z Entering 'third_party/nlohmann' 2024-06-26T07:07:11.8271415Z Entering 'third_party/onnx' 2024-06-26T07:07:11.8318541Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:11.8353202Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:11.8390532Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:11.8428803Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:11.8463478Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:11.8498376Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:11.8532997Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:11.8565956Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:11.8599594Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:11.8632123Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:11.8665029Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:11.8698639Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:11.8735503Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:11.8782038Z Entering 'third_party/pocketfft' 2024-06-26T07:07:11.8818815Z Entering 'third_party/protobuf' 2024-06-26T07:07:11.8856827Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:11.8890417Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:11.8925369Z Entering 'third_party/psimd' 2024-06-26T07:07:11.8961166Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:11.8992859Z Entering 'third_party/pybind11' 2024-06-26T07:07:11.9029563Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:11.9066171Z Entering 'third_party/sleef' 2024-06-26T07:07:11.9102288Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:11.9135304Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:11.9170215Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:11.9204938Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:11.9239569Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:11.9271953Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:11.9321961Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-06-26T07:07:11.9532364Z Entering 'android/libs/fbjni' 2024-06-26T07:07:11.9563964Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-06-26T07:07:11.9577591Z Entering 'third_party/FP16' 2024-06-26T07:07:11.9610343Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-06-26T07:07:11.9622346Z Entering 'third_party/FXdiv' 2024-06-26T07:07:11.9653401Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-06-26T07:07:11.9667539Z Entering 'third_party/NNPACK' 2024-06-26T07:07:11.9697278Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-06-26T07:07:11.9708712Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:11.9739563Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-06-26T07:07:11.9753144Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:11.9782445Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-06-26T07:07:11.9804015Z Entering 'third_party/benchmark' 2024-06-26T07:07:11.9835669Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:11.9848737Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:11.9879568Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-06-26T07:07:11.9893145Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:11.9924546Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:07:11.9938763Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:11.9969845Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-06-26T07:07:11.9983982Z Entering 'third_party/cutlass' 2024-06-26T07:07:12.0014779Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:07:12.0033925Z Entering 'third_party/eigen' 2024-06-26T07:07:12.0066179Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-06-26T07:07:12.0080371Z Entering 'third_party/fbgemm' 2024-06-26T07:07:12.0112065Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-06-26T07:07:12.0125714Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:12.0156240Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-06-26T07:07:12.0168136Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:12.0198516Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:07:12.0216396Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:12.0247635Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:07:12.0264515Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:12.0295483Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:12.0308427Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:12.0339530Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-06-26T07:07:12.0353601Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:12.0387650Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-06-26T07:07:12.0403986Z Entering 'third_party/fmt' 2024-06-26T07:07:12.0434320Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:07:12.0447997Z Entering 'third_party/foxi' 2024-06-26T07:07:12.0479713Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-06-26T07:07:12.0492634Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:12.0523919Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-06-26T07:07:12.0537775Z Entering 'third_party/gloo' 2024-06-26T07:07:12.0569843Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-06-26T07:07:12.0583341Z Entering 'third_party/googletest' 2024-06-26T07:07:12.0616386Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:12.0628050Z Entering 'third_party/ideep' 2024-06-26T07:07:12.0659151Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-06-26T07:07:12.0671313Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:12.0703362Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-06-26T07:07:12.0722742Z Entering 'third_party/ittapi' 2024-06-26T07:07:12.0755500Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-06-26T07:07:12.0767912Z Entering 'third_party/kineto' 2024-06-26T07:07:12.0799556Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-06-26T07:07:12.0813339Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:12.0846497Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-06-26T07:07:12.0859629Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:12.0891304Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-06-26T07:07:12.0905558Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:12.0937230Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-06-26T07:07:12.0950469Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:12.0983605Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:07:12.0996706Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:12.1028798Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-06-26T07:07:12.1041028Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:12.1073109Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-06-26T07:07:12.1086752Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:12.1118438Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-06-26T07:07:12.1131334Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:12.1163407Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:12.1176462Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:12.1207938Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-06-26T07:07:12.1221127Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:12.1253232Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-06-26T07:07:12.1266490Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:12.1298974Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-06-26T07:07:12.1311569Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:12.1343635Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-06-26T07:07:12.1357437Z Entering 'third_party/mimalloc' 2024-06-26T07:07:12.1391284Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-06-26T07:07:12.1404157Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:12.1435896Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-06-26T07:07:12.1448729Z Entering 'third_party/nlohmann' 2024-06-26T07:07:12.1480967Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-06-26T07:07:12.1495333Z Entering 'third_party/onnx' 2024-06-26T07:07:12.1526421Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-06-26T07:07:12.1551431Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:12.1582761Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:12.1596207Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:12.1628346Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:07:12.1643937Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:12.1675836Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-06-26T07:07:12.1688425Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:12.1719756Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:12.1731968Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:12.1763938Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:12.1777437Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:12.1808718Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-06-26T07:07:12.1821520Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:12.1852406Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-06-26T07:07:12.1865824Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:12.1897382Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-06-26T07:07:12.1909524Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:12.1941569Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-06-26T07:07:12.1952439Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:12.1986478Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-06-26T07:07:12.1999682Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:12.2029863Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-06-26T07:07:12.2044871Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:12.2077422Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-06-26T07:07:12.2091314Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:12.2121601Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-06-26T07:07:12.2148261Z Entering 'third_party/pocketfft' 2024-06-26T07:07:12.2180152Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-06-26T07:07:12.2193338Z Entering 'third_party/protobuf' 2024-06-26T07:07:12.2224999Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-06-26T07:07:12.2241281Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:12.2272937Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:12.2285396Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:12.2316281Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:12.2329976Z Entering 'third_party/psimd' 2024-06-26T07:07:12.2362437Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-06-26T07:07:12.2374771Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:12.2406769Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-06-26T07:07:12.2419265Z Entering 'third_party/pybind11' 2024-06-26T07:07:12.2454027Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:07:12.2467954Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:12.2499954Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-06-26T07:07:12.2511923Z Entering 'third_party/sleef' 2024-06-26T07:07:12.2542744Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-06-26T07:07:12.2555876Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:12.2589886Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-06-26T07:07:12.2603880Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:12.2634534Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:12.2647805Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:12.2680202Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-06-26T07:07:12.2691932Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:12.2722665Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-06-26T07:07:12.2735959Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:12.2767232Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:07:12.2777711Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:12.2809469Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-06-26T07:07:12.3702939Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-06-26T07:07:12.3915143Z Entering 'android/libs/fbjni' 2024-06-26T07:07:12.3941471Z Entering 'third_party/FP16' 2024-06-26T07:07:12.3971270Z Entering 'third_party/FXdiv' 2024-06-26T07:07:12.4000797Z Entering 'third_party/NNPACK' 2024-06-26T07:07:12.4028933Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:12.4056870Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:12.4094907Z Entering 'third_party/benchmark' 2024-06-26T07:07:12.4123601Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:12.4151572Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:12.4177622Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:12.4207770Z Entering 'third_party/cutlass' 2024-06-26T07:07:12.4241091Z Entering 'third_party/eigen' 2024-06-26T07:07:12.4271302Z Entering 'third_party/fbgemm' 2024-06-26T07:07:12.4301274Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:12.4328956Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:12.4357250Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:12.4389417Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:12.4418748Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:12.4447047Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:12.4475972Z Entering 'third_party/fmt' 2024-06-26T07:07:12.4502633Z Entering 'third_party/foxi' 2024-06-26T07:07:12.4530280Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:12.4558928Z Entering 'third_party/gloo' 2024-06-26T07:07:12.4588825Z Entering 'third_party/googletest' 2024-06-26T07:07:12.4615696Z Entering 'third_party/ideep' 2024-06-26T07:07:12.4642225Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:12.4675240Z Entering 'third_party/ittapi' 2024-06-26T07:07:12.4702406Z Entering 'third_party/kineto' 2024-06-26T07:07:12.4728828Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:12.4756977Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:12.4786794Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:12.4815082Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:12.4842615Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:12.4869983Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:12.4899880Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:12.4927915Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:12.4956356Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:12.4983441Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:12.5013247Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:12.5041218Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:12.5071658Z Entering 'third_party/mimalloc' 2024-06-26T07:07:12.5100997Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:12.5129080Z Entering 'third_party/nlohmann' 2024-06-26T07:07:12.5156114Z Entering 'third_party/onnx' 2024-06-26T07:07:12.5197007Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:12.5224229Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:12.5254319Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:12.5286027Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:12.5312355Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:12.5338654Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:12.5367118Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:12.5394862Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:12.5420635Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:12.5448303Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:12.5474806Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:12.5502349Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:12.5528258Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:12.5572881Z Entering 'third_party/pocketfft' 2024-06-26T07:07:12.5602823Z Entering 'third_party/protobuf' 2024-06-26T07:07:12.5632926Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:12.5660806Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:12.5690105Z Entering 'third_party/psimd' 2024-06-26T07:07:12.5720021Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:12.5746992Z Entering 'third_party/pybind11' 2024-06-26T07:07:12.5776525Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:12.5803272Z Entering 'third_party/sleef' 2024-06-26T07:07:12.5831551Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:12.5857897Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:12.5886336Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:12.5911617Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:12.5936555Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:12.5960847Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:12.5998905Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-06-26T07:07:12.6200662Z Entering 'android/libs/fbjni' 2024-06-26T07:07:12.6228155Z Entering 'third_party/FP16' 2024-06-26T07:07:12.6256389Z Entering 'third_party/FXdiv' 2024-06-26T07:07:12.6281783Z Entering 'third_party/NNPACK' 2024-06-26T07:07:12.6310400Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:12.6335861Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:12.6372490Z Entering 'third_party/benchmark' 2024-06-26T07:07:12.6400751Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:12.6429332Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:12.6456715Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:12.6485565Z Entering 'third_party/cutlass' 2024-06-26T07:07:12.6518738Z Entering 'third_party/eigen' 2024-06-26T07:07:12.6549316Z Entering 'third_party/fbgemm' 2024-06-26T07:07:12.6575780Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:12.6605323Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:12.6631486Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:12.6664237Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:12.6691378Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:12.6719695Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:12.6751612Z Entering 'third_party/fmt' 2024-06-26T07:07:12.6778261Z Entering 'third_party/foxi' 2024-06-26T07:07:12.6805060Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:12.6833853Z Entering 'third_party/gloo' 2024-06-26T07:07:12.6862466Z Entering 'third_party/googletest' 2024-06-26T07:07:12.6887866Z Entering 'third_party/ideep' 2024-06-26T07:07:12.6913890Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:12.6948074Z Entering 'third_party/ittapi' 2024-06-26T07:07:12.6974718Z Entering 'third_party/kineto' 2024-06-26T07:07:12.7002013Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:12.7029532Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:12.7056265Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:12.7082530Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:12.7110670Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:12.7137664Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:12.7166418Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:12.7193275Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:12.7222884Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:12.7251464Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:12.7281316Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:12.7308757Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:12.7337173Z Entering 'third_party/mimalloc' 2024-06-26T07:07:12.7365690Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:12.7393177Z Entering 'third_party/nlohmann' 2024-06-26T07:07:12.7420037Z Entering 'third_party/onnx' 2024-06-26T07:07:12.7460956Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:12.7489916Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:12.7519786Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:12.7549493Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:12.7576830Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:12.7603702Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:12.7629813Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:12.7657484Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:12.7685102Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:12.7711967Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:12.7737426Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:12.7767348Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:12.7794359Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:12.7838822Z Entering 'third_party/pocketfft' 2024-06-26T07:07:12.7867591Z Entering 'third_party/protobuf' 2024-06-26T07:07:12.7897162Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:12.7925107Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:12.7953669Z Entering 'third_party/psimd' 2024-06-26T07:07:12.7983149Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:12.8008591Z Entering 'third_party/pybind11' 2024-06-26T07:07:12.8036958Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:12.8067487Z Entering 'third_party/sleef' 2024-06-26T07:07:12.8094563Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:12.8123979Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:12.8151492Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:12.8178001Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:12.8206051Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:12.8233099Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:12.8277765Z ##[endgroup] 2024-06-26T07:07:12.8307696Z [command]/usr/bin/git log -1 --format='%H' 2024-06-26T07:07:12.8324311Z 'b8c4c54d347aa776934c60784e35936878ef18dc' 2024-06-26T07:07:12.8495414Z Prepare all required actions 2024-06-26T07:07:12.8495875Z Getting action download info 2024-06-26T07:07:12.9838754Z ##[group]Run ./.github/actions/setup-linux 2024-06-26T07:07:12.9839369Z env: 2024-06-26T07:07:12.9839668Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:12.9840036Z ##[endgroup] 2024-06-26T07:07:12.9899578Z ##[group]Run set -euo pipefail 2024-06-26T07:07:12.9900014Z set -euo pipefail 2024-06-26T07:07:12.9900420Z function get_ec2_metadata() { 2024-06-26T07:07:12.9900969Z  # Pulled from instance metadata endpoint for EC2 2024-06-26T07:07:12.9901877Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-06-26T07:07:12.9902660Z  category=$1 2024-06-26T07:07:12.9903195Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-06-26T07:07:12.9903844Z  runner_name_str=i-036744df35e90073e 2024-06-26T07:07:12.9904358Z  if [[ -f /.inarc ]]; then 2024-06-26T07:07:12.9904863Z  echo "ARC Runner, no info on ec2 metadata" 2024-06-26T07:07:12.9905447Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-06-26T07:07:12.9906167Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-06-26T07:07:12.9906776Z  else 2024-06-26T07:07:12.9907275Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-06-26T07:07:12.9907864Z  fi 2024-06-26T07:07:12.9908161Z } 2024-06-26T07:07:12.9908527Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-06-26T07:07:12.9909128Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-06-26T07:07:12.9909799Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-06-26T07:07:12.9910374Z echo "system info $(uname -a)" 2024-06-26T07:07:12.9918250Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:12.9918759Z env: 2024-06-26T07:07:12.9919055Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:12.9919416Z ##[endgroup] 2024-06-26T07:07:12.9992950Z ami-id: ami-0ce0c36d7a00b20e2 2024-06-26T07:07:13.0038690Z instance-id: i-036744df35e90073e 2024-06-26T07:07:13.0081218Z instance-type: g5.4xlarge 2024-06-26T07:07:13.0088250Z system info Linux ip-10-0-27-148.ec2.internal 4.14.336-257.562.amzn2.x86_64 #1 SMP Sat Feb 24 09:50:35 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-06-26T07:07:13.0108181Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:07:13.0109122Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:07:13.0117066Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:13.0117582Z env: 2024-06-26T07:07:13.0117864Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:13.0118366Z ##[endgroup] 2024-06-26T07:07:13.0178976Z ##[group]Run if systemctl is-active --quiet docker; then 2024-06-26T07:07:13.0179610Z if systemctl is-active --quiet docker; then 2024-06-26T07:07:13.0180169Z  echo "Docker daemon is running..."; 2024-06-26T07:07:13.0180774Z else 2024-06-26T07:07:13.0181290Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-06-26T07:07:13.0181901Z fi 2024-06-26T07:07:13.0189298Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:13.0189800Z env: 2024-06-26T07:07:13.0190089Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:13.0190443Z ##[endgroup] 2024-06-26T07:07:13.0230523Z Docker daemon is running... 2024-06-26T07:07:13.0272880Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:07:13.0273440Z with: 2024-06-26T07:07:13.0273723Z shell: bash 2024-06-26T07:07:13.0274015Z timeout_minutes: 5 2024-06-26T07:07:13.0274349Z max_attempts: 3 2024-06-26T07:07:13.0274673Z retry_wait_seconds: 30 2024-06-26T07:07:13.0276188Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-06-26T07:07:13.0277647Z polling_interval_seconds: 1 2024-06-26T07:07:13.0278228Z warning_on_retry: true 2024-06-26T07:07:13.0278705Z continue_on_error: false 2024-06-26T07:07:13.0279066Z env: 2024-06-26T07:07:13.0279335Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:13.0279706Z AWS_RETRY_MODE: standard 2024-06-26T07:07:13.0280066Z AWS_MAX_ATTEMPTS: 5 2024-06-26T07:07:13.0280416Z AWS_DEFAULT_REGION: us-east-1 2024-06-26T07:07:13.0280806Z ##[endgroup] 2024-06-26T07:07:13.8635655Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:07:13.8636480Z Configure a credential helper to remove this warning. See 2024-06-26T07:07:13.8637315Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:07:13.8637833Z 2024-06-26T07:07:13.8639713Z Login Succeeded 2024-06-26T07:07:14.0783347Z Command completed after 1 attempt(s). 2024-06-26T07:07:14.0824264Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:07:14.0825025Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:07:14.0825714Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:07:14.0833710Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:14.0834217Z env: 2024-06-26T07:07:14.0834513Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:14.0834877Z ##[endgroup] 2024-06-26T07:07:14.0895089Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:07:14.0895897Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:07:14.0896549Z # shellcheck disable=SC2046 2024-06-26T07:07:14.0897055Z docker stop $(docker ps -q) || true 2024-06-26T07:07:14.0897702Z # Prune all of the docker images 2024-06-26T07:07:14.0898164Z docker system prune -af 2024-06-26T07:07:14.0905908Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:14.0906440Z env: 2024-06-26T07:07:14.0906731Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:14.0907087Z ##[endgroup] 2024-06-26T07:07:14.1146091Z "docker stop" requires at least 1 argument. 2024-06-26T07:07:14.1146724Z See 'docker stop --help'. 2024-06-26T07:07:14.1146960Z 2024-06-26T07:07:14.1147186Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-06-26T07:07:14.1147569Z 2024-06-26T07:07:14.1147723Z Stop one or more running containers 2024-06-26T07:07:14.1297080Z Total reclaimed space: 0B 2024-06-26T07:07:14.1325965Z ##[group]Run set +e 2024-06-26T07:07:14.1326326Z set +e 2024-06-26T07:07:14.1326637Z set -x 2024-06-26T07:07:14.1326941Z  2024-06-26T07:07:14.1327266Z PT_DOMAIN=download.pytorch.org 2024-06-26T07:07:14.1328078Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-06-26T07:07:14.1329182Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-06-26T07:07:14.1330052Z # one is returned at random 2024-06-26T07:07:14.1330612Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-06-26T07:07:14.1331131Z  2024-06-26T07:07:14.1331456Z if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:07:14.1332087Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-06-26T07:07:14.1332857Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-06-26T07:07:14.1333411Z  2024-06-26T07:07:14.1333750Z  if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:07:14.1334312Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-06-26T07:07:14.1334833Z  exit 1 2024-06-26T07:07:14.1335147Z  fi 2024-06-26T07:07:14.1335440Z fi 2024-06-26T07:07:14.1335815Z  2024-06-26T07:07:14.1336197Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-06-26T07:07:14.1336756Z  # Clean up any old records first 2024-06-26T07:07:14.1337383Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-06-26T07:07:14.1337853Z fi 2024-06-26T07:07:14.1338131Z  2024-06-26T07:07:14.1338575Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-06-26T07:07:14.1339137Z cat /etc/hosts 2024-06-26T07:07:14.1346919Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:14.1347430Z env: 2024-06-26T07:07:14.1347719Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:14.1348084Z ##[endgroup] 2024-06-26T07:07:14.1367383Z + PT_DOMAIN=download.pytorch.org 2024-06-26T07:07:14.1371567Z ++ dig -4 +short download.pytorch.org 2024-06-26T07:07:14.1372051Z ++ tail -n1 2024-06-26T07:07:14.1460512Z + RESOLVED_IP=18.160.10.28 2024-06-26T07:07:14.1461177Z + '[' -z 18.160.10.28 ']' 2024-06-26T07:07:14.1461772Z + grep -r download.pytorch.org /etc/hosts 2024-06-26T07:07:14.1467151Z 18.160.18.34 download.pytorch.org 2024-06-26T07:07:14.1468048Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-06-26T07:07:14.1552670Z + echo '18.160.10.28 download.pytorch.org' 2024-06-26T07:07:14.1553355Z + sudo tee -a /etc/hosts 2024-06-26T07:07:14.1614428Z 18.160.10.28 download.pytorch.org 2024-06-26T07:07:14.1627688Z + cat /etc/hosts 2024-06-26T07:07:14.1632612Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-06-26T07:07:14.1645752Z ::1 localhost6 localhost6.localdomain6 2024-06-26T07:07:14.1646269Z 18.160.10.28 download.pytorch.org 2024-06-26T07:07:14.1832623Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-06-26T07:07:14.1833234Z with: 2024-06-26T07:07:14.1834337Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.1835551Z docker-build-dir: .ci/docker 2024-06-26T07:07:14.1835927Z working-directory: . 2024-06-26T07:07:14.1836416Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:14.1836948Z force-push: false 2024-06-26T07:07:14.1837248Z env: 2024-06-26T07:07:14.1837525Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:14.1837864Z ##[endgroup] 2024-06-26T07:07:14.1855115Z ##[group]Run set -ex 2024-06-26T07:07:14.1855484Z set -ex 2024-06-26T07:07:14.1855783Z  2024-06-26T07:07:14.1856355Z # If the docker build directory or the build script doesn't exist, the action will 2024-06-26T07:07:14.1857453Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-06-26T07:07:14.1858276Z # job could then download the pre-built image as usual 2024-06-26T07:07:14.1859039Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-06-26T07:07:14.1859737Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:14.1860396Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:14.1861089Z  2024-06-26T07:07:14.1861610Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-06-26T07:07:14.1862264Z  exit 0 2024-06-26T07:07:14.1862585Z else 2024-06-26T07:07:14.1862974Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:14.1863433Z fi 2024-06-26T07:07:14.1863724Z  2024-06-26T07:07:14.1864214Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-06-26T07:07:14.1865097Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-06-26T07:07:14.1865885Z  # use it as it is, but first let's extract the tag 2024-06-26T07:07:14.1866602Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-06-26T07:07:14.1867338Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:14.1868055Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:14.1868641Z else 2024-06-26T07:07:14.1869093Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-06-26T07:07:14.1869776Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:14.1870688Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:14.1871443Z fi 2024-06-26T07:07:14.1879525Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:14.1880037Z env: 2024-06-26T07:07:14.1880332Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:14.1880706Z REPO_NAME: pytorch 2024-06-26T07:07:14.1881871Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.1883099Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:07:14.1883616Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:14.1884152Z ##[endgroup] 2024-06-26T07:07:14.1904151Z + [[ ! -d .ci/docker ]] 2024-06-26T07:07:14.1904599Z + [[ ! -f .ci/docker/build.sh ]] 2024-06-26T07:07:14.1904991Z + echo skip=false 2024-06-26T07:07:14.1906971Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-06-26T07:07:14.1910051Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.1911736Z ++ awk -F '[:,]' '{print $2}' 2024-06-26T07:07:14.1916784Z + DOCKER_TAG=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.1917680Z + echo docker-tag=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.1919666Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.1946110Z ##[group]Run set +e 2024-06-26T07:07:14.1946491Z set +e 2024-06-26T07:07:14.1947060Z set -x 2024-06-26T07:07:14.1947362Z  2024-06-26T07:07:14.1947660Z login() { 2024-06-26T07:07:14.1948362Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:07:14.1949102Z } 2024-06-26T07:07:14.1949384Z  2024-06-26T07:07:14.1949682Z retry () { 2024-06-26T07:07:14.1950088Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:07:14.1950564Z } 2024-06-26T07:07:14.1950846Z  2024-06-26T07:07:14.1951184Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:07:14.1951614Z  2024-06-26T07:07:14.1952103Z # Check if image already exists, if it does then skip building it 2024-06-26T07:07:14.1952941Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-06-26T07:07:14.1953468Z  exit 0 2024-06-26T07:07:14.1953789Z fi 2024-06-26T07:07:14.1954072Z  2024-06-26T07:07:14.1954597Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-06-26T07:07:14.1955459Z # be empty. The default action would be to continue rebuild the image 2024-06-26T07:07:14.1956239Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-06-26T07:07:14.1956938Z  # if we're on the base branch then use the parent commit 2024-06-26T07:07:14.1957562Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-06-26T07:07:14.1958189Z else 2024-06-26T07:07:14.1958734Z  # otherwise we're on a PR, so use the most recent base commit 2024-06-26T07:07:14.1959438Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-06-26T07:07:14.1959976Z fi 2024-06-26T07:07:14.1960268Z  2024-06-26T07:07:14.1960607Z if [[ -z "${MERGE_BASE}" ]]; then 2024-06-26T07:07:14.1961136Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:14.1961647Z  2024-06-26T07:07:14.1962325Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-06-26T07:07:14.1963108Z  exit 0 2024-06-26T07:07:14.1963412Z fi 2024-06-26T07:07:14.1963706Z  2024-06-26T07:07:14.1964161Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-06-26T07:07:14.1965158Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-06-26T07:07:14.1965984Z  exit 1 2024-06-26T07:07:14.1966297Z fi 2024-06-26T07:07:14.1966590Z  2024-06-26T07:07:14.1967120Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-06-26T07:07:14.1968112Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-06-26T07:07:14.1969018Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-06-26T07:07:14.1970021Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-06-26T07:07:14.1971159Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-06-26T07:07:14.1971996Z fi 2024-06-26T07:07:14.1972290Z  2024-06-26T07:07:14.1972681Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:14.1980689Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:14.1981222Z env: 2024-06-26T07:07:14.1981519Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:14.1981915Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:07:14.1982397Z BASE_REVISION: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:07:14.1983704Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.1984974Z DOCKER_TAG: 91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.1985620Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:14.1986160Z ##[endgroup] 2024-06-26T07:07:14.2008602Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:14.2010449Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:14.2011667Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:07:14.2012842Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:14.6072814Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:07:14.6073696Z Configure a credential helper to remove this warning. See 2024-06-26T07:07:14.6074925Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:07:14.6075454Z 2024-06-26T07:07:14.6075579Z Login Succeeded 2024-06-26T07:07:14.6084715Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.8028875Z { 2024-06-26T07:07:14.8029266Z "schemaVersion": 2, 2024-06-26T07:07:14.8029948Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-06-26T07:07:14.8030532Z "config": { 2024-06-26T07:07:14.8030978Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-06-26T07:07:14.8032069Z "size": 48681, 2024-06-26T07:07:14.8032795Z "digest": "sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd" 2024-06-26T07:07:14.8033642Z }, 2024-06-26T07:07:14.8033983Z "layers": [ 2024-06-26T07:07:14.8034363Z { 2024-06-26T07:07:14.8034806Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8035521Z "size": 28580681, 2024-06-26T07:07:14.8036205Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-06-26T07:07:14.8037053Z }, 2024-06-26T07:07:14.8037315Z { 2024-06-26T07:07:14.8037903Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8039029Z "size": 7943451, 2024-06-26T07:07:14.8039730Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-06-26T07:07:14.8040709Z }, 2024-06-26T07:07:14.8041096Z { 2024-06-26T07:07:14.8041664Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8042411Z "size": 55728572, 2024-06-26T07:07:14.8043142Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-06-26T07:07:14.8043966Z }, 2024-06-26T07:07:14.8044283Z { 2024-06-26T07:07:14.8045109Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8045678Z "size": 186, 2024-06-26T07:07:14.8046223Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-06-26T07:07:14.8046842Z }, 2024-06-26T07:07:14.8047086Z { 2024-06-26T07:07:14.8047518Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8048199Z + exit 0 2024-06-26T07:07:14.8048465Z "size": 6886, 2024-06-26T07:07:14.8049003Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-06-26T07:07:14.8049803Z }, 2024-06-26T07:07:14.8050054Z { 2024-06-26T07:07:14.8050491Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8051060Z "size": 1329001756, 2024-06-26T07:07:14.8051648Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-06-26T07:07:14.8052266Z }, 2024-06-26T07:07:14.8052511Z { 2024-06-26T07:07:14.8052941Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8053494Z "size": 62501, 2024-06-26T07:07:14.8054029Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-06-26T07:07:14.8054648Z }, 2024-06-26T07:07:14.8054897Z { 2024-06-26T07:07:14.8055317Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8055867Z "size": 1684, 2024-06-26T07:07:14.8056398Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-06-26T07:07:14.8057006Z }, 2024-06-26T07:07:14.8057258Z { 2024-06-26T07:07:14.8057812Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8058376Z "size": 1523, 2024-06-26T07:07:14.8058952Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-06-26T07:07:14.8059574Z }, 2024-06-26T07:07:14.8059825Z { 2024-06-26T07:07:14.8060262Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8060811Z "size": 2528295403, 2024-06-26T07:07:14.8061492Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-06-26T07:07:14.8062115Z }, 2024-06-26T07:07:14.8062369Z { 2024-06-26T07:07:14.8062795Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8063361Z "size": 86016, 2024-06-26T07:07:14.8063922Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-06-26T07:07:14.8064537Z }, 2024-06-26T07:07:14.8064779Z { 2024-06-26T07:07:14.8065223Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8065777Z "size": 1823, 2024-06-26T07:07:14.8066319Z "digest": "sha256:309b872da805fcdf74137a887464876fa0615dc8b24199307119eb65fc2bfb49" 2024-06-26T07:07:14.8066916Z }, 2024-06-26T07:07:14.8067166Z { 2024-06-26T07:07:14.8067604Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8068150Z "size": 246737542, 2024-06-26T07:07:14.8068707Z "digest": "sha256:e82932cb002c06a782077b8e5965f8733eb8563406a23cab260d9d8c4a994f7d" 2024-06-26T07:07:14.8069322Z }, 2024-06-26T07:07:14.8069572Z { 2024-06-26T07:07:14.8069999Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8070552Z "size": 545, 2024-06-26T07:07:14.8071158Z "digest": "sha256:ca9dd73e3e9378225ebdcfd12199ef44649e34fe55003cfbc8afe2e8a4527c5b" 2024-06-26T07:07:14.8071781Z }, 2024-06-26T07:07:14.8072024Z { 2024-06-26T07:07:14.8072461Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8073012Z "size": 1283, 2024-06-26T07:07:14.8073546Z "digest": "sha256:16d76a79d5d6616e228537785ec83a869f68a402520bbc6e826201621ca6cb98" 2024-06-26T07:07:14.8074153Z }, 2024-06-26T07:07:14.8074401Z { 2024-06-26T07:07:14.8074838Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8075380Z "size": 485, 2024-06-26T07:07:14.8075930Z "digest": "sha256:4bdf41c9ecbe1651e0986ccbea4663a0556af27b7145a679cfa45c35dc96b891" 2024-06-26T07:07:14.8076569Z }, 2024-06-26T07:07:14.8076808Z { 2024-06-26T07:07:14.8077231Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8077783Z "size": 91710937, 2024-06-26T07:07:14.8078581Z "digest": "sha256:f4863801e30cdbcd1d4e6cb1cbbcde7dbcaf6ae317ca80429295741ce60ac31d" 2024-06-26T07:07:14.8079220Z }, 2024-06-26T07:07:14.8079471Z { 2024-06-26T07:07:14.8079911Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8080584Z "size": 3233, 2024-06-26T07:07:14.8081135Z "digest": "sha256:d37dd9ed8948a0d02e1198bbedadbca36477a0980dd553196385aab0c025581d" 2024-06-26T07:07:14.8081816Z }, 2024-06-26T07:07:14.8082071Z { 2024-06-26T07:07:14.8082514Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8083068Z "size": 1905, 2024-06-26T07:07:14.8083631Z "digest": "sha256:cf2f34f225dbed17c1a9a8b9ea4ad030aba797774a716dc0a82fe212324aa751" 2024-06-26T07:07:14.8084283Z }, 2024-06-26T07:07:14.8084548Z { 2024-06-26T07:07:14.8084987Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8085549Z "size": 702, 2024-06-26T07:07:14.8086128Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:07:14.8086771Z }, 2024-06-26T07:07:14.8087027Z { 2024-06-26T07:07:14.8087476Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8088054Z "size": 2784789002, 2024-06-26T07:07:14.8088638Z "digest": "sha256:3d914950f8addfc4d657a2365c76e1143eb80b710cd3f6e9ae05640217fc6347" 2024-06-26T07:07:14.8089273Z }, 2024-06-26T07:07:14.8089550Z { 2024-06-26T07:07:14.8090009Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8090557Z "size": 381, 2024-06-26T07:07:14.8091115Z "digest": "sha256:96d0e2032184c57e62a652aaffcf0241f0d0c7a23e894ca0ad192c4a64a36583" 2024-06-26T07:07:14.8091786Z }, 2024-06-26T07:07:14.8092128Z { 2024-06-26T07:07:14.8092575Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8093156Z "size": 11879, 2024-06-26T07:07:14.8093727Z "digest": "sha256:06e12ce59790cb8b091e5d619c2a62a28329008ee184a8b9a562e3db12471f30" 2024-06-26T07:07:14.8094334Z }, 2024-06-26T07:07:14.8094610Z { 2024-06-26T07:07:14.8095073Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8095653Z "size": 803, 2024-06-26T07:07:14.8096223Z "digest": "sha256:ed712f63cde7070bdec2a6179c4c35e752c6991ddbc69fe9f2724a9903250ceb" 2024-06-26T07:07:14.8096862Z }, 2024-06-26T07:07:14.8097133Z { 2024-06-26T07:07:14.8097649Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8098208Z "size": 106, 2024-06-26T07:07:14.8098756Z "digest": "sha256:a97a71ef8a4e66234dfc2098834f8395ac5630240e60164d1a251fdb0115ba85" 2024-06-26T07:07:14.8099371Z }, 2024-06-26T07:07:14.8099639Z { 2024-06-26T07:07:14.8100092Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8100669Z "size": 503, 2024-06-26T07:07:14.8101234Z "digest": "sha256:25cd0c482ad612f936b35b4f220c580e16be982226aac8801a6a849a6bfdc221" 2024-06-26T07:07:14.8101853Z }, 2024-06-26T07:07:14.8102131Z { 2024-06-26T07:07:14.8102576Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8103164Z "size": 121477177, 2024-06-26T07:07:14.8103757Z "digest": "sha256:9bf94bc456584365fda21d0ed83b7cc690b08a71727b71c4c1e667e15028a386" 2024-06-26T07:07:14.8104381Z }, 2024-06-26T07:07:14.8104649Z { 2024-06-26T07:07:14.8105101Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8105662Z "size": 109, 2024-06-26T07:07:14.8106229Z "digest": "sha256:de45a14c9b227e4b0c9d45d42223b50a71c03522ddd9e2ca0d9517835206c823" 2024-06-26T07:07:14.8106854Z }, 2024-06-26T07:07:14.8107107Z { 2024-06-26T07:07:14.8107560Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8108144Z "size": 491, 2024-06-26T07:07:14.8108706Z "digest": "sha256:e99241e77209b0b35a0fb59a61c0821fc161190178af7e4af45e6eeaa043375e" 2024-06-26T07:07:14.8109322Z }, 2024-06-26T07:07:14.8109587Z { 2024-06-26T07:07:14.8110034Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8110588Z "size": 291, 2024-06-26T07:07:14.8111138Z "digest": "sha256:35e8f4e2a42f9df0c969dedd2199cf71719cf5ecfa676dfa014bbdeb4fdb295f" 2024-06-26T07:07:14.8111867Z }, 2024-06-26T07:07:14.8112129Z { 2024-06-26T07:07:14.8112571Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8113148Z "size": 101, 2024-06-26T07:07:14.8113715Z "digest": "sha256:a823ffa4c541394b52953391dc0a6867e26c37fe4a71fa022a2cc3f07f0ba70d" 2024-06-26T07:07:14.8114346Z }, 2024-06-26T07:07:14.8114590Z { 2024-06-26T07:07:14.8115041Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8115621Z "size": 1473, 2024-06-26T07:07:14.8116178Z "digest": "sha256:44ba272eae2064603605a6238b3f35cc6578859d660068e702a047a9cd8ab2fd" 2024-06-26T07:07:14.8116786Z }, 2024-06-26T07:07:14.8117045Z { 2024-06-26T07:07:14.8117491Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8118233Z "size": 446710781, 2024-06-26T07:07:14.8118793Z "digest": "sha256:da02d278e5240c285bc855ce4e42cd6121d5b903787f5c14f03e575ad1268650" 2024-06-26T07:07:14.8119412Z }, 2024-06-26T07:07:14.8119683Z { 2024-06-26T07:07:14.8120121Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8120681Z "size": 161, 2024-06-26T07:07:14.8121218Z "digest": "sha256:73a0ce681c295135456e190578733273276b38b4dd5babbb39298f384d43b1c6" 2024-06-26T07:07:14.8121832Z }, 2024-06-26T07:07:14.8122080Z { 2024-06-26T07:07:14.8122529Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8123093Z "size": 564, 2024-06-26T07:07:14.8123743Z "digest": "sha256:49737bd5ae709dbf7956e21fc8b05ba009e9e6c9b130b0126a73bd3d3f5efc78" 2024-06-26T07:07:14.8124366Z }, 2024-06-26T07:07:14.8124629Z { 2024-06-26T07:07:14.8125074Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8125647Z "size": 35874600, 2024-06-26T07:07:14.8126213Z "digest": "sha256:9cd4c745f8e4f9d5c9bb8050923575afad21f5b7852975a4a1981bc9eff282c5" 2024-06-26T07:07:14.8126837Z }, 2024-06-26T07:07:14.8127102Z { 2024-06-26T07:07:14.8127538Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8128106Z "size": 104, 2024-06-26T07:07:14.8128646Z "digest": "sha256:b2950a0365345d4c17a48627c905fd8930b3b3154c727f0526eb9ca547102bb7" 2024-06-26T07:07:14.8129252Z }, 2024-06-26T07:07:14.8129499Z { 2024-06-26T07:07:14.8129939Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8130503Z "size": 425, 2024-06-26T07:07:14.8131045Z "digest": "sha256:25e19d81d9fc081962ff4cbc09e098f4035419822712c19a410f5282300dfb9e" 2024-06-26T07:07:14.8131642Z }, 2024-06-26T07:07:14.8131901Z { 2024-06-26T07:07:14.8132343Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8132890Z "size": 20262084, 2024-06-26T07:07:14.8133470Z "digest": "sha256:60156b40fcbec2c25f0170b0a82fa8cd9303e0daae8bb1811fd0fc6562e88e8e" 2024-06-26T07:07:14.8134089Z }, 2024-06-26T07:07:14.8134352Z { 2024-06-26T07:07:14.8134794Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8135355Z "size": 439, 2024-06-26T07:07:14.8135899Z "digest": "sha256:7a868239227fd19b30a19927bbc8f058d247403fe5b343c55f4b4b8987918911" 2024-06-26T07:07:14.8136504Z }, 2024-06-26T07:07:14.8136750Z { 2024-06-26T07:07:14.8137196Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8137829Z "size": 702, 2024-06-26T07:07:14.8138382Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:07:14.8138997Z }, 2024-06-26T07:07:14.8139250Z { 2024-06-26T07:07:14.8139694Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8140247Z "size": 143, 2024-06-26T07:07:14.8140798Z "digest": "sha256:83911d7c7ecf11b5932bb7449975c52e7f737a5b867691935e1ccb85ceb25b52" 2024-06-26T07:07:14.8141418Z }, 2024-06-26T07:07:14.8141678Z { 2024-06-26T07:07:14.8142105Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8142748Z "size": 135, 2024-06-26T07:07:14.8143314Z "digest": "sha256:b3c5d66c679f58c71febdd502889bd8cd3d36e5001754cdf1e30eabe21135cce" 2024-06-26T07:07:14.8143960Z }, 2024-06-26T07:07:14.8144204Z { 2024-06-26T07:07:14.8144651Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8145213Z "size": 4911495247, 2024-06-26T07:07:14.8145769Z "digest": "sha256:84b2c7a899820f6acdb22148c5e8346d8b26c33269fd01842c9573515251e1e1" 2024-06-26T07:07:14.8146387Z }, 2024-06-26T07:07:14.8146644Z { 2024-06-26T07:07:14.8147086Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8147634Z "size": 190, 2024-06-26T07:07:14.8148189Z "digest": "sha256:c1068d03a811d8dead5da9e82fdc8869205fc4ee9cb92d4f7a4acfbb5ba83950" 2024-06-26T07:07:14.8148828Z }, 2024-06-26T07:07:14.8149085Z { 2024-06-26T07:07:14.8149520Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8150087Z "size": 565, 2024-06-26T07:07:14.8150641Z "digest": "sha256:671b4a301d81074fca17d36851f81801552c4ecb2f90ff3412aa9ab6a1a91371" 2024-06-26T07:07:14.8151246Z }, 2024-06-26T07:07:14.8151492Z { 2024-06-26T07:07:14.8151935Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8152502Z "size": 43163095, 2024-06-26T07:07:14.8153064Z "digest": "sha256:67760cf5ec46ab58d3ca98637fd9b26cb47a67446bfd0db176fe5e023606aa0d" 2024-06-26T07:07:14.8153696Z }, 2024-06-26T07:07:14.8154018Z { 2024-06-26T07:07:14.8154463Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8155017Z "size": 106, 2024-06-26T07:07:14.8155570Z "digest": "sha256:ddc9740bcea2b301e99b50525059aea8d9c7121a1c8eab566561a8968438a8c2" 2024-06-26T07:07:14.8156199Z }, 2024-06-26T07:07:14.8156457Z { 2024-06-26T07:07:14.8156894Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8157480Z "size": 1213, 2024-06-26T07:07:14.8158181Z "digest": "sha256:fb2e977685609fcc01de170ba7f50a573e73e6520e47253717e92a28952c5ee8" 2024-06-26T07:07:14.8158796Z }, 2024-06-26T07:07:14.8159039Z { 2024-06-26T07:07:14.8159469Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8160037Z "size": 702, 2024-06-26T07:07:14.8160580Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:07:14.8161221Z }, 2024-06-26T07:07:14.8161475Z { 2024-06-26T07:07:14.8161958Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8162531Z "size": 138, 2024-06-26T07:07:14.8163095Z "digest": "sha256:b599c5a8c7cb461cbd7b3c98b99960b8eb04506e05e79db1cc0cfadb14a47e91" 2024-06-26T07:07:14.8163729Z }, 2024-06-26T07:07:14.8163983Z { 2024-06-26T07:07:14.8164418Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8164990Z "size": 120, 2024-06-26T07:07:14.8165561Z "digest": "sha256:e08e88c1366827455ee7ddbf92b6ae615ceb3cbd3fb11cd03ddd31bd3fa94a9c" 2024-06-26T07:07:14.8166185Z }, 2024-06-26T07:07:14.8166448Z { 2024-06-26T07:07:14.8166898Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8167467Z "size": 1904157495, 2024-06-26T07:07:14.8168041Z "digest": "sha256:12ba7431ff38de8cf6af3052a2a52e79f4e712b8c99f5bafde3c20e63b1f570f" 2024-06-26T07:07:14.8168662Z }, 2024-06-26T07:07:14.8168912Z { 2024-06-26T07:07:14.8169339Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8169891Z "size": 172, 2024-06-26T07:07:14.8170455Z "digest": "sha256:cf7c2cb896cba6a594028cc0fd246e127a2f4481d48905a3aa12c5f5872332d4" 2024-06-26T07:07:14.8171084Z }, 2024-06-26T07:07:14.8171378Z { 2024-06-26T07:07:14.8171834Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8172405Z "size": 908, 2024-06-26T07:07:14.8172971Z "digest": "sha256:3a778ddfac2af5b3a11f49ddf2f3f1ad1731f74ba691f434ab69966d1e2a189f" 2024-06-26T07:07:14.8173685Z }, 2024-06-26T07:07:14.8173948Z { 2024-06-26T07:07:14.8174392Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8174952Z "size": 702, 2024-06-26T07:07:14.8175505Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:07:14.8176138Z }, 2024-06-26T07:07:14.8176396Z { 2024-06-26T07:07:14.8176851Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8177473Z "size": 134, 2024-06-26T07:07:14.8178010Z "digest": "sha256:236c8f0245f4b40616a4322eaf1e1b3e178d5990db1fcaba39622ea4e79c06a1" 2024-06-26T07:07:14.8178623Z }, 2024-06-26T07:07:14.8178867Z { 2024-06-26T07:07:14.8179303Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8179859Z "size": 32, 2024-06-26T07:07:14.8180412Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:14.8181042Z }, 2024-06-26T07:07:14.8181306Z { 2024-06-26T07:07:14.8181748Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8182311Z "size": 156, 2024-06-26T07:07:14.8182867Z "digest": "sha256:dda1ce302b9a9df0c8f1145d2e3eaa03c96909bb45a792451f1fcb555b06738d" 2024-06-26T07:07:14.8183511Z }, 2024-06-26T07:07:14.8183768Z { 2024-06-26T07:07:14.8184200Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8184763Z "size": 1841, 2024-06-26T07:07:14.8185413Z "digest": "sha256:e797fe86caef19319e03f8dca92cf96838ab3bb3644f4447c6dfc04230118afa" 2024-06-26T07:07:14.8186036Z }, 2024-06-26T07:07:14.8186280Z { 2024-06-26T07:07:14.8186725Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8187280Z "size": 7529784, 2024-06-26T07:07:14.8187851Z "digest": "sha256:967190076fc0ab2bb2fc0d74e58edd3d662a537d702ff3f8b6e876230154a16f" 2024-06-26T07:07:14.8188460Z }, 2024-06-26T07:07:14.8188729Z { 2024-06-26T07:07:14.8189178Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8189742Z "size": 165, 2024-06-26T07:07:14.8190285Z "digest": "sha256:4fb0a83f06855534ef82b80dcf7c685303edb2bcbd446196066dc2ad2dd71281" 2024-06-26T07:07:14.8190908Z }, 2024-06-26T07:07:14.8191170Z { 2024-06-26T07:07:14.8191604Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8192170Z "size": 7944, 2024-06-26T07:07:14.8192731Z "digest": "sha256:68af2ec4e30a33f26947ece9f3a31b7d3b04c2bca5865d4b48618457832a1da2" 2024-06-26T07:07:14.8193365Z }, 2024-06-26T07:07:14.8193611Z { 2024-06-26T07:07:14.8194054Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8194625Z "size": 8070, 2024-06-26T07:07:14.8195174Z "digest": "sha256:cc91933e985a044c6df53caa15488d2285896f5e991f38ec068c253ceec53f14" 2024-06-26T07:07:14.8195775Z }, 2024-06-26T07:07:14.8196029Z { 2024-06-26T07:07:14.8196477Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8197044Z "size": 303, 2024-06-26T07:07:14.8197587Z "digest": "sha256:9f3187916e7dcad33cc572f8f0e949d2f507b887f99cfd6acefd14826b44960d" 2024-06-26T07:07:14.8198387Z }, 2024-06-26T07:07:14.8198653Z { 2024-06-26T07:07:14.8199085Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8199642Z "size": 7629071, 2024-06-26T07:07:14.8200193Z "digest": "sha256:52608399485738d10d779254946c9e1d536010dd45593c69bf1eb9f62f85dc24" 2024-06-26T07:07:14.8200809Z }, 2024-06-26T07:07:14.8201062Z { 2024-06-26T07:07:14.8201554Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8202114Z "size": 108, 2024-06-26T07:07:14.8202673Z "digest": "sha256:05a6f6dcc1c00bac6660923a22adc7d7c34416718f8c89daf9b3351d9365f694" 2024-06-26T07:07:14.8203281Z }, 2024-06-26T07:07:14.8203543Z { 2024-06-26T07:07:14.8203993Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8204644Z "size": 54145775, 2024-06-26T07:07:14.8205219Z "digest": "sha256:6bc9dc8315b09b27c976bfba9700767ba412d72fd0528b18bbe8f8e62005ec22" 2024-06-26T07:07:14.8205857Z }, 2024-06-26T07:07:14.8206115Z { 2024-06-26T07:07:14.8206552Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8207107Z "size": 474, 2024-06-26T07:07:14.8207656Z "digest": "sha256:1b0e27c35f07b3fbf0aeb548d1949bf1d60a420136da21789d94d196c891d981" 2024-06-26T07:07:14.8208290Z }, 2024-06-26T07:07:14.8208537Z { 2024-06-26T07:07:14.8208979Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8209550Z "size": 1374858747, 2024-06-26T07:07:14.8210118Z "digest": "sha256:95237e7311a1c5ce6937db7196c866632f44e8788514a5c0eb4a3f04da4017e6" 2024-06-26T07:07:14.8210707Z }, 2024-06-26T07:07:14.8210959Z { 2024-06-26T07:07:14.8211446Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8211980Z "size": 106, 2024-06-26T07:07:14.8212537Z "digest": "sha256:789eff6c51aada70bf75eb1acafed63c0d839b768638001500616ff6e53b854a" 2024-06-26T07:07:14.8213177Z }, 2024-06-26T07:07:14.8213441Z { 2024-06-26T07:07:14.8213865Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8214425Z "size": 558, 2024-06-26T07:07:14.8214984Z "digest": "sha256:de0f8a8c45c6077b3c5ba3b84d7f07de98353fd618dad9a1e2f368ffe00493c7" 2024-06-26T07:07:14.8215615Z }, 2024-06-26T07:07:14.8215940Z { 2024-06-26T07:07:14.8216374Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8216934Z "size": 46248513, 2024-06-26T07:07:14.8217527Z "digest": "sha256:2060028f74888d218bebdd10d723f66c33ee1c708935c4ffd4654d34912ac4dc" 2024-06-26T07:07:14.8218154Z }, 2024-06-26T07:07:14.8218413Z { 2024-06-26T07:07:14.8218856Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8219392Z "size": 111, 2024-06-26T07:07:14.8219936Z "digest": "sha256:e412c7f3b2ded440992955bd92512345cbece8a8894cd3592f714ebc758ad1a0" 2024-06-26T07:07:14.8220547Z }, 2024-06-26T07:07:14.8220793Z { 2024-06-26T07:07:14.8221223Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8221771Z "size": 32, 2024-06-26T07:07:14.8222310Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:14.8222924Z }, 2024-06-26T07:07:14.8223159Z { 2024-06-26T07:07:14.8223591Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8224154Z "size": 32, 2024-06-26T07:07:14.8224685Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:14.8225302Z }, 2024-06-26T07:07:14.8225549Z { 2024-06-26T07:07:14.8225978Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8226518Z "size": 32, 2024-06-26T07:07:14.8227071Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:14.8227687Z }, 2024-06-26T07:07:14.8227933Z { 2024-06-26T07:07:14.8228357Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:14.8228899Z "size": 32, 2024-06-26T07:07:14.8229438Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:14.8230043Z } 2024-06-26T07:07:14.8230302Z ] 2024-06-26T07:07:14.8230560Z } 2024-06-26T07:07:14.8375774Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:07:14.8376353Z tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:07:14.8376932Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-06-26T07:07:14.8385079Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:14.8385597Z env: 2024-06-26T07:07:14.8385881Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:14.8387055Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.8388233Z ##[endgroup] 2024-06-26T07:07:14.8414838Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.8464912Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-06-26T07:07:14.8465500Z with: 2024-06-26T07:07:14.8466587Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.8467905Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:14.8468431Z env: 2024-06-26T07:07:14.8468729Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:14.8469102Z ##[endgroup] 2024-06-26T07:07:14.8485038Z ##[group]Run set -x 2024-06-26T07:07:14.8485392Z set -x 2024-06-26T07:07:14.8485715Z set +e 2024-06-26T07:07:14.8486034Z  2024-06-26T07:07:14.8486322Z login() { 2024-06-26T07:07:14.8487002Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:07:14.8487726Z } 2024-06-26T07:07:14.8488017Z  2024-06-26T07:07:14.8488333Z retry () { 2024-06-26T07:07:14.8488747Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:07:14.8489201Z } 2024-06-26T07:07:14.8489487Z  2024-06-26T07:07:14.8489816Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:07:14.8490352Z  2024-06-26T07:07:14.8490654Z set -e 2024-06-26T07:07:14.8491229Z # ignore output since only exit code is used for conditional 2024-06-26T07:07:14.8491972Z # only pull docker image if it's not available locally 2024-06-26T07:07:14.8492770Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-06-26T07:07:14.8493496Z  retry docker pull "${DOCKER_IMAGE}" 2024-06-26T07:07:14.8493953Z fi 2024-06-26T07:07:14.8501838Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:14.8502359Z env: 2024-06-26T07:07:14.8502656Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:14.8503850Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:14.8505194Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:14.8505757Z ##[endgroup] 2024-06-26T07:07:14.8526586Z + set +e 2024-06-26T07:07:14.8527409Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:14.8528042Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:14.8530098Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:07:14.8531538Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:15.2615987Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:07:15.2616934Z Configure a credential helper to remove this warning. See 2024-06-26T07:07:15.2617825Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:07:15.2618363Z 2024-06-26T07:07:15.2618495Z Login Succeeded 2024-06-26T07:07:15.2628058Z + set -e 2024-06-26T07:07:15.2629464Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:15.2749029Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:15.2751179Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:15.4623723Z 91382da70d5719cd7007b6b80b71d2f48398f6b7: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-06-26T07:07:15.4625164Z 7a2c55901189: Pulling fs layer 2024-06-26T07:07:15.4625822Z 224fe954d725: Pulling fs layer 2024-06-26T07:07:15.4626675Z 75722010b82e: Pulling fs layer 2024-06-26T07:07:15.4627345Z d527cbbb87e3: Pulling fs layer 2024-06-26T07:07:15.4628012Z b57676e46aee: Pulling fs layer 2024-06-26T07:07:15.4628696Z a8c1e85b5e14: Pulling fs layer 2024-06-26T07:07:15.4629394Z a41a8d1c11c8: Pulling fs layer 2024-06-26T07:07:15.4630013Z 0c1227890755: Pulling fs layer 2024-06-26T07:07:15.4630638Z d8d1234baab3: Pulling fs layer 2024-06-26T07:07:15.4631301Z 7ed32bc8e469: Pulling fs layer 2024-06-26T07:07:15.4631893Z ec1e7978c1fe: Pulling fs layer 2024-06-26T07:07:15.4632359Z 309b872da805: Pulling fs layer 2024-06-26T07:07:15.4632815Z e82932cb002c: Pulling fs layer 2024-06-26T07:07:15.4633283Z ca9dd73e3e93: Pulling fs layer 2024-06-26T07:07:15.4633742Z d527cbbb87e3: Waiting 2024-06-26T07:07:15.4634162Z 16d76a79d5d6: Pulling fs layer 2024-06-26T07:07:15.4634609Z b57676e46aee: Waiting 2024-06-26T07:07:15.4635039Z 4bdf41c9ecbe: Pulling fs layer 2024-06-26T07:07:15.4635501Z a8c1e85b5e14: Waiting 2024-06-26T07:07:15.4635926Z f4863801e30c: Pulling fs layer 2024-06-26T07:07:15.4636398Z d37dd9ed8948: Pulling fs layer 2024-06-26T07:07:15.4636840Z a41a8d1c11c8: Waiting 2024-06-26T07:07:15.4637308Z cf2f34f225db: Pulling fs layer 2024-06-26T07:07:15.4637762Z df546d7b3e34: Pulling fs layer 2024-06-26T07:07:15.4638646Z 0c1227890755: Waiting 2024-06-26T07:07:15.4639111Z 3d914950f8ad: Pulling fs layer 2024-06-26T07:07:15.4639578Z 96d0e2032184: Pulling fs layer 2024-06-26T07:07:15.4640041Z 06e12ce59790: Pulling fs layer 2024-06-26T07:07:15.4640495Z d8d1234baab3: Waiting 2024-06-26T07:07:15.4640920Z ed712f63cde7: Pulling fs layer 2024-06-26T07:07:15.4641368Z 7ed32bc8e469: Waiting 2024-06-26T07:07:15.4641814Z ca9dd73e3e93: Waiting 2024-06-26T07:07:15.4642257Z ec1e7978c1fe: Waiting 2024-06-26T07:07:15.4642660Z f4863801e30c: Waiting 2024-06-26T07:07:15.4643074Z 16d76a79d5d6: Waiting 2024-06-26T07:07:15.4643480Z cf2f34f225db: Waiting 2024-06-26T07:07:15.4643923Z df546d7b3e34: Waiting 2024-06-26T07:07:15.4644327Z e82932cb002c: Waiting 2024-06-26T07:07:15.4644785Z 4bdf41c9ecbe: Waiting 2024-06-26T07:07:15.4645223Z 96d0e2032184: Waiting 2024-06-26T07:07:15.4645618Z 06e12ce59790: Waiting 2024-06-26T07:07:15.4646031Z a97a71ef8a4e: Pulling fs layer 2024-06-26T07:07:15.4646496Z 25cd0c482ad6: Pulling fs layer 2024-06-26T07:07:15.4646953Z 3d914950f8ad: Waiting 2024-06-26T07:07:15.4647359Z 9bf94bc45658: Pulling fs layer 2024-06-26T07:07:15.4647819Z de45a14c9b22: Pulling fs layer 2024-06-26T07:07:15.4648278Z e99241e77209: Pulling fs layer 2024-06-26T07:07:15.4648719Z 25cd0c482ad6: Waiting 2024-06-26T07:07:15.4649114Z 9bf94bc45658: Waiting 2024-06-26T07:07:15.4649541Z 35e8f4e2a42f: Pulling fs layer 2024-06-26T07:07:15.4650042Z a97a71ef8a4e: Waiting 2024-06-26T07:07:15.4650447Z e99241e77209: Waiting 2024-06-26T07:07:15.4650843Z de45a14c9b22: Waiting 2024-06-26T07:07:15.4651251Z a823ffa4c541: Pulling fs layer 2024-06-26T07:07:15.4651703Z d37dd9ed8948: Waiting 2024-06-26T07:07:15.4652169Z 44ba272eae20: Pulling fs layer 2024-06-26T07:07:15.4652673Z da02d278e524: Pulling fs layer 2024-06-26T07:07:15.4653130Z a823ffa4c541: Waiting 2024-06-26T07:07:15.4653583Z 73a0ce681c29: Pulling fs layer 2024-06-26T07:07:15.4654037Z da02d278e524: Waiting 2024-06-26T07:07:15.4654446Z 49737bd5ae70: Pulling fs layer 2024-06-26T07:07:15.4654890Z 73a0ce681c29: Waiting 2024-06-26T07:07:15.4655313Z 9cd4c745f8e4: Pulling fs layer 2024-06-26T07:07:15.4655764Z b2950a036534: Pulling fs layer 2024-06-26T07:07:15.4656225Z 25e19d81d9fc: Pulling fs layer 2024-06-26T07:07:15.4656690Z 60156b40fcbe: Pulling fs layer 2024-06-26T07:07:15.4657146Z 7a868239227f: Pulling fs layer 2024-06-26T07:07:15.4657659Z 49737bd5ae70: Waiting 2024-06-26T07:07:15.4658082Z 83911d7c7ecf: Pulling fs layer 2024-06-26T07:07:15.4658535Z 25e19d81d9fc: Waiting 2024-06-26T07:07:15.4658958Z b3c5d66c679f: Pulling fs layer 2024-06-26T07:07:15.4659400Z 9cd4c745f8e4: Waiting 2024-06-26T07:07:15.4659825Z 84b2c7a89982: Pulling fs layer 2024-06-26T07:07:15.4660290Z c1068d03a811: Pulling fs layer 2024-06-26T07:07:15.4660730Z 60156b40fcbe: Waiting 2024-06-26T07:07:15.4661288Z 671b4a301d81: Pulling fs layer 2024-06-26T07:07:15.4661757Z 67760cf5ec46: Pulling fs layer 2024-06-26T07:07:15.4662229Z ddc9740bcea2: Pulling fs layer 2024-06-26T07:07:15.4662675Z 83911d7c7ecf: Waiting 2024-06-26T07:07:15.4663097Z 7a868239227f: Waiting 2024-06-26T07:07:15.4663514Z fb2e97768560: Pulling fs layer 2024-06-26T07:07:15.4663971Z 84b2c7a89982: Waiting 2024-06-26T07:07:15.4664386Z b599c5a8c7cb: Pulling fs layer 2024-06-26T07:07:15.4664847Z fb2e97768560: Waiting 2024-06-26T07:07:15.4665266Z e08e88c13668: Pulling fs layer 2024-06-26T07:07:15.4665710Z c1068d03a811: Waiting 2024-06-26T07:07:15.4666135Z 12ba7431ff38: Pulling fs layer 2024-06-26T07:07:15.4666609Z cf7c2cb896cb: Pulling fs layer 2024-06-26T07:07:15.4667079Z 3a778ddfac2a: Pulling fs layer 2024-06-26T07:07:15.4667536Z 236c8f0245f4: Pulling fs layer 2024-06-26T07:07:15.4667987Z 3a778ddfac2a: Waiting 2024-06-26T07:07:15.4668423Z 4f4fb700ef54: Pulling fs layer 2024-06-26T07:07:15.4668877Z 236c8f0245f4: Waiting 2024-06-26T07:07:15.4669301Z dda1ce302b9a: Pulling fs layer 2024-06-26T07:07:15.4669774Z e797fe86caef: Pulling fs layer 2024-06-26T07:07:15.4670238Z 967190076fc0: Pulling fs layer 2024-06-26T07:07:15.4670691Z 4fb0a83f0685: Pulling fs layer 2024-06-26T07:07:15.4671249Z dda1ce302b9a: Waiting 2024-06-26T07:07:15.4671700Z 68af2ec4e30a: Pulling fs layer 2024-06-26T07:07:15.4672175Z cc91933e985a: Pulling fs layer 2024-06-26T07:07:15.4672630Z 9f3187916e7d: Pulling fs layer 2024-06-26T07:07:15.4673145Z 526083994857: Pulling fs layer 2024-06-26T07:07:15.4673586Z 68af2ec4e30a: Waiting 2024-06-26T07:07:15.4673991Z 967190076fc0: Waiting 2024-06-26T07:07:15.4674395Z 05a6f6dcc1c0: Pulling fs layer 2024-06-26T07:07:15.4674840Z e797fe86caef: Waiting 2024-06-26T07:07:15.4675240Z cc91933e985a: Waiting 2024-06-26T07:07:15.4675635Z 4f4fb700ef54: Waiting 2024-06-26T07:07:15.4676034Z 9f3187916e7d: Waiting 2024-06-26T07:07:15.4676492Z 6bc9dc8315b0: Pulling fs layer 2024-06-26T07:07:15.4676958Z 1b0e27c35f07: Pulling fs layer 2024-06-26T07:07:15.4677491Z 95237e7311a1: Pulling fs layer 2024-06-26T07:07:15.4678115Z 789eff6c51aa: Pulling fs layer 2024-06-26T07:07:15.4678568Z 4fb0a83f0685: Waiting 2024-06-26T07:07:15.4678984Z de0f8a8c45c6: Pulling fs layer 2024-06-26T07:07:15.4679448Z 2060028f7488: Pulling fs layer 2024-06-26T07:07:15.4679896Z 789eff6c51aa: Waiting 2024-06-26T07:07:15.4680309Z e412c7f3b2de: Pulling fs layer 2024-06-26T07:07:15.4680745Z 95237e7311a1: Waiting 2024-06-26T07:07:15.4681149Z 6bc9dc8315b0: Waiting 2024-06-26T07:07:15.4681554Z de0f8a8c45c6: Waiting 2024-06-26T07:07:15.4681957Z e412c7f3b2de: Waiting 2024-06-26T07:07:15.4682350Z 1b0e27c35f07: Waiting 2024-06-26T07:07:15.5987479Z 224fe954d725: Verifying Checksum 2024-06-26T07:07:15.5988154Z 224fe954d725: Download complete 2024-06-26T07:07:15.6698918Z d527cbbb87e3: Verifying Checksum 2024-06-26T07:07:15.6699497Z d527cbbb87e3: Download complete 2024-06-26T07:07:15.7509589Z b57676e46aee: Verifying Checksum 2024-06-26T07:07:15.7510183Z b57676e46aee: Download complete 2024-06-26T07:07:15.8333251Z 7a2c55901189: Verifying Checksum 2024-06-26T07:07:15.8333934Z 7a2c55901189: Download complete 2024-06-26T07:07:15.8982064Z a41a8d1c11c8: Verifying Checksum 2024-06-26T07:07:15.8982672Z a41a8d1c11c8: Download complete 2024-06-26T07:07:15.9704566Z 0c1227890755: Verifying Checksum 2024-06-26T07:07:15.9705248Z 0c1227890755: Download complete 2024-06-26T07:07:16.0423180Z d8d1234baab3: Verifying Checksum 2024-06-26T07:07:16.0423986Z d8d1234baab3: Download complete 2024-06-26T07:07:16.0749893Z 75722010b82e: Verifying Checksum 2024-06-26T07:07:16.0750711Z 75722010b82e: Download complete 2024-06-26T07:07:16.1489489Z ec1e7978c1fe: Verifying Checksum 2024-06-26T07:07:16.1490610Z ec1e7978c1fe: Download complete 2024-06-26T07:07:16.2211083Z 309b872da805: Download complete 2024-06-26T07:07:16.8747049Z 7a2c55901189: Pull complete 2024-06-26T07:07:17.1470793Z 224fe954d725: Pull complete 2024-06-26T07:07:18.0788012Z 75722010b82e: Pull complete 2024-06-26T07:07:18.1280833Z d527cbbb87e3: Pull complete 2024-06-26T07:07:18.1854018Z b57676e46aee: Pull complete 2024-06-26T07:07:18.7445092Z e82932cb002c: Verifying Checksum 2024-06-26T07:07:18.7445749Z e82932cb002c: Download complete 2024-06-26T07:07:18.8155665Z ca9dd73e3e93: Verifying Checksum 2024-06-26T07:07:18.8156207Z ca9dd73e3e93: Download complete 2024-06-26T07:07:18.8953952Z 16d76a79d5d6: Verifying Checksum 2024-06-26T07:07:18.8954486Z 16d76a79d5d6: Download complete 2024-06-26T07:07:18.9562365Z 4bdf41c9ecbe: Verifying Checksum 2024-06-26T07:07:18.9562881Z 4bdf41c9ecbe: Download complete 2024-06-26T07:07:19.9284450Z f4863801e30c: Verifying Checksum 2024-06-26T07:07:19.9285022Z f4863801e30c: Download complete 2024-06-26T07:07:20.0283875Z d37dd9ed8948: Verifying Checksum 2024-06-26T07:07:20.0284447Z d37dd9ed8948: Download complete 2024-06-26T07:07:20.1111608Z cf2f34f225db: Verifying Checksum 2024-06-26T07:07:20.1112275Z cf2f34f225db: Download complete 2024-06-26T07:07:20.1716415Z df546d7b3e34: Download complete 2024-06-26T07:07:29.0902904Z a8c1e85b5e14: Verifying Checksum 2024-06-26T07:07:29.0903402Z a8c1e85b5e14: Download complete 2024-06-26T07:07:29.1864171Z 96d0e2032184: Verifying Checksum 2024-06-26T07:07:29.1864655Z 96d0e2032184: Download complete 2024-06-26T07:07:29.2685743Z 06e12ce59790: Download complete 2024-06-26T07:07:29.3381080Z ed712f63cde7: Verifying Checksum 2024-06-26T07:07:29.3381573Z ed712f63cde7: Download complete 2024-06-26T07:07:29.4103861Z a97a71ef8a4e: Verifying Checksum 2024-06-26T07:07:29.4104398Z a97a71ef8a4e: Download complete 2024-06-26T07:07:29.4988051Z 25cd0c482ad6: Verifying Checksum 2024-06-26T07:07:30.7677930Z 9bf94bc45658: Verifying Checksum 2024-06-26T07:07:30.7678757Z 9bf94bc45658: Download complete 2024-06-26T07:07:30.8398121Z de45a14c9b22: Verifying Checksum 2024-06-26T07:07:30.8398671Z de45a14c9b22: Download complete 2024-06-26T07:07:30.9140956Z e99241e77209: Download complete 2024-06-26T07:07:30.9855141Z 35e8f4e2a42f: Download complete 2024-06-26T07:07:31.0902439Z a823ffa4c541: Verifying Checksum 2024-06-26T07:07:31.2095979Z a823ffa4c541: Download complete 2024-06-26T07:07:31.2096610Z 44ba272eae20: Verifying Checksum 2024-06-26T07:07:31.2097103Z 44ba272eae20: Download complete 2024-06-26T07:07:35.7328573Z da02d278e524: Verifying Checksum 2024-06-26T07:07:35.7329131Z da02d278e524: Download complete 2024-06-26T07:07:35.8138378Z 73a0ce681c29: Download complete 2024-06-26T07:07:35.8885052Z 49737bd5ae70: Verifying Checksum 2024-06-26T07:07:35.8885762Z 49737bd5ae70: Download complete 2024-06-26T07:07:36.3099151Z 9cd4c745f8e4: Verifying Checksum 2024-06-26T07:07:36.3099756Z 9cd4c745f8e4: Download complete 2024-06-26T07:07:36.3872206Z b2950a036534: Verifying Checksum 2024-06-26T07:07:36.3872877Z b2950a036534: Download complete 2024-06-26T07:07:36.4564180Z 25e19d81d9fc: Verifying Checksum 2024-06-26T07:07:36.4564759Z 25e19d81d9fc: Download complete 2024-06-26T07:07:36.7089915Z 60156b40fcbe: Verifying Checksum 2024-06-26T07:07:36.7090530Z 60156b40fcbe: Download complete 2024-06-26T07:07:36.7805423Z 7a868239227f: Verifying Checksum 2024-06-26T07:07:36.7806092Z 7a868239227f: Download complete 2024-06-26T07:07:36.8688114Z 83911d7c7ecf: Download complete 2024-06-26T07:07:36.9583515Z b3c5d66c679f: Verifying Checksum 2024-06-26T07:07:36.9584218Z b3c5d66c679f: Download complete 2024-06-26T07:07:41.4447227Z 7ed32bc8e469: Verifying Checksum 2024-06-26T07:07:41.4447806Z 7ed32bc8e469: Download complete 2024-06-26T07:07:41.5212859Z c1068d03a811: Download complete 2024-06-26T07:07:41.6201376Z 671b4a301d81: Download complete 2024-06-26T07:07:41.8875012Z a8c1e85b5e14: Pull complete 2024-06-26T07:07:42.0286603Z a41a8d1c11c8: Pull complete 2024-06-26T07:07:42.1197327Z 67760cf5ec46: Download complete 2024-06-26T07:07:42.1909224Z ddc9740bcea2: Verifying Checksum 2024-06-26T07:07:42.1909878Z ddc9740bcea2: Download complete 2024-06-26T07:07:42.2228272Z 0c1227890755: Pull complete 2024-06-26T07:07:42.2688486Z fb2e97768560: Download complete 2024-06-26T07:07:42.3802400Z b599c5a8c7cb: Download complete 2024-06-26T07:07:42.4599962Z d8d1234baab3: Pull complete 2024-06-26T07:07:42.4704709Z e08e88c13668: Download complete 2024-06-26T07:07:48.0694480Z 3d914950f8ad: Verifying Checksum 2024-06-26T07:07:48.0695229Z 3d914950f8ad: Download complete 2024-06-26T07:07:48.1575216Z cf7c2cb896cb: Download complete 2024-06-26T07:07:48.2321421Z 3a778ddfac2a: Verifying Checksum 2024-06-26T07:07:48.2321947Z 3a778ddfac2a: Download complete 2024-06-26T07:07:48.3093915Z 236c8f0245f4: Verifying Checksum 2024-06-26T07:07:48.3094682Z 236c8f0245f4: Download complete 2024-06-26T07:07:48.3174044Z 4f4fb700ef54: Download complete 2024-06-26T07:07:48.3894141Z dda1ce302b9a: Verifying Checksum 2024-06-26T07:07:48.3894745Z dda1ce302b9a: Download complete 2024-06-26T07:07:48.4677380Z e797fe86caef: Verifying Checksum 2024-06-26T07:07:48.4678484Z e797fe86caef: Download complete 2024-06-26T07:07:48.6016397Z 967190076fc0: Verifying Checksum 2024-06-26T07:07:48.6017098Z 967190076fc0: Download complete 2024-06-26T07:07:48.6705978Z 4fb0a83f0685: Verifying Checksum 2024-06-26T07:07:48.6706526Z 4fb0a83f0685: Download complete 2024-06-26T07:07:48.7475188Z 68af2ec4e30a: Verifying Checksum 2024-06-26T07:07:48.7475888Z 68af2ec4e30a: Download complete 2024-06-26T07:07:48.9373238Z 9f3187916e7d: Download complete 2024-06-26T07:07:49.0627309Z 526083994857: Verifying Checksum 2024-06-26T07:07:49.0628210Z 526083994857: Download complete 2024-06-26T07:07:49.1387327Z 05a6f6dcc1c0: Verifying Checksum 2024-06-26T07:07:49.1387999Z 05a6f6dcc1c0: Download complete 2024-06-26T07:07:49.7556862Z 6bc9dc8315b0: Verifying Checksum 2024-06-26T07:07:49.7557639Z 6bc9dc8315b0: Download complete 2024-06-26T07:07:49.8348899Z 1b0e27c35f07: Download complete 2024-06-26T07:08:12.7964687Z 12ba7431ff38: Verifying Checksum 2024-06-26T07:08:12.7965233Z 12ba7431ff38: Download complete 2024-06-26T07:08:12.8719644Z 789eff6c51aa: Verifying Checksum 2024-06-26T07:08:12.8720160Z 789eff6c51aa: Download complete 2024-06-26T07:08:12.9550323Z de0f8a8c45c6: Verifying Checksum 2024-06-26T07:08:12.9550924Z de0f8a8c45c6: Download complete 2024-06-26T07:08:14.0329528Z 2060028f7488: Verifying Checksum 2024-06-26T07:08:14.0330072Z 2060028f7488: Download complete 2024-06-26T07:08:14.1476441Z e412c7f3b2de: Download complete 2024-06-26T07:08:17.1792252Z 95237e7311a1: Verifying Checksum 2024-06-26T07:08:17.1792721Z 95237e7311a1: Download complete 2024-06-26T07:08:40.5202472Z 84b2c7a89982: Verifying Checksum 2024-06-26T07:08:40.5202944Z 84b2c7a89982: Download complete 2024-06-26T07:08:57.8426397Z 7ed32bc8e469: Pull complete 2024-06-26T07:08:58.0544966Z ec1e7978c1fe: Pull complete 2024-06-26T07:08:58.2786350Z 309b872da805: Pull complete 2024-06-26T07:09:05.6411435Z e82932cb002c: Pull complete 2024-06-26T07:09:05.8651999Z ca9dd73e3e93: Pull complete 2024-06-26T07:09:06.0814801Z 16d76a79d5d6: Pull complete 2024-06-26T07:09:06.2991582Z 4bdf41c9ecbe: Pull complete 2024-06-26T07:09:08.6519642Z f4863801e30c: Pull complete 2024-06-26T07:09:08.8755086Z d37dd9ed8948: Pull complete 2024-06-26T07:09:09.1005694Z cf2f34f225db: Pull complete 2024-06-26T07:09:09.3266629Z df546d7b3e34: Pull complete 2024-06-26T07:09:58.5132451Z 3d914950f8ad: Pull complete 2024-06-26T07:09:58.7357143Z 96d0e2032184: Pull complete 2024-06-26T07:09:58.9806367Z 06e12ce59790: Pull complete 2024-06-26T07:09:59.1855339Z ed712f63cde7: Pull complete 2024-06-26T07:09:59.3578544Z a97a71ef8a4e: Pull complete 2024-06-26T07:09:59.4741768Z 25cd0c482ad6: Pull complete 2024-06-26T07:10:02.9254365Z 9bf94bc45658: Pull complete 2024-06-26T07:10:03.0479806Z de45a14c9b22: Pull complete 2024-06-26T07:10:03.1908171Z e99241e77209: Pull complete 2024-06-26T07:10:03.4050103Z 35e8f4e2a42f: Pull complete 2024-06-26T07:10:03.6308383Z a823ffa4c541: Pull complete 2024-06-26T07:10:03.8661581Z 44ba272eae20: Pull complete 2024-06-26T07:10:12.0854884Z da02d278e524: Pull complete 2024-06-26T07:10:12.2636046Z 73a0ce681c29: Pull complete 2024-06-26T07:10:12.4321214Z 49737bd5ae70: Pull complete 2024-06-26T07:10:13.3602510Z 9cd4c745f8e4: Pull complete 2024-06-26T07:10:13.5175660Z b2950a036534: Pull complete 2024-06-26T07:10:13.7163146Z 25e19d81d9fc: Pull complete 2024-06-26T07:10:14.1598668Z 60156b40fcbe: Pull complete 2024-06-26T07:10:14.3330131Z 7a868239227f: Pull complete 2024-06-26T07:10:14.6843294Z 83911d7c7ecf: Pull complete 2024-06-26T07:10:14.9084565Z b3c5d66c679f: Pull complete 2024-06-26T07:10:53.6565533Z 84b2c7a89982: Pull complete 2024-06-26T07:10:53.8523310Z c1068d03a811: Pull complete 2024-06-26T07:10:54.0659898Z 671b4a301d81: Pull complete 2024-06-26T07:10:56.0832205Z 67760cf5ec46: Pull complete 2024-06-26T07:10:56.1667891Z ddc9740bcea2: Pull complete 2024-06-26T07:10:56.2079062Z fb2e97768560: Pull complete 2024-06-26T07:10:56.3146516Z b599c5a8c7cb: Pull complete 2024-06-26T07:10:56.3538839Z e08e88c13668: Pull complete 2024-06-26T07:11:30.5258126Z 12ba7431ff38: Pull complete 2024-06-26T07:11:30.7190055Z cf7c2cb896cb: Pull complete 2024-06-26T07:11:30.8666193Z 3a778ddfac2a: Pull complete 2024-06-26T07:11:31.3015879Z 236c8f0245f4: Pull complete 2024-06-26T07:11:31.5206714Z 4f4fb700ef54: Pull complete 2024-06-26T07:11:31.7455790Z dda1ce302b9a: Pull complete 2024-06-26T07:11:31.9619639Z e797fe86caef: Pull complete 2024-06-26T07:11:32.3264282Z 967190076fc0: Pull complete 2024-06-26T07:11:32.5427111Z 4fb0a83f0685: Pull complete 2024-06-26T07:11:32.7613387Z 68af2ec4e30a: Pull complete 2024-06-26T07:11:32.9796070Z cc91933e985a: Pull complete 2024-06-26T07:11:33.2003559Z 9f3187916e7d: Pull complete 2024-06-26T07:11:34.2895487Z 526083994857: Pull complete 2024-06-26T07:11:34.4869234Z 05a6f6dcc1c0: Pull complete 2024-06-26T07:11:36.5082274Z 6bc9dc8315b0: Pull complete 2024-06-26T07:11:36.7359737Z 1b0e27c35f07: Pull complete 2024-06-26T07:11:50.1643204Z 95237e7311a1: Pull complete 2024-06-26T07:11:50.2347868Z 789eff6c51aa: Pull complete 2024-06-26T07:11:50.2711173Z de0f8a8c45c6: Pull complete 2024-06-26T07:11:50.9828257Z 2060028f7488: Pull complete 2024-06-26T07:11:51.0286000Z e412c7f3b2de: Pull complete 2024-06-26T07:11:51.3052307Z Digest: sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb 2024-06-26T07:11:51.3098774Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:11:51.3130296Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:11:51.3181433Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:11:51.3182386Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:11:51.3190562Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:51.3191059Z env: 2024-06-26T07:11:51.3191338Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:51.3191698Z ##[endgroup] 2024-06-26T07:11:51.3344801Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-06-26T07:11:51.3345364Z with: 2024-06-26T07:11:51.3345668Z driver-version: 550.54.15 2024-06-26T07:11:51.3346031Z env: 2024-06-26T07:11:51.3346310Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:51.3346675Z ##[endgroup] 2024-06-26T07:11:51.3386782Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:11:51.3387371Z with: 2024-06-26T07:11:51.3387682Z timeout_minutes: 10 2024-06-26T07:11:51.3388018Z max_attempts: 3 2024-06-26T07:11:51.3419160Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" 2024-06-26T07:11:51.3450800Z retry_wait_seconds: 10 2024-06-26T07:11:51.3451199Z polling_interval_seconds: 1 2024-06-26T07:11:51.3451606Z warning_on_retry: true 2024-06-26T07:11:51.3451999Z continue_on_error: false 2024-06-26T07:11:51.3452380Z env: 2024-06-26T07:11:51.3452679Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:51.3453080Z DRIVER_VERSION: 550.54.15 2024-06-26T07:11:51.3453465Z ##[endgroup] 2024-06-26T07:11:51.3982774Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-06-26T07:11:51.3983670Z + pre_install_nvidia_driver_amzn2 2024-06-26T07:11:51.3984464Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-06-26T07:11:51.7025739Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:11:51.7412224Z No Match for argument: nvidia-driver-latest-dkms 2024-06-26T07:11:51.7648198Z No Packages marked for removal 2024-06-26T07:11:51.7769930Z + install_nvidia_driver_common 2024-06-26T07:11:51.7772294Z + echo 'Before installing NVIDIA driver' 2024-06-26T07:11:51.7772876Z + lspci 2024-06-26T07:11:51.7773272Z Before installing NVIDIA driver 2024-06-26T07:11:51.7851186Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-06-26T07:11:51.7852162Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T07:11:51.7853288Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-06-26T07:11:51.7854352Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-06-26T07:11:51.7855360Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 2024-06-26T07:11:51.7856185Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T07:11:51.7857007Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1) 2024-06-26T07:11:51.7857913Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-06-26T07:11:51.7858510Z + lsmod 2024-06-26T07:11:51.7864796Z Module Size Used by 2024-06-26T07:11:51.7865282Z nvidia_modeset 1339392 0 2024-06-26T07:11:51.7865806Z veth 16384 0 2024-06-26T07:11:51.7866259Z nvidia_uvm 4599808 0 2024-06-26T07:11:51.7866754Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:11:51.7867430Z drm 421888 1 nvidia 2024-06-26T07:11:51.7868177Z i2c_core 77824 2 nvidia,drm 2024-06-26T07:11:51.7868806Z backlight 16384 1 nvidia_modeset 2024-06-26T07:11:51.7869393Z xt_conntrack 16384 1 2024-06-26T07:11:51.7869910Z ipt_MASQUERADE 16384 1 2024-06-26T07:11:51.7870373Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T07:11:51.7870945Z nf_conntrack_netlink 49152 0 2024-06-26T07:11:51.7871564Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T07:11:51.7872082Z xfrm_user 45056 1 2024-06-26T07:11:51.7872493Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:11:51.7873043Z iptable_nat 16384 1 2024-06-26T07:11:51.7873569Z nf_conntrack_ipv4 16384 3 2024-06-26T07:11:51.7874161Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T07:11:51.7874807Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T07:11:51.7875432Z nf_nat 32768 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 2024-06-26T07:11:51.7876391Z nf_conntrack 155648 7 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink 2024-06-26T07:11:51.7877236Z xt_addrtype 16384 2 2024-06-26T07:11:51.7877771Z iptable_filter 16384 1 2024-06-26T07:11:51.7878627Z br_netfilter 24576 0 2024-06-26T07:11:51.7879204Z bridge 172032 1 br_netfilter 2024-06-26T07:11:51.7879824Z stp 16384 1 bridge 2024-06-26T07:11:51.7880404Z llc 16384 2 bridge,stp 2024-06-26T07:11:51.7880834Z overlay 86016 0 2024-06-26T07:11:51.7881211Z sunrpc 393216 1 2024-06-26T07:11:51.7881592Z dm_mirror 28672 0 2024-06-26T07:11:51.7882003Z dm_region_hash 20480 1 dm_mirror 2024-06-26T07:11:51.7882497Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T07:11:51.7883015Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T07:11:51.7883471Z dax 69632 1 dm_mod 2024-06-26T07:11:51.7883887Z crc32_pclmul 16384 0 2024-06-26T07:11:51.7884273Z ghash_clmulni_intel 16384 0 2024-06-26T07:11:51.7884661Z pcbc 16384 0 2024-06-26T07:11:51.7885036Z aesni_intel 188416 0 2024-06-26T07:11:51.7885437Z aes_x86_64 20480 1 aesni_intel 2024-06-26T07:11:51.7885943Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:11:51.7886405Z glue_helper 16384 1 aesni_intel 2024-06-26T07:11:51.7887222Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T07:11:51.7887819Z mousedev 24576 0 2024-06-26T07:11:51.7888216Z evdev 20480 3 2024-06-26T07:11:51.7888617Z psmouse 32768 0 2024-06-26T07:11:51.7889017Z button 16384 0 2024-06-26T07:11:51.7889402Z ena 139264 0 2024-06-26T07:11:51.7889803Z ptp 20480 1 ena 2024-06-26T07:11:51.7890226Z pps_core 20480 1 ptp 2024-06-26T07:11:51.7890661Z crc32c_intel 24576 0 2024-06-26T07:11:51.7891045Z autofs4 49152 2 2024-06-26T07:11:51.7891433Z + modinfo nvidia 2024-06-26T07:11:51.7892152Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T07:11:51.7892856Z alias: char-major-195-* 2024-06-26T07:11:51.7893244Z version: 550.54.15 2024-06-26T07:11:51.7893599Z supported: external 2024-06-26T07:11:51.7893954Z license: NVIDIA 2024-06-26T07:11:51.7894343Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:11:51.7894828Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:11:51.7895298Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T07:11:51.7895780Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:11:51.7896288Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:11:51.7896780Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:11:51.7897278Z depends: i2c-core,drm 2024-06-26T07:11:51.7897865Z retpoline: Y 2024-06-26T07:11:51.7898184Z name: nvidia 2024-06-26T07:11:51.7898748Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T07:11:51.7899414Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:11:51.7900073Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:11:51.7900667Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:11:51.7901129Z parm: NVreg_RmLogonRC:int 2024-06-26T07:11:51.7901577Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:11:51.7902043Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:11:51.7902480Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:11:51.7902934Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:11:51.7903474Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:11:51.7904046Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:11:51.7904530Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:11:51.7904978Z parm: NVreg_EnableMSI:int 2024-06-26T07:11:51.7905402Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:11:51.7905870Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:11:51.7906405Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:11:51.7906997Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:11:51.7907566Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:11:51.7908197Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:11:51.7908798Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:11:51.7909424Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:11:51.7910023Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:11:51.7910522Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:11:51.7911063Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:11:51.7911626Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:11:51.7912137Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:11:51.7912604Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:11:51.7913087Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:11:51.7913561Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:11:51.7914016Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:11:51.7914516Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:11:51.7915141Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:11:51.7915635Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:11:51.7916135Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:11:51.7916633Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:11:51.7917127Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:11:51.7917636Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:11:51.7918394Z parm: NVreg_RmMsg:charp 2024-06-26T07:11:51.7918832Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:11:51.7919318Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:11:51.7919807Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:11:51.7920283Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:11:51.7920773Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:11:51.7921264Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:11:51.7921741Z parm: rm_firmware_active:charp 2024-06-26T07:11:51.7922167Z + HAS_NVIDIA_DRIVER=0 2024-06-26T07:11:51.7922560Z ++ command -v nvidia-smi 2024-06-26T07:11:51.7922964Z + '[' -x /usr/bin/nvidia-smi ']' 2024-06-26T07:11:51.7923358Z + set +e 2024-06-26T07:11:51.7923899Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-06-26T07:11:53.8033306Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-06-26T07:11:53.8033941Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:11:53.8034520Z + '[' 0 -ne 0 ']' 2024-06-26T07:11:53.8034879Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-06-26T07:11:53.8035568Z + HAS_NVIDIA_DRIVER=1 2024-06-26T07:11:53.8036295Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-06-26T07:11:53.8036997Z + set -e 2024-06-26T07:11:53.8037290Z + '[' 1 -eq 0 ']' 2024-06-26T07:11:53.8037875Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-06-26T07:11:53.8038746Z + post_install_nvidia_driver_common 2024-06-26T07:11:53.8039158Z + sudo modprobe nvidia 2024-06-26T07:11:53.8120064Z + echo 'After installing NVIDIA driver' 2024-06-26T07:11:53.8120503Z + lspci 2024-06-26T07:11:53.8121174Z After installing NVIDIA driver 2024-06-26T07:11:53.8199033Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-06-26T07:11:53.8199792Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T07:11:53.8200723Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-06-26T07:11:53.8201533Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-06-26T07:11:53.8202294Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 2024-06-26T07:11:53.8203050Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T07:11:53.8203784Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1) 2024-06-26T07:11:53.8204580Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-06-26T07:11:53.8205177Z + lsmod 2024-06-26T07:11:53.8215735Z Module Size Used by 2024-06-26T07:11:53.8216340Z nvidia_modeset 1339392 0 2024-06-26T07:11:53.8216837Z veth 16384 0 2024-06-26T07:11:53.8217426Z nvidia_uvm 4599808 0 2024-06-26T07:11:53.8217881Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:11:53.8218368Z drm 421888 1 nvidia 2024-06-26T07:11:53.8218843Z i2c_core 77824 2 nvidia,drm 2024-06-26T07:11:53.8219477Z backlight 16384 1 nvidia_modeset 2024-06-26T07:11:53.8219967Z xt_conntrack 16384 1 2024-06-26T07:11:53.8220367Z ipt_MASQUERADE 16384 1 2024-06-26T07:11:53.8220822Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T07:11:53.8221305Z nf_conntrack_netlink 49152 0 2024-06-26T07:11:53.8221760Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T07:11:53.8222239Z xfrm_user 45056 1 2024-06-26T07:11:53.8222648Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:11:53.8223315Z iptable_nat 16384 1 2024-06-26T07:11:53.8223852Z nf_conntrack_ipv4 16384 3 2024-06-26T07:11:53.8224439Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T07:11:53.8225088Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T07:11:53.8225713Z nf_nat 32768 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 2024-06-26T07:11:53.8226655Z nf_conntrack 155648 7 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink 2024-06-26T07:11:53.8227504Z xt_addrtype 16384 2 2024-06-26T07:11:53.8227887Z iptable_filter 16384 1 2024-06-26T07:11:53.8228270Z br_netfilter 24576 0 2024-06-26T07:11:53.8228676Z bridge 172032 1 br_netfilter 2024-06-26T07:11:53.8229126Z stp 16384 1 bridge 2024-06-26T07:11:53.8229549Z llc 16384 2 bridge,stp 2024-06-26T07:11:53.8229958Z overlay 86016 0 2024-06-26T07:11:53.8230340Z sunrpc 393216 1 2024-06-26T07:11:53.8230710Z dm_mirror 28672 0 2024-06-26T07:11:53.8231105Z dm_region_hash 20480 1 dm_mirror 2024-06-26T07:11:53.8231587Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T07:11:53.8232098Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T07:11:53.8232663Z dax 69632 1 dm_mod 2024-06-26T07:11:53.8233104Z crc32_pclmul 16384 0 2024-06-26T07:11:53.8233538Z ghash_clmulni_intel 16384 0 2024-06-26T07:11:53.8234043Z pcbc 16384 0 2024-06-26T07:11:53.8234435Z aesni_intel 188416 0 2024-06-26T07:11:53.8234848Z aes_x86_64 20480 1 aesni_intel 2024-06-26T07:11:53.8235317Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:11:53.8235772Z glue_helper 16384 1 aesni_intel 2024-06-26T07:11:53.8236335Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T07:11:53.8236878Z mousedev 24576 0 2024-06-26T07:11:53.8237267Z evdev 20480 3 2024-06-26T07:11:53.8237840Z psmouse 32768 0 2024-06-26T07:11:53.8238390Z button 16384 0 2024-06-26T07:11:53.8238767Z ena 139264 0 2024-06-26T07:11:53.8239143Z ptp 20480 1 ena 2024-06-26T07:11:53.8239552Z pps_core 20480 1 ptp 2024-06-26T07:11:53.8239957Z crc32c_intel 24576 0 2024-06-26T07:11:53.8240341Z autofs4 49152 2 2024-06-26T07:11:53.8240701Z + modinfo nvidia 2024-06-26T07:11:53.8241372Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T07:11:53.8242079Z alias: char-major-195-* 2024-06-26T07:11:53.8242470Z version: 550.54.15 2024-06-26T07:11:53.8242821Z supported: external 2024-06-26T07:11:53.8243178Z license: NVIDIA 2024-06-26T07:11:53.8243573Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:11:53.8244064Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:11:53.8244547Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T07:11:53.8245028Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:11:53.8245537Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:11:53.8246035Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:11:53.8246530Z depends: i2c-core,drm 2024-06-26T07:11:53.8246905Z retpoline: Y 2024-06-26T07:11:53.8247217Z name: nvidia 2024-06-26T07:11:53.8247769Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T07:11:53.8248431Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:11:53.8249090Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:11:53.8249682Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:11:53.8250121Z parm: NVreg_RmLogonRC:int 2024-06-26T07:11:53.8250557Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:11:53.8251017Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:11:53.8251550Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:11:53.8251987Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:11:53.8252523Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:11:53.8253086Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:11:53.8253578Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:11:53.8254005Z parm: NVreg_EnableMSI:int 2024-06-26T07:11:53.8254437Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:11:53.8254916Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:11:53.8255492Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:11:53.8256075Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:11:53.8256645Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:11:53.8257256Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:11:53.8257938Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:11:53.8258566Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:11:53.8259171Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:11:53.8259672Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:11:53.8260213Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:11:53.8260777Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:11:53.8261289Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:11:53.8261862Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:11:53.8262354Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:11:53.8262836Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:11:53.8263292Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:11:53.8263802Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:11:53.8264324Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:11:53.8264809Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:11:53.8265359Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:11:53.8265868Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:11:53.8266351Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:11:53.8266857Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:11:53.8267341Z parm: NVreg_RmMsg:charp 2024-06-26T07:11:53.8267793Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:11:53.8268347Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:11:53.8268839Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:11:53.8269301Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:11:53.8269786Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:11:53.8270268Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:11:53.8270738Z parm: rm_firmware_active:charp 2024-06-26T07:11:53.8271140Z + set +e 2024-06-26T07:11:53.8271437Z + nvidia-smi 2024-06-26T07:11:55.3990436Z Wed Jun 26 07:11:55 2024 2024-06-26T07:11:55.3991407Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:11:55.3992406Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-06-26T07:11:55.3993220Z |-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:11:55.3994038Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-06-26T07:11:55.3994945Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-06-26T07:11:55.3995643Z | | | MIG M. | 2024-06-26T07:11:55.3996201Z |=========================================+========================+======================| 2024-06-26T07:11:55.4133942Z | 0 NVIDIA A10G Off | 00000000:00:1E.0 Off | 0 | 2024-06-26T07:11:55.4134736Z | 0% 33C P0 58W / 300W | 0MiB / 23028MiB | 0% Default | 2024-06-26T07:11:55.4135775Z | | | N/A | 2024-06-26T07:11:55.4136654Z +-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:11:55.4137260Z 2024-06-26T07:11:55.4138020Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:11:55.4138809Z | Processes: | 2024-06-26T07:11:55.4139613Z | GPU GI CI PID Type Process name GPU Memory | 2024-06-26T07:11:55.4140324Z | ID ID Usage | 2024-06-26T07:11:55.4140909Z |=========================================================================================| 2024-06-26T07:11:55.4141578Z | No running processes found | 2024-06-26T07:11:55.4142359Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:11:56.0895912Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-06-26T07:11:57.6572799Z NVIDIA A10G 2024-06-26T07:11:58.1071767Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:11:58.1072441Z + '[' 0 -eq 0 ']' 2024-06-26T07:11:58.1072844Z + echo 'INFO: Ignoring allowed status 0' 2024-06-26T07:11:58.1073537Z + set -e 2024-06-26T07:11:58.1073867Z INFO: Ignoring allowed status 0 2024-06-26T07:11:58.1076324Z == Installing nvidia container toolkit for amzn2 == 2024-06-26T07:11:58.1079305Z + sudo yum install -y yum-utils 2024-06-26T07:11:58.4094811Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:11:59.8728073Z Package yum-utils-1.1.31-46.amzn2.0.1.noarch already installed and latest version 2024-06-26T07:11:59.8728781Z Nothing to do 2024-06-26T07:12:00.0208710Z + [[ amzn2 == \a\m\z\n\2\0\2\3 ]] 2024-06-26T07:12:00.0209512Z + YUM_REPO_URL=https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:12:00.0225003Z + sudo yum-config-manager --add-repo https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:12:00.3219775Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:12:00.3480954Z adding repo from: https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:12:00.3482152Z grabbing file https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T07:12:00.3483119Z repo saved to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T07:12:00.3595613Z + sudo yum install -y nvidia-docker2 2024-06-26T07:12:00.6428045Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:12:02.1060459Z Package nvidia-docker2-2.13.0-1.noarch already installed and latest version 2024-06-26T07:12:02.1061794Z Nothing to do 2024-06-26T07:12:02.2540914Z + sudo systemctl restart docker 2024-06-26T07:12:41.4455535Z Command completed after 1 attempt(s). 2024-06-26T07:12:41.4514058Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:12:41.4514855Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:12:41.4515572Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-06-26T07:12:41.4516251Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:12:41.4524407Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:41.4524930Z env: 2024-06-26T07:12:41.4525229Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:41.4525685Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:41.4526181Z ##[endgroup] 2024-06-26T07:12:41.6728993Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T07:12:41.6881701Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.7/site-packages (5.9.1) 2024-06-26T07:12:41.6981133Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.7/site-packages (11.525.84) 2024-06-26T07:12:41.7916722Z Prepare all required actions 2024-06-26T07:12:41.7917288Z Getting action download info 2024-06-26T07:12:41.9047452Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-06-26T07:12:42.1317177Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-06-26T07:12:42.2571074Z ##[group]Run ./.github/actions/download-build-artifacts 2024-06-26T07:12:42.2571572Z with: 2024-06-26T07:12:42.2571917Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:12:42.2572389Z s3-bucket: gha-artifacts 2024-06-26T07:12:42.2572739Z env: 2024-06-26T07:12:42.2573027Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:42.2573503Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:42.2574004Z ##[endgroup] 2024-06-26T07:12:42.2606048Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:12:42.2606506Z with: 2024-06-26T07:12:42.2606881Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:12:42.2607351Z s3-bucket: gha-artifacts 2024-06-26T07:12:42.2607711Z region: us-east-1 2024-06-26T07:12:42.2608012Z env: 2024-06-26T07:12:42.2608276Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:42.2608736Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:42.2609225Z ##[endgroup] 2024-06-26T07:12:42.6858706Z (node:19663) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:12:42.6859395Z 2024-06-26T07:12:42.6859669Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:12:42.6860387Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:12:42.6861371Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:12:42.7638561Z Found 1 objects with prefix pytorch/pytorch/9673646806/linux-focal-cuda12.1-py3.10-gcc9-sm86/ 2024-06-26T07:12:42.7639668Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:12:50.3140895Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:12:50.3146314Z Artifact download has finished successfully 2024-06-26T07:12:50.3301097Z ##[group]Run unzip -o artifacts.zip 2024-06-26T07:12:50.3301539Z unzip -o artifacts.zip 2024-06-26T07:12:50.3309201Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:50.3309710Z env: 2024-06-26T07:12:50.3310002Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:50.3310463Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:50.3310959Z ##[endgroup] 2024-06-26T07:12:50.3339383Z Archive: artifacts.zip 2024-06-26T07:12:50.3340988Z creating: dist/ 2024-06-26T07:12:52.3171387Z inflating: dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:12:52.3172098Z creating: build/custom_test_artifacts/ 2024-06-26T07:12:52.3172700Z creating: build/custom_test_artifacts/custom-op-build/ 2024-06-26T07:12:52.3173439Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-06-26T07:12:52.3174288Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:12:52.3178930Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:12:52.3179874Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-06-26T07:12:52.3180837Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:12:52.3181847Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:12:52.3182813Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:12:52.3184243Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:12:52.3185600Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:12:52.3186590Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:12:52.3187574Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:12:52.3188718Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:12:52.3189871Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:12:52.3191008Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:12:52.3192133Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:12:52.3193292Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:12:52.3194457Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:12:52.3195490Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:12:52.3196493Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:12:52.3233013Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:12:52.3270952Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:12:52.3272414Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:12:52.3315686Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:12:52.3317309Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:12:52.3319145Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:12:52.3320616Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:12:52.3322031Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:12:52.3323420Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:12:52.3324814Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:12:52.3326209Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:12:52.3327579Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:12:52.3328875Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:12:52.3330117Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:12:52.3331345Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:12:52.3332581Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:12:52.3333781Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:12:52.3335111Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:12:52.3385882Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:12:52.3445184Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:12:52.3447537Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:12:52.3449492Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:12:52.3450875Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:12:52.3451780Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-06-26T07:12:52.3452710Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-06-26T07:12:52.3453730Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-06-26T07:12:52.3454911Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-06-26T07:12:52.3456028Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-06-26T07:12:52.3457084Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-06-26T07:12:52.3458256Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-06-26T07:12:52.3459340Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-06-26T07:12:52.3460435Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-06-26T07:12:52.3461519Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-06-26T07:12:52.3462601Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-06-26T07:12:52.3470191Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-06-26T07:12:52.3597929Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-06-26T07:12:52.3599151Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-06-26T07:12:52.3600259Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-06-26T07:12:52.3601480Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-06-26T07:12:52.3602645Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-06-26T07:12:52.3603749Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-06-26T07:12:52.3604886Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-06-26T07:12:52.3606052Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-06-26T07:12:52.3607200Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-06-26T07:12:52.3608341Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-06-26T07:12:52.3609494Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-06-26T07:12:52.3623133Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-06-26T07:12:52.3702213Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-06-26T07:12:52.3703395Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:12:52.3704575Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:12:52.3705660Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-06-26T07:12:52.3706568Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-06-26T07:12:52.3707464Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-06-26T07:12:52.3708362Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-06-26T07:12:52.3709201Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-06-26T07:12:52.3709978Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-06-26T07:12:52.3710791Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-06-26T07:12:52.3817067Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-06-26T07:12:52.3877331Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-06-26T07:12:52.3878254Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-06-26T07:12:52.3878969Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-06-26T07:12:52.3879787Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:12:52.3885063Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:12:52.3885988Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-06-26T07:12:52.3886918Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:12:52.3887908Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:12:52.3888871Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:12:52.3889959Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:12:52.3891116Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:12:52.3892109Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:12:52.3893095Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:12:52.3894240Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:12:52.3895384Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:12:52.3896517Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:12:52.3897720Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:12:52.3898879Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:12:52.3900042Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:12:52.3901067Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:12:52.3902070Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:12:52.3935978Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:12:52.3973797Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:12:52.3975211Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:12:52.4017639Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:12:52.4019154Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:12:52.4020623Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:12:52.4022073Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:12:52.4023460Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:12:52.4024822Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:12:52.4026195Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:12:52.4027563Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:12:52.4028915Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:12:52.4030186Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:12:52.4031448Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:12:52.4032656Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:12:52.4033869Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:12:52.4035051Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:12:52.4036261Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:12:52.4084478Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:12:52.4141592Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:12:52.4142763Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:12:52.4143737Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:12:52.4144592Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:12:52.4145474Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-06-26T07:12:52.4146408Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-06-26T07:12:52.4147466Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-06-26T07:12:52.4148662Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-06-26T07:12:52.4149807Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-06-26T07:12:52.4150872Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-06-26T07:12:52.4151980Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-06-26T07:12:52.4153093Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-06-26T07:12:52.4154215Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-06-26T07:12:52.4155307Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-06-26T07:12:52.4156399Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-06-26T07:12:52.4166479Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-06-26T07:12:52.4229293Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-06-26T07:12:52.4231422Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:12:52.4232514Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:12:52.4233513Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-06-26T07:12:52.4234425Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-06-26T07:12:52.4235341Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-06-26T07:12:52.4236249Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-06-26T07:12:52.4237123Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-06-26T07:12:52.4237905Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-06-26T07:12:52.4238876Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-06-26T07:12:52.4281442Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-06-26T07:12:52.4282216Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-06-26T07:12:52.4302184Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-06-26T07:12:52.4303100Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:12:52.4304394Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:12:52.4305385Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-06-26T07:12:52.4306390Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:12:52.4307456Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:12:52.4308495Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:12:52.4309653Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:12:52.4310853Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:12:52.4311947Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:12:52.4313000Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:12:52.4314217Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:12:52.4315442Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:12:52.4316642Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:12:52.4317835Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:12:52.4319247Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:12:52.4320476Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:12:52.4321569Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:12:52.4322635Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:12:52.4339050Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:12:52.4376706Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:12:52.4378405Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:12:52.4420615Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:12:52.4422367Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:12:52.4424076Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:12:52.4425839Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:12:52.4427531Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:12:52.4441346Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:12:52.4442847Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:12:52.4444313Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:12:52.4445755Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:12:52.4447094Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:12:52.4448379Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:12:52.4449650Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:12:52.4450929Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:12:52.4452175Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:12:52.4453447Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:12:52.4506043Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:12:52.4564161Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:12:52.4565423Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:12:52.4566481Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:12:52.4567402Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:12:52.4568364Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-06-26T07:12:52.4569379Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-06-26T07:12:52.4570501Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-06-26T07:12:52.4571809Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-06-26T07:12:52.4573018Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-06-26T07:12:52.4574141Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-06-26T07:12:52.4575462Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-06-26T07:12:52.4576747Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-06-26T07:12:52.4577995Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-06-26T07:12:52.4579165Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-06-26T07:12:52.4580334Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-06-26T07:12:52.4581620Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-06-26T07:12:52.4694855Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-06-26T07:12:52.4696042Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-06-26T07:12:52.4697311Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-06-26T07:12:52.4698612Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-06-26T07:12:52.4699864Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-06-26T07:12:52.4701055Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-06-26T07:12:52.4702292Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-06-26T07:12:52.4703538Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-06-26T07:12:52.4704911Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-06-26T07:12:52.4706145Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-06-26T07:12:52.4707377Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-06-26T07:12:52.4718626Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-06-26T07:12:52.4771731Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-06-26T07:12:52.4773052Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:12:52.4774185Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:12:52.4775224Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-06-26T07:12:52.4776188Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-06-26T07:12:52.4777162Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-06-26T07:12:52.4778199Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-06-26T07:12:52.4779107Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-06-26T07:12:52.4779950Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-06-26T07:12:52.4780828Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-06-26T07:12:52.4877349Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-06-26T07:12:52.4918560Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-06-26T07:12:52.4919212Z creating: build/lib/ 2024-06-26T07:12:52.5004518Z inflating: build/lib/libprotobuf-lite.a 2024-06-26T07:12:52.5444735Z inflating: build/lib/libprotobuf.a 2024-06-26T07:12:52.5454165Z inflating: build/lib/libpthreadpool.a 2024-06-26T07:12:52.5462194Z inflating: build/lib/libcpuinfo.a 2024-06-26T07:12:52.5469972Z inflating: build/lib/libcpuinfo_internals.a 2024-06-26T07:12:52.5470813Z inflating: build/lib/libclog.a 2024-06-26T07:12:52.5488198Z inflating: build/lib/libnnpack.a 2024-06-26T07:12:52.5490276Z inflating: build/lib/libnnpack_reference_layers.a 2024-06-26T07:12:52.5552767Z inflating: build/lib/libgtest.a 2024-06-26T07:12:52.5624819Z inflating: build/lib/libbenchmark.a 2024-06-26T07:12:52.5686617Z inflating: build/lib/libasmjit.a 2024-06-26T07:12:52.5693802Z inflating: build/lib/libittnotify.a 2024-06-26T07:12:52.5720753Z inflating: build/lib/libtensorpipe_uv.a 2024-06-26T07:12:52.5843625Z inflating: build/lib/libgloo.a 2024-06-26T07:12:52.5864661Z inflating: build/lib/libfmt.a 2024-06-26T07:12:52.5953796Z inflating: build/lib/libc10.so 2024-06-26T07:12:52.5955352Z inflating: build/lib/libcaffe2_nvrtc.so 2024-06-26T07:12:52.5955846Z inflating: build/lib/libfoxi_loader.a 2024-06-26T07:12:52.5957424Z inflating: build/lib/libtorch_global_deps.so 2024-06-26T07:12:52.5976288Z inflating: build/lib/libpytorch_qnnpack.a 2024-06-26T07:12:52.5993755Z inflating: build/lib/libgmock.a 2024-06-26T07:12:52.5994220Z inflating: build/lib/libgtest_main.a 2024-06-26T07:12:52.5994844Z inflating: build/lib/libbenchmark_main.a 2024-06-26T07:12:52.6480732Z inflating: build/lib/libprotoc.a 2024-06-26T07:12:52.6852243Z inflating: build/lib/libgloo_cuda.a 2024-06-26T07:12:53.6585199Z inflating: build/lib/libdnnl.a 2024-06-26T07:12:53.7135083Z inflating: build/lib/libtensorpipe.a 2024-06-26T07:12:53.7187919Z inflating: build/lib/libc10_cuda.so 2024-06-26T07:12:53.7188418Z inflating: build/lib/libgmock_main.a 2024-06-26T07:12:53.8391312Z inflating: build/lib/libfbgemm.a 2024-06-26T07:12:53.8636388Z inflating: build/lib/libtensorpipe_cuda.a 2024-06-26T07:12:53.9134820Z inflating: build/lib/libkineto.a 2024-06-26T07:12:53.9313175Z inflating: build/lib/libXNNPACK.a 2024-06-26T07:12:53.9353808Z inflating: build/lib/libonnx_proto.a 2024-06-26T07:12:54.0018399Z inflating: build/lib/libonnx.a 2024-06-26T07:12:56.3319315Z inflating: build/lib/libtorch_cpu.so 2024-06-26T07:12:56.3323701Z inflating: build/lib/libunbox_lib.a 2024-06-26T07:12:56.3327523Z inflating: build/lib/libshm.so 2024-06-26T07:12:58.3207180Z inflating: build/lib/libtorch_cuda.so 2024-06-26T07:12:58.3208209Z inflating: build/lib/libtorch.so 2024-06-26T07:12:59.1162954Z inflating: build/lib/libtorch_cuda_linalg.so 2024-06-26T07:12:59.1164076Z inflating: build/lib/libc10d_cuda_test.so 2024-06-26T07:12:59.3061608Z inflating: build/lib/libtorch_python.so 2024-06-26T07:12:59.3127847Z inflating: build/lib/libtorchbind_test.so 2024-06-26T07:12:59.3147743Z inflating: build/lib/libjitbackend_test.so 2024-06-26T07:12:59.3172343Z inflating: build/lib/libbackend_with_compiler.so 2024-06-26T07:12:59.3205747Z inflating: build/lib/libnnapi_backend.so 2024-06-26T07:12:59.3206283Z creating: build/bin/ 2024-06-26T07:12:59.3255413Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-06-26T07:12:59.3304308Z inflating: build/bin/c10_DeviceGuard_test 2024-06-26T07:12:59.3353029Z inflating: build/bin/c10_Device_test 2024-06-26T07:12:59.3410242Z inflating: build/bin/c10_DispatchKeySet_test 2024-06-26T07:12:59.3462547Z inflating: build/bin/c10_Scalar_test 2024-06-26T07:12:59.3510084Z inflating: build/bin/c10_StreamGuard_test 2024-06-26T07:12:59.3563204Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-06-26T07:12:59.3612236Z inflating: build/bin/c10_SymInt_test 2024-06-26T07:12:59.3666338Z inflating: build/bin/c10_InlineStreamGuard_test 2024-06-26T07:12:59.3720721Z inflating: build/bin/c10_SizesAndStrides_test 2024-06-26T07:12:59.3772067Z inflating: build/bin/c10_Bitset_test 2024-06-26T07:12:59.3840989Z inflating: build/bin/c10_cow_test 2024-06-26T07:12:59.3888559Z inflating: build/bin/c10_ConstexprCrc_test 2024-06-26T07:12:59.3936767Z inflating: build/bin/c10_DeadlockDetection_test 2024-06-26T07:12:59.3985873Z inflating: build/bin/c10_Half_test 2024-06-26T07:12:59.4041095Z inflating: build/bin/c10_LeftRight_test 2024-06-26T07:12:59.4093593Z inflating: build/bin/c10_Metaprogramming_test 2024-06-26T07:12:59.4142611Z inflating: build/bin/c10_Synchronized_test 2024-06-26T07:12:59.4196691Z inflating: build/bin/c10_ThreadLocal_test 2024-06-26T07:12:59.4245820Z inflating: build/bin/c10_TypeIndex_test 2024-06-26T07:12:59.4296053Z inflating: build/bin/c10_TypeList_test 2024-06-26T07:12:59.4343043Z inflating: build/bin/c10_TypeTraits_test 2024-06-26T07:12:59.4393364Z inflating: build/bin/c10_accumulate_test 2024-06-26T07:12:59.4446707Z inflating: build/bin/c10_bfloat16_test 2024-06-26T07:12:59.4495528Z inflating: build/bin/c10_bit_cast_test 2024-06-26T07:12:59.4550081Z inflating: build/bin/c10_complex_math_test 2024-06-26T07:12:59.4604217Z inflating: build/bin/c10_complex_test 2024-06-26T07:12:59.4655048Z inflating: build/bin/c10_exception_test 2024-06-26T07:12:59.4703880Z inflating: build/bin/c10_flags_test 2024-06-26T07:12:59.4752331Z inflating: build/bin/c10_generic_math_test 2024-06-26T07:12:59.4910820Z inflating: build/bin/c10_intrusive_ptr_test 2024-06-26T07:12:59.4960306Z inflating: build/bin/c10_irange_test 2024-06-26T07:12:59.5012079Z inflating: build/bin/c10_lazy_test 2024-06-26T07:12:59.5067436Z inflating: build/bin/c10_logging_test 2024-06-26T07:12:59.5140752Z inflating: build/bin/c10_optional_test 2024-06-26T07:12:59.5200279Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-06-26T07:12:59.5252338Z inflating: build/bin/c10_registry_test 2024-06-26T07:12:59.5398357Z inflating: build/bin/c10_small_vector_test 2024-06-26T07:12:59.5448124Z inflating: build/bin/c10_ssize_test 2024-06-26T07:12:59.5498408Z inflating: build/bin/c10_string_util_test 2024-06-26T07:12:59.5555706Z inflating: build/bin/c10_string_view_test 2024-06-26T07:12:59.5604738Z inflating: build/bin/c10_tempfile_test 2024-06-26T07:12:59.5651806Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-06-26T07:12:59.5706241Z inflating: build/bin/c10_typeid_test 2024-06-26T07:12:59.6139488Z inflating: build/bin/protoc-3.13.0.0 2024-06-26T07:12:59.6572975Z inflating: build/bin/protoc 2024-06-26T07:12:59.6624461Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-06-26T07:12:59.6675626Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-06-26T07:12:59.6726755Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-06-26T07:12:59.6777128Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-06-26T07:12:59.6828425Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-06-26T07:12:59.6880793Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-06-26T07:12:59.6927346Z inflating: build/bin/c10_cuda_CUDATest 2024-06-26T07:12:59.6979540Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-06-26T07:12:59.7297438Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-06-26T07:12:59.7629936Z inflating: build/bin/vec_test_all_types_AVX512 2024-06-26T07:12:59.7976297Z inflating: build/bin/vec_test_all_types_AVX2 2024-06-26T07:12:59.8027799Z inflating: build/bin/FileStoreTest 2024-06-26T07:12:59.8081789Z inflating: build/bin/TCPStoreTest 2024-06-26T07:12:59.8133634Z inflating: build/bin/HashStoreTest 2024-06-26T07:12:59.8146710Z inflating: build/bin/ProcessGroupMPITest 2024-06-26T07:12:59.8198889Z inflating: build/bin/test_edge_op_registration 2024-06-26T07:12:59.8202913Z inflating: build/bin/torch_shm_manager 2024-06-26T07:12:59.8205598Z inflating: build/bin/example_allreduce 2024-06-26T07:12:59.8259443Z inflating: build/bin/test_dist_autograd 2024-06-26T07:12:59.8325954Z inflating: build/bin/test_cpp_rpc 2024-06-26T07:12:59.8328112Z inflating: build/bin/parallel_benchmark 2024-06-26T07:12:59.8392986Z inflating: build/bin/test_mobile_nnc 2024-06-26T07:12:59.8401691Z inflating: build/bin/aot_model_compiler_test 2024-06-26T07:12:59.8735188Z inflating: build/bin/test_lazy 2024-06-26T07:12:59.9882153Z inflating: build/bin/test_api 2024-06-26T07:12:59.9953451Z inflating: build/bin/Dict_test 2024-06-26T07:13:00.0004036Z inflating: build/bin/Dimname_test 2024-06-26T07:13:00.0066558Z inflating: build/bin/MaybeOwned_test 2024-06-26T07:13:00.0122274Z inflating: build/bin/NamedTensor_test 2024-06-26T07:13:00.0179505Z inflating: build/bin/apply_utils_test 2024-06-26T07:13:00.0237426Z inflating: build/bin/atest 2024-06-26T07:13:00.0298089Z inflating: build/bin/basic 2024-06-26T07:13:00.0351143Z inflating: build/bin/broadcast_test 2024-06-26T07:13:00.0401175Z inflating: build/bin/cpu_allocator_test 2024-06-26T07:13:00.0456829Z inflating: build/bin/cpu_generator_test 2024-06-26T07:13:00.0508313Z inflating: build/bin/cpu_profiling_allocator_test 2024-06-26T07:13:00.0598801Z inflating: build/bin/cpu_rng_test 2024-06-26T07:13:00.0647099Z inflating: build/bin/dispatch_key_set_test 2024-06-26T07:13:00.0695870Z inflating: build/bin/dlconvertor_test 2024-06-26T07:13:00.0752381Z inflating: build/bin/extension_backend_test 2024-06-26T07:13:00.0805959Z inflating: build/bin/half_test 2024-06-26T07:13:00.0899068Z inflating: build/bin/ivalue_test 2024-06-26T07:13:00.0947225Z inflating: build/bin/lazy_tensor_test 2024-06-26T07:13:00.1000770Z inflating: build/bin/math_kernel_test 2024-06-26T07:13:00.1052665Z inflating: build/bin/memory_format_test 2024-06-26T07:13:00.1104729Z inflating: build/bin/memory_overlapping_test 2024-06-26T07:13:00.1156263Z inflating: build/bin/mobile_memory_cleanup 2024-06-26T07:13:00.1211017Z inflating: build/bin/native_test 2024-06-26T07:13:00.1259972Z inflating: build/bin/operator_name_test 2024-06-26T07:13:00.1309847Z inflating: build/bin/operators_test 2024-06-26T07:13:00.1360883Z inflating: build/bin/packedtensoraccessor_test 2024-06-26T07:13:00.1425241Z inflating: build/bin/pow_test 2024-06-26T07:13:00.1480762Z inflating: build/bin/quantized_test 2024-06-26T07:13:00.1529235Z inflating: build/bin/reduce_ops_test 2024-06-26T07:13:00.1578883Z inflating: build/bin/reportMemoryUsage_test 2024-06-26T07:13:00.1633844Z inflating: build/bin/scalar_tensor_test 2024-06-26T07:13:00.1690610Z inflating: build/bin/scalar_test 2024-06-26T07:13:00.1740875Z inflating: build/bin/StorageUtils_test 2024-06-26T07:13:00.1792220Z inflating: build/bin/stride_properties_test 2024-06-26T07:13:00.1868549Z inflating: build/bin/tensor_iterator_test 2024-06-26T07:13:00.1921744Z inflating: build/bin/test_parallel 2024-06-26T07:13:00.1923825Z inflating: build/bin/thread_init_test 2024-06-26T07:13:00.1978997Z inflating: build/bin/type_ptr_test 2024-06-26T07:13:00.2036300Z inflating: build/bin/type_test 2024-06-26T07:13:00.2087852Z inflating: build/bin/undefined_tensor_test 2024-06-26T07:13:00.2088943Z inflating: build/bin/verify_api_visibility 2024-06-26T07:13:00.2156267Z inflating: build/bin/legacy_vmap_test 2024-06-26T07:13:00.2205619Z inflating: build/bin/weakref_test 2024-06-26T07:13:00.2255550Z inflating: build/bin/wrapdim_test 2024-06-26T07:13:00.2305601Z inflating: build/bin/xla_tensor_test 2024-06-26T07:13:00.2363988Z inflating: build/bin/IListRef_test 2024-06-26T07:13:00.2466678Z inflating: build/bin/List_test 2024-06-26T07:13:00.2531237Z inflating: build/bin/KernelFunction_test 2024-06-26T07:13:00.2649282Z inflating: build/bin/kernel_function_legacy_test 2024-06-26T07:13:00.2743012Z inflating: build/bin/kernel_function_test 2024-06-26T07:13:00.2867130Z inflating: build/bin/kernel_lambda_legacy_test 2024-06-26T07:13:00.2968821Z inflating: build/bin/kernel_lambda_test 2024-06-26T07:13:00.3028511Z inflating: build/bin/kernel_stackbased_test 2024-06-26T07:13:00.3121635Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-06-26T07:13:00.3171085Z inflating: build/bin/CppSignature_test 2024-06-26T07:13:00.3225684Z inflating: build/bin/backend_fallback_test 2024-06-26T07:13:00.3273102Z inflating: build/bin/op_allowlist_test 2024-06-26T07:13:00.3564925Z inflating: build/bin/op_registration_test 2024-06-26T07:13:00.3626650Z inflating: build/bin/inline_container_test 2024-06-26T07:13:00.3678380Z inflating: build/bin/cuda_apply_test 2024-06-26T07:13:00.3727392Z inflating: build/bin/cuda_allocator_test 2024-06-26T07:13:00.3784706Z inflating: build/bin/cuda_atomic_ops_test 2024-06-26T07:13:00.3837518Z inflating: build/bin/cuda_caching_host_allocator_test 2024-06-26T07:13:00.3905379Z inflating: build/bin/cuda_complex_math_test 2024-06-26T07:13:00.3963066Z inflating: build/bin/cuda_complex_test 2024-06-26T07:13:00.4010720Z inflating: build/bin/cuda_device_test 2024-06-26T07:13:00.4066482Z inflating: build/bin/cuda_cub_test 2024-06-26T07:13:00.4115704Z inflating: build/bin/cuda_dlconvertor_test 2024-06-26T07:13:00.4178440Z inflating: build/bin/cuda_distributions_test 2024-06-26T07:13:00.4234302Z inflating: build/bin/cuda_generator_test 2024-06-26T07:13:00.4282609Z inflating: build/bin/cuda_half_test 2024-06-26T07:13:00.4331357Z inflating: build/bin/cuda_integer_divider_test 2024-06-26T07:13:00.4379087Z inflating: build/bin/cuda_optional_test 2024-06-26T07:13:00.4429058Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-06-26T07:13:00.4480899Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-06-26T07:13:00.4528322Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-06-26T07:13:00.4586977Z inflating: build/bin/cuda_stream_test 2024-06-26T07:13:00.4635492Z inflating: build/bin/cuda_cudnn_test 2024-06-26T07:13:00.4685208Z inflating: build/bin/cuda_vectorized_test 2024-06-26T07:13:00.4699627Z inflating: build/bin/tutorial_tensorexpr 2024-06-26T07:13:00.4763422Z inflating: build/bin/ProcessGroupGlooTest 2024-06-26T07:13:00.4819043Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-06-26T07:13:00.4880280Z inflating: build/bin/ProcessGroupNCCLTest 2024-06-26T07:13:00.4940567Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-06-26T07:13:00.5745862Z inflating: build/bin/test_tensorexpr 2024-06-26T07:13:00.6307003Z inflating: build/bin/test_jit 2024-06-26T07:13:00.6307532Z creating: .additional_ci_files/ 2024-06-26T07:13:00.6360892Z inflating: .additional_ci_files/test-times.json 2024-06-26T07:13:00.6572507Z inflating: .additional_ci_files/test-class-times.json 2024-06-26T07:13:00.6600757Z ##[group]Run rm artifacts.zip 2024-06-26T07:13:00.6601180Z rm artifacts.zip 2024-06-26T07:13:00.6610446Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:00.6610968Z env: 2024-06-26T07:13:00.6611264Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:00.6611742Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:00.6612248Z ##[endgroup] 2024-06-26T07:13:00.7320570Z ##[group]Run df -H 2024-06-26T07:13:00.7321264Z df -H 2024-06-26T07:13:00.7329969Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:00.7330494Z env: 2024-06-26T07:13:00.7330785Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:00.7331258Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:00.7331761Z ##[endgroup] 2024-06-26T07:13:00.7359413Z Filesystem Size Used Avail Use% Mounted on 2024-06-26T07:13:00.7359935Z devtmpfs 34G 0 34G 0% /dev 2024-06-26T07:13:00.7360418Z tmpfs 34G 0 34G 0% /dev/shm 2024-06-26T07:13:00.7361328Z tmpfs 34G 512k 34G 1% /run 2024-06-26T07:13:00.7361827Z tmpfs 34G 0 34G 0% /sys/fs/cgroup 2024-06-26T07:13:00.7362306Z /dev/nvme0n1p1 162G 43G 119G 27% / 2024-06-26T07:13:00.7362878Z tmpfs 6.7G 0 6.7G 0% /run/user/0 2024-06-26T07:13:00.7401353Z Prepare all required actions 2024-06-26T07:13:00.7401809Z Getting action download info 2024-06-26T07:13:00.8636814Z ##[group]Run ./.github/actions/download-td-artifacts 2024-06-26T07:13:00.8637297Z with: 2024-06-26T07:13:00.8637582Z env: 2024-06-26T07:13:00.8637881Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:00.8638506Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:00.8639004Z ##[endgroup] 2024-06-26T07:13:00.8670067Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:13:00.8670518Z with: 2024-06-26T07:13:00.8670785Z name: td_results 2024-06-26T07:13:00.8671107Z s3-bucket: gha-artifacts 2024-06-26T07:13:00.8671465Z region: us-east-1 2024-06-26T07:13:00.8671779Z env: 2024-06-26T07:13:00.8672051Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:00.8672510Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:00.8673053Z ##[endgroup] 2024-06-26T07:13:01.2861400Z (node:19684) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:13:01.2862212Z 2024-06-26T07:13:01.2862487Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:13:01.2863211Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:13:01.2864109Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:13:01.3496158Z Found 0 objects with prefix pytorch/pytorch/9673646806/td_results/ 2024-06-26T07:13:01.3502983Z Artifact download has finished successfully 2024-06-26T07:13:01.3632196Z ##[group]Run mkdir -p .additional_ci_files 2024-06-26T07:13:01.3632700Z mkdir -p .additional_ci_files 2024-06-26T07:13:01.3633284Z mv td_results.json .additional_ci_files/td_results.json 2024-06-26T07:13:01.3641575Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:01.3642087Z env: 2024-06-26T07:13:01.3642379Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:01.3642863Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:01.3643358Z ##[endgroup] 2024-06-26T07:13:01.3677061Z mv: cannot stat 'td_results.json': No such file or directory 2024-06-26T07:13:01.3687177Z ##[error]Process completed with exit code 1. 2024-06-26T07:13:01.3730775Z ##[group]Run .github/scripts/parse_ref.py 2024-06-26T07:13:01.3731259Z .github/scripts/parse_ref.py 2024-06-26T07:13:01.3738325Z shell: /usr/bin/bash -e {0} 2024-06-26T07:13:01.3738678Z env: 2024-06-26T07:13:01.3738967Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:01.3739429Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:01.3739925Z ##[endgroup] 2024-06-26T07:13:01.3966162Z Prepare all required actions 2024-06-26T07:13:01.4017688Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-06-26T07:13:01.4018164Z with: 2024-06-26T07:13:01.4018715Z github-token: *** 2024-06-26T07:13:01.4019025Z env: 2024-06-26T07:13:01.4019298Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:01.4019758Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:01.4020244Z ##[endgroup] 2024-06-26T07:13:01.4035362Z ##[group]Run set -eux 2024-06-26T07:13:01.4035722Z set -eux 2024-06-26T07:13:01.4036340Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-06-26T07:13:01.4044267Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:01.4044769Z env: 2024-06-26T07:13:01.4045058Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:01.4045513Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:01.4046163Z GITHUB_TOKEN: *** 2024-06-26T07:13:01.4046473Z ##[endgroup] 2024-06-26T07:13:01.4068059Z + python3 .github/scripts/get_workflow_job_id.py 9673646806 i-036744df35e90073e 2024-06-26T07:13:01.9941561Z setting job-id=26688915911 2024-06-26T07:13:01.9942590Z setting job-name=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:13:02.0149697Z Prepare all required actions 2024-06-26T07:13:02.0150193Z Getting action download info 2024-06-26T07:13:02.1225425Z ##[group]Run ./.github/actions/filter-test-configs 2024-06-26T07:13:02.1225887Z with: 2024-06-26T07:13:02.1226357Z github-token: *** 2024-06-26T07:13:02.1232907Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-06-26T07:13:02.1240377Z job-name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:13:02.1241258Z env: 2024-06-26T07:13:02.1241550Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:02.1242023Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:02.1242520Z ##[endgroup] 2024-06-26T07:13:02.1283032Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:13:02.1283591Z with: 2024-06-26T07:13:02.1283874Z shell: bash 2024-06-26T07:13:02.1284180Z timeout_minutes: 10 2024-06-26T07:13:02.1284512Z max_attempts: 5 2024-06-26T07:13:02.1284825Z retry_wait_seconds: 30 2024-06-26T07:13:02.1285938Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:13:02.1287116Z polling_interval_seconds: 1 2024-06-26T07:13:02.1287494Z warning_on_retry: true 2024-06-26T07:13:02.1287843Z continue_on_error: false 2024-06-26T07:13:02.1288192Z env: 2024-06-26T07:13:02.1288484Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:02.1288951Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:02.1289595Z GITHUB_TOKEN: *** 2024-06-26T07:13:02.1289922Z ##[endgroup] 2024-06-26T07:13:02.1783441Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:13:02.3929649Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T07:13:02.4076195Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.7/site-packages (2.27.1) 2024-06-26T07:13:02.4220565Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.7/site-packages (6.0.1) 2024-06-26T07:13:02.4228615Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (3.7) 2024-06-26T07:13:02.4241691Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2.0.12) 2024-06-26T07:13:02.4261687Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2024.6.2) 2024-06-26T07:13:02.4269559Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (1.26.19) 2024-06-26T07:13:03.1786911Z Command completed after 1 attempt(s). 2024-06-26T07:13:03.1827435Z ##[group]Run set -x 2024-06-26T07:13:03.1827765Z set -x 2024-06-26T07:13:03.1828059Z  2024-06-26T07:13:03.1828617Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:13:03.1829299Z # in runner workspace 2024-06-26T07:13:03.1829846Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-06-26T07:13:03.1837755Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:03.1838499Z env: 2024-06-26T07:13:03.1838804Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:03.1839273Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:03.1839775Z ##[endgroup] 2024-06-26T07:13:03.1860381Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-06-26T07:13:03.2056234Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:13:03.2056818Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:13:03.2057436Z echo "Job name: ${JOB_NAME}" 2024-06-26T07:13:03.2057844Z  2024-06-26T07:13:03.2058390Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:13:03.2059065Z # in runner workspace 2024-06-26T07:13:03.2059670Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-06-26T07:13:03.2060317Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-06-26T07:13:03.2060783Z  --job-name "${JOB_NAME}" \ 2024-06-26T07:13:03.2067516Z  --test-matrix "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-06-26T07:13:03.2074479Z  --selected-test-configs "" \ 2024-06-26T07:13:03.2074937Z  --pr-number "${PR_NUMBER}" \ 2024-06-26T07:13:03.2075494Z  --tag "${TAG}" \ 2024-06-26T07:13:03.2092943Z  --event-name "${EVENT_NAME}" \ 2024-06-26T07:13:03.2093443Z  --schedule "${SCHEDULE}" \ 2024-06-26T07:13:03.2094029Z  --branch "${HEAD_BRANCH}" 2024-06-26T07:13:03.2101696Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:03.2102196Z env: 2024-06-26T07:13:03.2102473Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:03.2102936Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:03.2103627Z GITHUB_TOKEN: *** 2024-06-26T07:13:03.2104460Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:13:03.2105352Z PR_NUMBER: 2024-06-26T07:13:03.2105670Z TAG: ciflow/inductor/129470 2024-06-26T07:13:03.2106027Z EVENT_NAME: push 2024-06-26T07:13:03.2106344Z SCHEDULE: 2024-06-26T07:13:03.2106646Z HEAD_BRANCH: 2024-06-26T07:13:03.2106935Z ##[endgroup] 2024-06-26T07:13:03.2127665Z Workflow: inductor-periodic 2024-06-26T07:13:03.2129184Z Job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:13:03.4231444Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-06-26T07:13:03.5574496Z ##[group]Run echo "Filtered matrix:" 2024-06-26T07:13:03.5574958Z echo "Filtered matrix:" 2024-06-26T07:13:03.5581700Z echo "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" 2024-06-26T07:13:03.5588691Z  2024-06-26T07:13:03.5588985Z echo 2024-06-26T07:13:03.5589390Z echo "Is the current job unstable? False" 2024-06-26T07:13:03.5589867Z  2024-06-26T07:13:03.5590146Z echo 2024-06-26T07:13:03.5590527Z echo "Is keep-going label set? False" 2024-06-26T07:13:03.5590982Z  2024-06-26T07:13:03.5591259Z echo 2024-06-26T07:13:03.5591599Z echo "Renabled issues? " 2024-06-26T07:13:03.5599529Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:03.5600038Z env: 2024-06-26T07:13:03.5600321Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:03.5600793Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:03.5601304Z ##[endgroup] 2024-06-26T07:13:03.5620920Z Filtered matrix: 2024-06-26T07:13:03.5629642Z {include: [{config: dynamo_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}]} 2024-06-26T07:13:03.5636493Z 2024-06-26T07:13:03.5636645Z Is the current job unstable? False 2024-06-26T07:13:03.5636935Z 2024-06-26T07:13:03.5637217Z Is keep-going label set? False 2024-06-26T07:13:03.5637474Z 2024-06-26T07:13:03.5637605Z Renabled issues? 2024-06-26T07:13:03.5686320Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:13:03.5687102Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:13:03.5694155Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:03.5694667Z env: 2024-06-26T07:13:03.5694956Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:03.5695429Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:03.5695940Z JOB_TIMEOUT: 240 2024-06-26T07:13:03.5696251Z ##[endgroup] 2024-06-26T07:13:03.5767937Z ##[group]Run set -x 2024-06-26T07:13:03.5768352Z set -x 2024-06-26T07:13:03.5768681Z  2024-06-26T07:13:03.5769071Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-06-26T07:13:03.5769632Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-06-26T07:13:03.5770223Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-06-26T07:13:03.5770764Z  TEST_COMMAND=.ci/onnx/test.sh 2024-06-26T07:13:03.5771200Z else 2024-06-26T07:13:03.5771561Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:13:03.5772012Z fi 2024-06-26T07:13:03.5772322Z  2024-06-26T07:13:03.5772821Z # detached container should get cleaned up by teardown_ec2_linux 2024-06-26T07:13:03.5773595Z # TODO: Stop building test binaries as part of the build phase 2024-06-26T07:13:03.5774367Z # Used for GPU_FLAG since that doesn't play nice 2024-06-26T07:13:03.5774966Z # shellcheck disable=SC2086,SC2090 2024-06-26T07:13:03.5775450Z container_name=$(docker run \ 2024-06-26T07:13:03.5775905Z  ${GPU_FLAG:-} \ 2024-06-26T07:13:03.5776308Z  -e BUILD_ENVIRONMENT \ 2024-06-26T07:13:03.5776738Z  -e PR_NUMBER \ 2024-06-26T07:13:03.5777113Z  -e GITHUB_ACTIONS \ 2024-06-26T07:13:03.5777631Z  -e GITHUB_REPOSITORY \ 2024-06-26T07:13:03.5778050Z  -e GITHUB_WORKFLOW \ 2024-06-26T07:13:03.5778448Z  -e GITHUB_JOB \ 2024-06-26T07:13:03.5778816Z  -e GITHUB_RUN_ID \ 2024-06-26T07:13:03.5779222Z  -e GITHUB_RUN_NUMBER \ 2024-06-26T07:13:03.5779643Z  -e GITHUB_RUN_ATTEMPT \ 2024-06-26T07:13:03.5780052Z  -e JOB_ID \ 2024-06-26T07:13:03.5780407Z  -e JOB_NAME \ 2024-06-26T07:13:03.5780776Z  -e BASE_SHA \ 2024-06-26T07:13:03.5781140Z  -e BRANCH \ 2024-06-26T07:13:03.5781609Z  -e SHA1 \ 2024-06-26T07:13:03.5781964Z  -e AWS_DEFAULT_REGION \ 2024-06-26T07:13:03.5782378Z  -e IN_WHEEL_TEST \ 2024-06-26T07:13:03.5782771Z  -e SHARD_NUMBER \ 2024-06-26T07:13:03.5783142Z  -e TEST_CONFIG \ 2024-06-26T07:13:03.5783533Z  -e NUM_TEST_SHARDS \ 2024-06-26T07:13:03.5783987Z  -e REENABLED_ISSUES \ 2024-06-26T07:13:03.5784433Z  -e CONTINUE_THROUGH_ERROR \ 2024-06-26T07:13:03.5784866Z  -e VERBOSE_TEST_LOGS \ 2024-06-26T07:13:03.5785285Z  -e NO_TEST_TIMEOUT \ 2024-06-26T07:13:03.5785687Z  -e NO_TD \ 2024-06-26T07:13:03.5786042Z  -e TD_DISTRIBUTED \ 2024-06-26T07:13:03.5786440Z  -e PR_LABELS \ 2024-06-26T07:13:03.5786877Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-06-26T07:13:03.5787354Z  -e SCCACHE_BUCKET \ 2024-06-26T07:13:03.5787759Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-06-26T07:13:03.5788192Z  -e XLA_CUDA \ 2024-06-26T07:13:03.5788624Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-06-26T07:13:03.5789156Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-06-26T07:13:03.5789687Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-06-26T07:13:03.5790222Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-06-26T07:13:03.5790707Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-06-26T07:13:03.5791151Z  -e DASHBOARD_TAG \ 2024-06-26T07:13:03.5791635Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-06-26T07:13:03.5792222Z  --security-opt seccomp=unconfined \ 2024-06-26T07:13:03.5792717Z  --cap-add=SYS_PTRACE \ 2024-06-26T07:13:03.5793127Z  --ipc=host \ 2024-06-26T07:13:03.5793505Z  --shm-size="${SHM_SIZE}" \ 2024-06-26T07:13:03.5793919Z  --tty \ 2024-06-26T07:13:03.5794248Z  --detach \ 2024-06-26T07:13:03.5794630Z  --name="${container_name}" \ 2024-06-26T07:13:03.5795067Z  --user jenkins \ 2024-06-26T07:13:03.5795645Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-06-26T07:13:03.5796234Z  -w /var/lib/jenkins/workspace \ 2024-06-26T07:13:03.5796691Z  "${DOCKER_IMAGE}" 2024-06-26T07:13:03.5797049Z ) 2024-06-26T07:13:03.5797474Z # Propagate download.pytorch.org IP to container 2024-06-26T07:13:03.5798562Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-06-26T07:13:03.5799576Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-06-26T07:13:03.5800522Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-06-26T07:13:03.5808850Z shell: /usr/bin/bash -e {0} 2024-06-26T07:13:03.5809216Z env: 2024-06-26T07:13:03.5809499Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:03.5809950Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:03.5810587Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:13:03.5811113Z PR_NUMBER: 2024-06-26T07:13:03.5811442Z GITHUB_REPOSITORY: pytorch/pytorch 2024-06-26T07:13:03.5811877Z GITHUB_WORKFLOW: inductor-periodic 2024-06-26T07:13:03.5812297Z GITHUB_JOB: test 2024-06-26T07:13:03.5812622Z GITHUB_RUN_ID: 9673646806 2024-06-26T07:13:03.5812998Z GITHUB_RUN_NUMBER: 45567 2024-06-26T07:13:03.5813351Z GITHUB_RUN_ATTEMPT: 1 2024-06-26T07:13:03.5813690Z JOB_ID: 26688915911 2024-06-26T07:13:03.5814516Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:13:03.5815406Z BRANCH: 2024-06-26T07:13:03.5815744Z SHA1: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:03.5816274Z BASE_SHA: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:03.5816768Z TEST_CONFIG: dynamo_eager_timm 2024-06-26T07:13:03.5817155Z SHARD_NUMBER: 1 2024-06-26T07:13:03.5817613Z NUM_TEST_SHARDS: 2 2024-06-26T07:13:03.5817949Z REENABLED_ISSUES: 2024-06-26T07:13:03.5818311Z CONTINUE_THROUGH_ERROR: False 2024-06-26T07:13:03.5818701Z VERBOSE_TEST_LOGS: False 2024-06-26T07:13:03.5819073Z NO_TEST_TIMEOUT: False 2024-06-26T07:13:03.5819418Z NO_TD: False 2024-06-26T07:13:03.5819728Z TD_DISTRIBUTED: False 2024-06-26T07:13:03.5820153Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-06-26T07:13:03.5820683Z SCCACHE_S3_KEY_PREFIX: inductor-periodic 2024-06-26T07:13:03.5821132Z SHM_SIZE: 2g 2024-06-26T07:13:03.5822254Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:13:03.5823453Z XLA_CUDA: 2024-06-26T07:13:03.5824013Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:13:03.5824665Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-06-26T07:13:03.5825129Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-06-26T07:13:03.5825566Z DASHBOARD_TAG: 2024-06-26T07:13:03.5826114Z HUGGING_FACE_HUB_TOKEN: *** 2024-06-26T07:13:03.5826504Z ##[endgroup] 2024-06-26T07:13:03.5846213Z + [[ dynamo_eager_timm == \m\u\l\t\i\g\p\u ]] 2024-06-26T07:13:03.5846961Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]] 2024-06-26T07:13:03.5847479Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:13:03.5854040Z +++ nproc --ignore=2 2024-06-26T07:13:03.5867011Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_9673646806 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:13:14.8143485Z + container_name=a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:13:14.8144464Z + grep download.pytorch.org /etc/hosts 2024-06-26T07:13:14.8147372Z + docker exec -i a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 sudo bash -c '/bin/cat >> /etc/hosts' 2024-06-26T07:13:14.8793000Z + echo DOCKER_CONTAINER_ID=a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:13:14.8796269Z ++ echo dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:13:14.8798213Z + docker exec -t a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 sh -c 'pip install dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-06-26T07:13:15.2617690Z Processing ./dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+gitb8c4c54) 2024-06-26T07:13:15.5732471Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.13.1) 2024-06-26T07:13:15.5737616Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (4.12.2) 2024-06-26T07:13:15.5739842Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.12.1) 2024-06-26T07:13:15.5742923Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.8.8) 2024-06-26T07:13:15.5745866Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.1.4) 2024-06-26T07:13:15.5748321Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2024.2.0) 2024-06-26T07:13:15.5774212Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.3.0) 2024-06-26T07:13:15.5831067Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.21.2) 2024-06-26T07:13:15.6255116Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.1.5) 2024-06-26T07:13:15.6414098Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.3.0) 2024-06-26T07:13:16.9540225Z Installing collected packages: torch 2024-06-26T07:13:25.6892019Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-06-26T07:13:25.6893372Z timm 0.9.7 requires torchvision, which is not installed. 2024-06-26T07:13:25.6894048Z Successfully installed torch-2.5.0a0+gitb8c4c54 2024-06-26T07:13:25.7795166Z ++ dirname .ci/pytorch/test.sh 2024-06-26T07:13:25.7800535Z + source .ci/pytorch/common.sh 2024-06-26T07:13:25.7803141Z +++ dirname .ci/pytorch/common.sh 2024-06-26T07:13:25.7809574Z ++ source .ci/pytorch/common_utils.sh 2024-06-26T07:13:25.7810607Z +++ declare -f -t trap_add 2024-06-26T07:13:25.7815939Z ++ set -ex 2024-06-26T07:13:25.7816580Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-06-26T07:13:25.7817362Z ++ BUILD_TEST_LIBTORCH=0 2024-06-26T07:13:25.7817889Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]] 2024-06-26T07:13:25.7818875Z ++ stat -c %u /var/lib/jenkins/workspace 2024-06-26T07:13:25.7827036Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-06-26T07:13:25.7827620Z + trap_add cleanup_workspace EXIT 2024-06-26T07:13:25.7828062Z + trap_add_cmd=cleanup_workspace 2024-06-26T07:13:25.7828544Z + shift 2024-06-26T07:13:25.7828826Z + for trap_add_name in "$@" 2024-06-26T07:13:25.7831815Z +++ trap -p EXIT 2024-06-26T07:13:25.7833624Z ++ eval 'extract_trap_cmd ' 2024-06-26T07:13:25.7834131Z +++ extract_trap_cmd 2024-06-26T07:13:25.7834648Z +++ printf '%s\n' '' 2024-06-26T07:13:25.7835188Z ++ printf '%s\n' cleanup_workspace 2024-06-26T07:13:25.7835775Z + trap -- ' 2024-06-26T07:13:25.7836209Z cleanup_workspace' EXIT 2024-06-26T07:13:25.7836859Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-06-26T07:13:26.2919697Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-06-26T07:13:26.2932476Z + echo 'Environment variables:' 2024-06-26T07:13:26.2932904Z Environment variables: 2024-06-26T07:13:26.2933232Z + env 2024-06-26T07:13:26.2939135Z INSTALLED_DB=yes 2024-06-26T07:13:26.2939710Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-06-26T07:13:26.2940359Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:13:26.2940771Z NV_NVML_DEV_VERSION=12.1.105-1 2024-06-26T07:13:26.2941544Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:13:26.2942328Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:13:26.2943032Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-06-26T07:13:26.2943746Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:13:26.2944906Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:13:26.2945634Z HOSTNAME=a3eb29f592f4 2024-06-26T07:13:26.2946628Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_bd066f04-28e7-4034-88dd-ae5dd42eac27 2024-06-26T07:13:26.2947733Z GITHUB_ACTION=__self 2024-06-26T07:13:26.2948216Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:13:26.2952856Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-06-26T07:13:26.2956761Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-06-26T07:13:26.2957319Z NV_NVTX_VERSION=12.1.105-1 2024-06-26T07:13:26.2957696Z GITHUB_RUN_NUMBER=45567 2024-06-26T07:13:26.2958379Z TEST_CONFIG=dynamo_eager_timm 2024-06-26T07:13:26.2958800Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:13:26.2959316Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:13:26.2959821Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-06-26T07:13:26.2960260Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-06-26T07:13:26.2960691Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-06-26T07:13:26.2961139Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:13:26.2961644Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:13:26.2962110Z GITHUB_REF_TYPE=tag 2024-06-26T07:13:26.2962448Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:13:26.2962843Z NCCL_VERSION=2.17.1-1 2024-06-26T07:13:26.2963241Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:26.2963834Z XLA_CUDA= 2024-06-26T07:13:26.2964334Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:13:26.2965884Z *** 2024-06-26T07:13:26.2966244Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:13:26.2966667Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:13:26.2967033Z GITHUB_ACTIONS=true 2024-06-26T07:13:26.2967370Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:26.2967897Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-06-26T07:13:26.2968436Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-06-26T07:13:26.2968902Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:26.2969406Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:13:26.2969885Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:26.2970806Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:13:26.2971566Z UCC_HOME=/usr 2024-06-26T07:13:26.2971914Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-06-26T07:13:26.2972325Z VERBOSE_TEST_LOGS=False 2024-06-26T07:13:26.2972677Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:13:26.2973134Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-06-26T07:13:26.2973639Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:13:26.2974125Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-06-26T07:13:26.2974504Z SHARD_NUMBER=1 2024-06-26T07:13:26.2974820Z GITHUB_REF_PROTECTED=false 2024-06-26T07:13:26.2975177Z HOME=/var/lib/jenkins 2024-06-26T07:13:26.2975542Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:13:26.2975986Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:13:26.2976480Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:13:26.2977028Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-06-26T07:13:26.2977553Z CUDA_VERSION=12.1.1 2024-06-26T07:13:26.2977972Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-06-26T07:13:26.2978417Z NUM_TEST_SHARDS=2 2024-06-26T07:13:26.2978878Z UCX_HOME=/usr 2024-06-26T07:13:26.2979396Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-06-26T07:13:26.2980527Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_bd066f04-28e7-4034-88dd-ae5dd42eac27 2024-06-26T07:13:26.2981919Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:13:26.2983281Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_bd066f04-28e7-4034-88dd-ae5dd42eac27 2024-06-26T07:13:26.2984390Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:13:26.2985041Z GITHUB_EVENT_NAME=push 2024-06-26T07:13:26.2985378Z DASHBOARD_TAG= 2024-06-26T07:13:26.2985684Z GITHUB_RUN_ID=9673646806 2024-06-26T07:13:26.2986136Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-06-26T07:13:26.2986676Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-06-26T07:13:26.2987681Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_bd066f04-28e7-4034-88dd-ae5dd42eac27 2024-06-26T07:13:26.2988590Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:13:26.2988994Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-06-26T07:13:26.2989375Z PR_NUMBER= 2024-06-26T07:13:26.2989683Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:13:26.2990057Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:13:26.2990497Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:13:26.2990966Z TERM=xterm 2024-06-26T07:13:26.2991303Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-06-26T07:13:26.2991735Z INSTALLED_VISION=yes 2024-06-26T07:13:26.2992045Z BRANCH= 2024-06-26T07:13:26.2992333Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:13:26.2992745Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:13:26.2993172Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:13:26.2993931Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:13:26.2994688Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:13:26.2995197Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:13:26.2995747Z REENABLED_ISSUES= 2024-06-26T07:13:26.2996067Z SHLVL=1 2024-06-26T07:13:26.2996341Z MAX_JOBS=14 2024-06-26T07:13:26.2996663Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-06-26T07:13:26.2997024Z NVARCH=x86_64 2024-06-26T07:13:26.2997342Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:13:26.2997812Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:26.2998701Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:13:26.2999206Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-06-26T07:13:26.2999877Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:13:26.3000453Z GITHUB_JOB=test 2024-06-26T07:13:26.3000851Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-06-26T07:13:26.3001415Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:13:26.3001935Z NO_TEST_TIMEOUT=False 2024-06-26T07:13:26.3002271Z TD_DISTRIBUTED=False 2024-06-26T07:13:26.3002649Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-06-26T07:13:26.3003102Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:13:26.3003533Z NV_NVPROF_VERSION=12.1.105-1 2024-06-26T07:13:26.3003904Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:13:26.3004253Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:13:26.3004616Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:13:26.3005667Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:13:26.3006734Z GITHUB_BASE_REF= 2024-06-26T07:13:26.3007059Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:13:26.3007436Z CI=true 2024-06-26T07:13:26.3007758Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:13:26.3008174Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:13:26.3008556Z JOB_ID=26688915911 2024-06-26T07:13:26.3008877Z INSTALLED_PROTOBUF=yes 2024-06-26T07:13:26.3009213Z GITHUB_HEAD_REF= 2024-06-26T07:13:26.3009515Z GITHUB_ACTION_REF= 2024-06-26T07:13:26.3010068Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:13:26.3010609Z GITHUB_WORKFLOW=inductor-periodic 2024-06-26T07:13:26.3011017Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:13:26.3011926Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_bd066f04-28e7-4034-88dd-ae5dd42eac27 2024-06-26T07:13:26.3012749Z NO_TD=False 2024-06-26T07:13:26.3013075Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:13:26.3013444Z _=/usr/bin/env 2024-06-26T07:13:26.3013928Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-06-26T07:13:26.3085027Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-06-26T07:13:26.3086293Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-06-26T07:13:26.3087479Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-06-26T07:13:26.3088513Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-06-26T07:13:26.3089145Z + BUILD_DIR=build 2024-06-26T07:13:26.3089498Z + BUILD_RENAMED_DIR=build_renamed 2024-06-26T07:13:26.3090036Z + BUILD_BIN_DIR=build/bin 2024-06-26T07:13:26.3090507Z + SHARD_NUMBER=1 2024-06-26T07:13:26.3090828Z + NUM_TEST_SHARDS=2 2024-06-26T07:13:26.3091153Z + export VALGRIND=ON 2024-06-26T07:13:26.3091467Z + VALGRIND=ON 2024-06-26T07:13:26.3091945Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]] 2024-06-26T07:13:26.3092445Z + [[ 0 == \1 ]] 2024-06-26T07:13:26.3092742Z + [[ False == \1 ]] 2024-06-26T07:13:26.3093205Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]] 2024-06-26T07:13:26.3093742Z ++ realpath build/custom_test_artifacts 2024-06-26T07:13:26.3099222Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-06-26T07:13:26.3100152Z + [[ -n '' ]] 2024-06-26T07:13:26.3100519Z + echo 'Environment variables' 2024-06-26T07:13:26.3100924Z Environment variables 2024-06-26T07:13:26.3101264Z + env 2024-06-26T07:13:26.3106674Z INSTALLED_DB=yes 2024-06-26T07:13:26.3107233Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-06-26T07:13:26.3107943Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:13:26.3108479Z NV_NVML_DEV_VERSION=12.1.105-1 2024-06-26T07:13:26.3109281Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:13:26.3110081Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:13:26.3110780Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-06-26T07:13:26.3111466Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:13:26.3112093Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:13:26.3112768Z HOSTNAME=a3eb29f592f4 2024-06-26T07:13:26.3113842Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_bd066f04-28e7-4034-88dd-ae5dd42eac27 2024-06-26T07:13:26.3114651Z GITHUB_ACTION=__self 2024-06-26T07:13:26.3115047Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:13:26.3119055Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-06-26T07:13:26.3122885Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-06-26T07:13:26.3123404Z NV_NVTX_VERSION=12.1.105-1 2024-06-26T07:13:26.3123758Z GITHUB_RUN_NUMBER=45567 2024-06-26T07:13:26.3124119Z TEST_CONFIG=dynamo_eager_timm 2024-06-26T07:13:26.3124516Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:13:26.3124984Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:13:26.3125587Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-06-26T07:13:26.3126041Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-06-26T07:13:26.3126482Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-06-26T07:13:26.3126923Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:13:26.3127427Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:13:26.3127897Z GITHUB_REF_TYPE=tag 2024-06-26T07:13:26.3128230Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:13:26.3128604Z NCCL_VERSION=2.17.1-1 2024-06-26T07:13:26.3129001Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:26.3129449Z XLA_CUDA= 2024-06-26T07:13:26.3129928Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:13:26.3130319Z *** 2024-06-26T07:13:26.3130598Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:13:26.3130995Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:13:26.3131364Z GITHUB_ACTIONS=true 2024-06-26T07:13:26.3131689Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:26.3132173Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-06-26T07:13:26.3132712Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-06-26T07:13:26.3133181Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:26.3133685Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:13:26.3134174Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:26.3135086Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:13:26.3135836Z UCC_HOME=/usr 2024-06-26T07:13:26.3136171Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-06-26T07:13:26.3136573Z VERBOSE_TEST_LOGS=False 2024-06-26T07:13:26.3136918Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:13:26.3137443Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-06-26T07:13:26.3137946Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:13:26.3138414Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-06-26T07:13:26.3138790Z SHARD_NUMBER=1 2024-06-26T07:13:26.3139089Z GITHUB_REF_PROTECTED=false 2024-06-26T07:13:26.3139444Z HOME=/var/lib/jenkins 2024-06-26T07:13:26.3139814Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:13:26.3140343Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:13:26.3140833Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:13:26.3141371Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-06-26T07:13:26.3141790Z CUDA_VERSION=12.1.1 2024-06-26T07:13:26.3142199Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-06-26T07:13:26.3142642Z NUM_TEST_SHARDS=2 2024-06-26T07:13:26.3142948Z UCX_HOME=/usr 2024-06-26T07:13:26.3143458Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-06-26T07:13:26.3144514Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_bd066f04-28e7-4034-88dd-ae5dd42eac27 2024-06-26T07:13:26.3145890Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:13:26.3147244Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_bd066f04-28e7-4034-88dd-ae5dd42eac27 2024-06-26T07:13:26.3148359Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:13:26.3148999Z GITHUB_EVENT_NAME=push 2024-06-26T07:13:26.3149335Z DASHBOARD_TAG= 2024-06-26T07:13:26.3149636Z GITHUB_RUN_ID=9673646806 2024-06-26T07:13:26.3150090Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-06-26T07:13:26.3150613Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-06-26T07:13:26.3151599Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_bd066f04-28e7-4034-88dd-ae5dd42eac27 2024-06-26T07:13:26.3152494Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:13:26.3152899Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-06-26T07:13:26.3153264Z PR_NUMBER= 2024-06-26T07:13:26.3153561Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:13:26.3153882Z VALGRIND=ON 2024-06-26T07:13:26.3154183Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:13:26.3154617Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:13:26.3155150Z TERM=xterm 2024-06-26T07:13:26.3155483Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-06-26T07:13:26.3155897Z INSTALLED_VISION=yes 2024-06-26T07:13:26.3156211Z BRANCH= 2024-06-26T07:13:26.3156506Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:13:26.3156910Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:13:26.3157322Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:13:26.3158261Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:13:26.3159027Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:13:26.3159511Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:13:26.3172110Z REENABLED_ISSUES= 2024-06-26T07:13:26.3172441Z SHLVL=1 2024-06-26T07:13:26.3172716Z MAX_JOBS=14 2024-06-26T07:13:26.3173056Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-06-26T07:13:26.3173426Z NVARCH=x86_64 2024-06-26T07:13:26.3173731Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:13:26.3174194Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:26.3174739Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:13:26.3175238Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-06-26T07:13:26.3175900Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:13:26.3176471Z GITHUB_JOB=test 2024-06-26T07:13:26.3176863Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-06-26T07:13:26.3177532Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:13:26.3178066Z NO_TEST_TIMEOUT=False 2024-06-26T07:13:26.3178418Z TD_DISTRIBUTED=False 2024-06-26T07:13:26.3178803Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-06-26T07:13:26.3179257Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:13:26.3179697Z NV_NVPROF_VERSION=12.1.105-1 2024-06-26T07:13:26.3180071Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:13:26.3180435Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:13:26.3180800Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:13:26.3181852Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:13:26.3182922Z GITHUB_BASE_REF= 2024-06-26T07:13:26.3183386Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:13:26.3183779Z CI=true 2024-06-26T07:13:26.3184107Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:13:26.3184533Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:13:26.3184918Z JOB_ID=26688915911 2024-06-26T07:13:26.3185248Z INSTALLED_PROTOBUF=yes 2024-06-26T07:13:26.3185596Z GITHUB_HEAD_REF= 2024-06-26T07:13:26.3185918Z GITHUB_ACTION_REF= 2024-06-26T07:13:26.3186363Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:13:26.3186898Z GITHUB_WORKFLOW=inductor-periodic 2024-06-26T07:13:26.3187311Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:13:26.3188217Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_bd066f04-28e7-4034-88dd-ae5dd42eac27 2024-06-26T07:13:26.3189030Z NO_TD=False 2024-06-26T07:13:26.3189341Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:13:26.3189733Z _=/usr/bin/env 2024-06-26T07:13:26.3190097Z + echo 'Testing pytorch' 2024-06-26T07:13:26.3190457Z Testing pytorch 2024-06-26T07:13:26.3190805Z + export LANG=C.UTF-8 2024-06-26T07:13:26.3191154Z + LANG=C.UTF-8 2024-06-26T07:13:26.3191478Z + PR_NUMBER= 2024-06-26T07:13:26.3191817Z + [[ dynamo_eager_timm == \d\e\f\a\u\l\t ]] 2024-06-26T07:13:26.3192305Z + [[ dynamo_eager_timm == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-06-26T07:13:26.3192795Z + [[ dynamo_eager_timm == \s\l\o\w ]] 2024-06-26T07:13:26.3193412Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-06-26T07:13:26.3194111Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]] 2024-06-26T07:13:26.3194657Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:13:26.3195139Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:13:26.3195584Z + [[ dynamo_eager_timm == *crossref* ]] 2024-06-26T07:13:26.3196158Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-06-26T07:13:26.3196789Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]] 2024-06-26T07:13:26.3197532Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:13:26.3198313Z + pip_install --user ninja==1.10.2 2024-06-26T07:13:26.3198863Z + pip install --progress-bar off --user ninja==1.10.2 2024-06-26T07:13:26.6756774Z Collecting ninja==1.10.2 2024-06-26T07:13:26.6959137Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-06-26T07:13:26.7103667Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-06-26T07:13:28.0168607Z Installing collected packages: ninja 2024-06-26T07:13:28.0235423Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-06-26T07:13:28.0236695Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-06-26T07:13:28.0295460Z Successfully installed ninja-1.10.2 2024-06-26T07:13:28.1241809Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:13:28.1244709Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:13:28.1246679Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-06-26T07:13:28.1247410Z + install_tlparse 2024-06-26T07:13:28.1247953Z + pip_install --user tlparse==0.3.7 2024-06-26T07:13:28.1248717Z + pip install --progress-bar off --user tlparse==0.3.7 2024-06-26T07:13:28.4788900Z Collecting tlparse==0.3.7 2024-06-26T07:13:28.4960416Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-06-26T07:13:28.5089637Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-06-26T07:13:29.8249692Z Installing collected packages: tlparse 2024-06-26T07:13:29.8616131Z Successfully installed tlparse-0.3.7 2024-06-26T07:13:29.9498525Z ++ python -m site --user-base 2024-06-26T07:13:29.9651771Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:13:29.9653617Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]] 2024-06-26T07:13:29.9654464Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]] 2024-06-26T07:13:29.9655254Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:13:29.9656204Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-06-26T07:13:29.9657441Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass 2024-06-26T07:13:29.9658160Z + cd test 2024-06-26T07:13:29.9658685Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-06-26T07:13:31.4414195Z + [[ dynamo_eager_timm == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-06-26T07:13:31.4414842Z + [[ dynamo_eager_timm == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-06-26T07:13:31.4415619Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:13:31.4416132Z + pushd test 2024-06-26T07:13:31.4416453Z ~/workspace/test ~/workspace 2024-06-26T07:13:31.4416986Z ++ python -c 'import torch; print(torch.version.cuda)' 2024-06-26T07:13:32.9292415Z + CUDA_VERSION=12.1 2024-06-26T07:13:32.9292931Z + '[' 12.1 == 12.4 ']' 2024-06-26T07:13:32.9293280Z + ISCUDA124= 2024-06-26T07:13:32.9293583Z + popd 2024-06-26T07:13:32.9293862Z ~/workspace 2024-06-26T07:13:32.9298045Z + DYNAMO_BENCHMARK_FLAGS=() 2024-06-26T07:13:32.9298469Z + [[ dynamo_eager_timm == *dynamo_eager* ]] 2024-06-26T07:13:32.9299036Z + DYNAMO_BENCHMARK_FLAGS+=(--backend eager) 2024-06-26T07:13:32.9299512Z + [[ dynamo_eager_timm == *dynamic* ]] 2024-06-26T07:13:32.9299972Z + [[ dynamo_eager_timm == *cpu_inductor* ]] 2024-06-26T07:13:32.9300669Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-06-26T07:13:32.9329235Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-06-26T07:13:32.9330494Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-06-26T07:13:32.9331916Z + cd test 2024-06-26T07:13:32.9332487Z + python -c 'import torch; print(torch.__config__.show())' 2024-06-26T07:13:34.2851736Z PyTorch built with: 2024-06-26T07:13:34.2852314Z - GCC 9.4 2024-06-26T07:13:34.2852698Z - C++ Version: 201703 2024-06-26T07:13:34.2853536Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:13:34.2854632Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:13:34.2855487Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:13:34.2856008Z - LAPACK is enabled (usually provided by MKL) 2024-06-26T07:13:34.2856482Z - NNPACK is enabled 2024-06-26T07:13:34.2856875Z - CPU capability usage: AVX2 2024-06-26T07:13:34.2857399Z - CUDA Runtime 12.1 2024-06-26T07:13:34.2857934Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-06-26T07:13:34.2858524Z - CuDNN 90.1 (built against CUDA 12.4) 2024-06-26T07:13:34.2858968Z - Magma 2.6.1 2024-06-26T07:13:34.2866154Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-06-26T07:13:34.2872786Z 2024-06-26T07:13:34.5703395Z + cd test 2024-06-26T07:13:34.5704071Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-06-26T07:13:35.7952625Z ATen/Parallel: 2024-06-26T07:13:35.7953035Z at::get_num_threads() : 8 2024-06-26T07:13:35.7953570Z at::get_num_interop_threads() : 16 2024-06-26T07:13:35.7954146Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:13:35.7954652Z omp_get_max_threads() : 8 2024-06-26T07:13:35.7955730Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:13:35.7956725Z mkl_get_max_threads() : 8 2024-06-26T07:13:35.7957367Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:13:35.7958198Z std::thread::hardware_concurrency() : 16 2024-06-26T07:13:35.7958649Z Environment variables: 2024-06-26T07:13:35.7959011Z OMP_NUM_THREADS : [not set] 2024-06-26T07:13:35.7959387Z MKL_NUM_THREADS : [not set] 2024-06-26T07:13:35.7959776Z ATen parallel backend: OpenMP 2024-06-26T07:13:35.7960035Z 2024-06-26T07:13:36.0378326Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-06-26T07:13:36.0379064Z + [[ dynamo_eager_timm == *backward* ]] 2024-06-26T07:13:36.0379516Z + [[ dynamo_eager_timm == *xla* ]] 2024-06-26T07:13:36.0379951Z + [[ dynamo_eager_timm == *executorch* ]] 2024-06-26T07:13:36.0380439Z + [[ dynamo_eager_timm == \j\i\t\_\l\e\g\a\c\y ]] 2024-06-26T07:13:36.0381339Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-06-26T07:13:36.0382080Z + [[ dynamo_eager_timm == distributed ]] 2024-06-26T07:13:36.0382665Z + [[ dynamo_eager_timm == *inductor_distributed* ]] 2024-06-26T07:13:36.0383252Z + [[ dynamo_eager_timm == *inductor-halide* ]] 2024-06-26T07:13:36.0383846Z + [[ dynamo_eager_timm == *inductor-micro-benchmark* ]] 2024-06-26T07:13:36.0384375Z + [[ dynamo_eager_timm == *huggingface* ]] 2024-06-26T07:13:36.0384834Z + [[ dynamo_eager_timm == *timm* ]] 2024-06-26T07:13:36.0385249Z + install_torchvision 2024-06-26T07:13:36.0385583Z + local orig_preload 2024-06-26T07:13:36.0385912Z + local commit 2024-06-26T07:13:36.0386231Z ++ get_pinned_commit vision 2024-06-26T07:13:36.0386628Z ++ cat .github/ci_commit_pins/vision.txt 2024-06-26T07:13:36.0392551Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:36.0393092Z + orig_preload= 2024-06-26T07:13:36.0393463Z + '[' -n '' ']' 2024-06-26T07:13:36.0394300Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:36.0395738Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:36.3495450Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:36.3499843Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-df66a1qq 2024-06-26T07:13:36.3516125Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-df66a1qq 2024-06-26T07:13:37.7071482Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-06-26T07:13:37.7089962Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:38.9177112Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:39.1856983Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:41.4724375Z Preparing metadata (setup.py) ... [?25l- \ done 2024-06-26T07:13:41.4770573Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2) 2024-06-26T07:13:41.4773982Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+gitb8c4c54) 2024-06-26T07:13:41.4779628Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-06-26T07:13:41.4981729Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-06-26T07:13:41.4988723Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-06-26T07:13:41.4992088Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.12.1) 2024-06-26T07:13:41.4995308Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-06-26T07:13:41.4998773Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-06-26T07:13:41.5002020Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.2.0) 2024-06-26T07:13:41.5426960Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-06-26T07:13:41.5578687Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-06-26T07:13:41.5653240Z Building wheels for collected packages: torchvision 2024-06-26T07:14:53.5290334Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | done 2024-06-26T07:14:53.5317021Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2112752 sha256=78c6b78861702622e9a2634065d02569c82538afb9b66dc621502b9425cc85dd 2024-06-26T07:14:53.5319550Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-06-26T07:14:53.5354770Z Successfully built torchvision 2024-06-26T07:14:54.7124826Z Installing collected packages: torchvision 2024-06-26T07:14:55.0923444Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-06-26T07:14:55.2060259Z + '[' -n '' ']' 2024-06-26T07:14:55.2060572Z + id=0 2024-06-26T07:14:55.2060889Z + test_dynamo_benchmark timm_models 0 2024-06-26T07:14:55.2062829Z ++ pwd 2024-06-26T07:14:55.2065077Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:14:55.2066367Z + local suite=timm_models 2024-06-26T07:14:55.2066820Z + shift 2024-06-26T07:14:55.2067098Z + local shard_id=0 2024-06-26T07:14:55.2067430Z + shift 2024-06-26T07:14:55.2070418Z + [[ dynamo_eager_timm == *perf_compare* ]] 2024-06-26T07:14:55.2071023Z + [[ dynamo_eager_timm == *perf* ]] 2024-06-26T07:14:55.2071490Z + [[ dynamo_eager_timm == *cpu_inductor* ]] 2024-06-26T07:14:55.2071961Z + [[ dynamo_eager_timm == *aot_inductor* ]] 2024-06-26T07:14:55.2072696Z + test_single_dynamo_benchmark inference timm_models 0 --inference --bfloat16 2024-06-26T07:14:55.2073297Z ++ pwd 2024-06-26T07:14:55.2073790Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:14:55.2074509Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:14:55.2081971Z + local name=inference 2024-06-26T07:14:55.2082475Z + shift 2024-06-26T07:14:55.2082786Z + local suite=timm_models 2024-06-26T07:14:55.2083138Z + shift 2024-06-26T07:14:55.2083424Z + local shard_id=0 2024-06-26T07:14:55.2083740Z + shift 2024-06-26T07:14:55.2084020Z + partition_flags=() 2024-06-26T07:14:55.2084369Z + local partition_flags 2024-06-26T07:14:55.2085968Z + [[ -n 2 ]] 2024-06-26T07:14:55.2086305Z + [[ -n 0 ]] 2024-06-26T07:14:55.2087003Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-06-26T07:14:55.2087689Z + [[ dynamo_eager_timm == *perf_compare* ]] 2024-06-26T07:14:55.2088154Z + [[ dynamo_eager_timm == *perf* ]] 2024-06-26T07:14:55.2088590Z + [[ dynamo_eager_timm == *aot_inductor* ]] 2024-06-26T07:14:55.2090220Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --backend eager --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv 2024-06-26T07:15:00.9944210Z 2024-06-26T07:15:01.3203244Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:15:01.3203575Z 2024-06-26T07:15:01.4222733Z model.safetensors: 0% 0.00/95.5M [00:00 will be ignored 2024-06-26T07:20:52.7694618Z pass 2024-06-26T07:20:52.8261790Z TIMING: entire_frame_compile:10.25802 backend_compile:0.00071 2024-06-26T07:20:52.8262773Z STATS: call_* op count: 1028 | FakeTensor.__torch_dispatch__:4361 | FakeTensorMode.__torch_dispatch__:5847 2024-06-26T07:20:52.8263759Z Dynamo produced 3 graphs covering 1028 ops with 7 graph breaks (5 unique) 2024-06-26T07:20:56.5554509Z 2024-06-26T07:20:57.3303393Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:20:57.3304470Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:20:57.3305828Z cuda train botnet26t_256 2024-06-26T07:21:02.1501888Z pass 2024-06-26T07:21:02.1673127Z TIMING: entire_frame_compile:3.42434 backend_compile:0.00017 2024-06-26T07:21:02.1674004Z STATS: call_* op count: 341 | FakeTensor.__torch_dispatch__:259 | FakeTensorMode.__torch_dispatch__:4959 2024-06-26T07:21:02.1674931Z Dynamo produced 2 graphs covering 341 ops with 6 graph breaks (5 unique) 2024-06-26T07:21:05.7986072Z 2024-06-26T07:21:10.6898474Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:21:10.6899090Z loading model: 0it [00:04, ?it/s] 2024-06-26T07:21:10.6899656Z cuda train cait_m36_384 2024-06-26T07:21:10.6904751Z Traceback (most recent call last): 2024-06-26T07:21:10.6905726Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2321, in validate_model 2024-06-26T07:21:10.6906505Z self.model_iter_fn(model, example_inputs) 2024-06-26T07:21:10.6907554Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 371, in forward_and_backward_pass 2024-06-26T07:21:10.6909079Z pred = mod(*cloned_inputs) 2024-06-26T07:21:10.6910406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T07:21:10.6911438Z return self._call_impl(*args, **kwargs) 2024-06-26T07:21:10.6912597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:21:10.6913553Z return forward_call(*args, **kwargs) 2024-06-26T07:21:10.6914474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 361, in forward 2024-06-26T07:21:10.6915294Z x = self.forward_features(x) 2024-06-26T07:21:10.6916240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 346, in forward_features 2024-06-26T07:21:10.6917102Z x = self.blocks(x) 2024-06-26T07:21:10.6918410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T07:21:10.6919393Z return self._call_impl(*args, **kwargs) 2024-06-26T07:21:10.6920483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:21:10.6921629Z return forward_call(*args, **kwargs) 2024-06-26T07:21:10.6922881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/container.py", line 249, in forward 2024-06-26T07:21:10.6923909Z input = module(input) 2024-06-26T07:21:10.6924876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T07:21:10.6925774Z return self._call_impl(*args, **kwargs) 2024-06-26T07:21:10.6926701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:21:10.6927541Z return forward_call(*args, **kwargs) 2024-06-26T07:21:10.6928568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 197, in forward 2024-06-26T07:21:10.6929447Z x = x + self.drop_path(self.gamma_1 * self.attn(self.norm1(x))) 2024-06-26T07:21:10.6930551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T07:21:10.6931447Z return self._call_impl(*args, **kwargs) 2024-06-26T07:21:10.6932372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:21:10.6933219Z return forward_call(*args, **kwargs) 2024-06-26T07:21:10.6934069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 151, in forward 2024-06-26T07:21:10.6934906Z x = (attn @ v).transpose(1, 2).reshape(B, N, C) 2024-06-26T07:21:10.6938280Z torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 21.98 GiB of which 16.44 MiB is free. Process 23294 has 21.95 GiB memory in use. Of the allocated memory 21.21 GiB is allocated by PyTorch, and 430.66 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) 2024-06-26T07:21:10.6941081Z 2024-06-26T07:21:10.6941389Z The above exception was the direct cause of the following exception: 2024-06-26T07:21:10.6941840Z 2024-06-26T07:21:10.6941997Z Traceback (most recent call last): 2024-06-26T07:21:10.6942655Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4229, in run 2024-06-26T07:21:10.6943320Z ) = runner.load_model( 2024-06-26T07:21:10.6943992Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 303, in load_model 2024-06-26T07:21:10.6944762Z self.validate_model(model, example_inputs) 2024-06-26T07:21:10.6945618Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2323, in validate_model 2024-06-26T07:21:10.6946389Z raise RuntimeError("Eager run failed") from e 2024-06-26T07:21:10.6946866Z RuntimeError: Eager run failed 2024-06-26T07:21:10.6947122Z 2024-06-26T07:21:10.6947244Z eager_fail_to_run 2024-06-26T07:21:14.1871486Z 2024-06-26T07:21:14.8937880Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:21:14.8938765Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:21:14.8939638Z cuda train coat_lite_mini 2024-06-26T07:21:18.9876119Z pass 2024-06-26T07:21:19.0051162Z TIMING: entire_frame_compile:2.81313 backend_compile:0.00017 2024-06-26T07:21:19.0052029Z STATS: call_* op count: 480 | FakeTensor.__torch_dispatch__:958 | FakeTensorMode.__torch_dispatch__:8124 2024-06-26T07:21:19.0052944Z Dynamo produced 2 graphs covering 480 ops with 6 graph breaks (5 unique) 2024-06-26T07:21:22.6320189Z 2024-06-26T07:21:24.1477920Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:21:24.1478593Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:21:24.1479062Z cuda train convit_base 2024-06-26T07:21:24.1789205Z WARNING:common:fp64 golden ref were not generated for convit_base. Setting accuracy check to cosine 2024-06-26T07:21:30.9683205Z pass 2024-06-26T07:21:31.0171735Z TIMING: entire_frame_compile:5.93057 backend_compile:0.00036 2024-06-26T07:21:31.0172687Z STATS: call_* op count: 1216 | FakeTensor.__torch_dispatch__:1535 | FakeTensorMode.__torch_dispatch__:16273 2024-06-26T07:21:31.0173617Z Dynamo produced 3 graphs covering 1216 ops with 7 graph breaks (5 unique) 2024-06-26T07:21:34.6230138Z 2024-06-26T07:21:35.3314022Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:21:35.3314596Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:21:35.3315045Z cuda train convmixer_768_32 2024-06-26T07:21:43.7818454Z pass 2024-06-26T07:21:43.8408605Z TIMING: entire_frame_compile:2.408 backend_compile:0.00012 2024-06-26T07:21:43.8409651Z STATS: call_* op count: 235 | FakeTensor.__torch_dispatch__:1600 | FakeTensorMode.__torch_dispatch__:9359 2024-06-26T07:21:43.8410859Z Dynamo produced 2 graphs covering 235 ops with 5 graph breaks (4 unique) 2024-06-26T07:21:47.4118577Z 2024-06-26T07:21:48.9124633Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:21:48.9125227Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:21:48.9125738Z cuda train convnext_base 2024-06-26T07:21:58.2123004Z W0626 07:21:58.211000 140482701369984 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:22:11.8969807Z pass 2024-06-26T07:22:11.9660773Z TIMING: entire_frame_compile:18.15774 backend_compile:0.00113 2024-06-26T07:22:11.9661822Z STATS: call_* op count: 1621 | FakeTensor.__torch_dispatch__:7938 | FakeTensorMode.__torch_dispatch__:9256 2024-06-26T07:22:11.9662754Z Dynamo produced 3 graphs covering 1621 ops with 7 graph breaks (5 unique) 2024-06-26T07:22:15.8758521Z 2024-06-26T07:22:16.5049283Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:16.5049785Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:16.5050263Z cuda train crossvit_9_240 2024-06-26T07:22:21.1295019Z W0626 07:22:21.128000 140201755607680 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:22:31.0126332Z pass 2024-06-26T07:22:31.0263617Z TIMING: entire_frame_compile:12.94018 backend_compile:0.00091 2024-06-26T07:22:31.0264564Z STATS: call_* op count: 1403 | FakeTensor.__torch_dispatch__:6797 | FakeTensorMode.__torch_dispatch__:10631 2024-06-26T07:22:31.0267321Z Dynamo produced 3 graphs covering 1403 ops with 7 graph breaks (5 unique) 2024-06-26T07:22:34.8606901Z 2024-06-26T07:22:35.8493280Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:35.8493782Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:35.8494225Z cuda train cspdarknet53 2024-06-26T07:22:44.9223976Z W0626 07:22:44.921000 139784573252224 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:22:54.3383704Z pass 2024-06-26T07:22:54.3659725Z TIMING: entire_frame_compile:15.53105 backend_compile:0.00078 2024-06-26T07:22:54.3660811Z STATS: call_* op count: 1070 | FakeTensor.__torch_dispatch__:4347 | FakeTensorMode.__torch_dispatch__:7602 2024-06-26T07:22:54.3661781Z Dynamo produced 3 graphs covering 1070 ops with 7 graph breaks (5 unique) 2024-06-26T07:22:58.1805616Z 2024-06-26T07:22:59.5709997Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:59.5710628Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:22:59.5711084Z cuda train deit_base_distilled_patch16_224 2024-06-26T07:23:05.9721050Z W0626 07:23:05.971000 140192480596608 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:23:11.7319739Z pass 2024-06-26T07:23:11.7898268Z TIMING: entire_frame_compile:7.44344 backend_compile:0.00058 2024-06-26T07:23:11.7899192Z STATS: call_* op count: 841 | FakeTensor.__torch_dispatch__:3959 | FakeTensorMode.__torch_dispatch__:5417 2024-06-26T07:23:11.7900115Z Dynamo produced 3 graphs covering 841 ops with 7 graph breaks (5 unique) 2024-06-26T07:23:15.5086924Z 2024-06-26T07:23:16.6276806Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:16.6278652Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:23:16.6279112Z cuda train dla102 2024-06-26T07:23:23.6034897Z W0626 07:23:23.602000 140634899583616 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:23:38.2195206Z pass 2024-06-26T07:23:38.2476980Z TIMING: entire_frame_compile:18.22899 backend_compile:0.001 2024-06-26T07:23:38.2479385Z STATS: call_* op count: 1389 | FakeTensor.__torch_dispatch__:8000 | FakeTensorMode.__torch_dispatch__:13316 2024-06-26T07:23:38.2480441Z Dynamo produced 3 graphs covering 1389 ops with 7 graph breaks (5 unique) 2024-06-26T07:23:42.1786000Z 2024-06-26T07:23:43.6295559Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:43.6296045Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:23:43.6296493Z cuda train dm_nfnet_f0 2024-06-26T07:23:53.4644105Z pass 2024-06-26T07:23:53.5133636Z TIMING: entire_frame_compile:6.77029 backend_compile:0.00031 2024-06-26T07:23:53.5134630Z STATS: call_* op count: 620 | FakeTensorMode.__torch_dispatch__:7364 | FakeTensor.__torch_dispatch__:333 2024-06-26T07:23:53.5135557Z Dynamo produced 2 graphs covering 620 ops with 6 graph breaks (5 unique) 2024-06-26T07:23:57.1600318Z 2024-06-26T07:23:58.4801618Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:58.4802533Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:23:58.4803410Z cuda train dpn107 2024-06-26T07:24:15.9383991Z pass 2024-06-26T07:24:16.0152491Z TIMING: entire_frame_compile:10.69481 backend_compile:0.00032 2024-06-26T07:24:16.0153367Z STATS: call_* op count: 746 | FakeTensor.__torch_dispatch__:823 | FakeTensorMode.__torch_dispatch__:14860 2024-06-26T07:24:16.0154548Z Dynamo produced 2 graphs covering 746 ops with 6 graph breaks (5 unique) 2024-06-26T07:24:19.7462555Z 2024-06-26T07:24:20.4881662Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:20.4882274Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:20.4883544Z cuda train eca_botnext26ts_256 2024-06-26T07:24:25.3370738Z W0626 07:24:25.336000 139965568766592 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:24:30.3316615Z pass 2024-06-26T07:24:30.3478648Z TIMING: entire_frame_compile:8.48703 backend_compile:0.00048 2024-06-26T07:24:30.3479934Z STATS: call_* op count: 707 | FakeTensor.__torch_dispatch__:2333 | FakeTensorMode.__torch_dispatch__:5316 2024-06-26T07:24:30.3481296Z Dynamo produced 3 graphs covering 707 ops with 7 graph breaks (5 unique) 2024-06-26T07:24:34.0916160Z 2024-06-26T07:24:34.8709366Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:34.8709848Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:34.8710630Z cuda train eca_halonext26ts 2024-06-26T07:24:39.8130108Z W0626 07:24:39.812000 140312831509120 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:24:44.9965224Z pass 2024-06-26T07:24:45.0109751Z TIMING: entire_frame_compile:8.75916 backend_compile:0.0005 2024-06-26T07:24:45.0110882Z STATS: call_* op count: 740 | FakeTensor.__torch_dispatch__:2408 | FakeTensorMode.__torch_dispatch__:5481 2024-06-26T07:24:45.0113319Z Dynamo produced 3 graphs covering 740 ops with 7 graph breaks (5 unique) 2024-06-26T07:24:48.7117112Z 2024-06-26T07:24:49.3487800Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:49.3488348Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:49.3488790Z cuda train ese_vovnet19b_dw 2024-06-26T07:24:52.6078810Z W0626 07:24:52.607000 139815873905280 torch/_logging/_internal.py:1040] [7/0] Profiler function will be ignored 2024-06-26T07:24:57.3480718Z pass 2024-06-26T07:24:57.3556887Z TIMING: entire_frame_compile:7.16576 backend_compile:0.00044 2024-06-26T07:24:57.3557884Z STATS: call_* op count: 459 | FakeTensor.__torch_dispatch__:2120 | FakeTensorMode.__torch_dispatch__:3352 2024-06-26T07:24:57.3559052Z Dynamo produced 4 graphs covering 459 ops with 7 graph breaks (5 unique) 2024-06-26T07:25:01.0936665Z 2024-06-26T07:25:01.8092123Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:01.8092747Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:01.8093300Z cuda train fbnetc_100 2024-06-26T07:25:07.8326789Z W0626 07:25:07.831000 140353156088448 torch/_logging/_internal.py:1040] [7/0] Profiler function will be ignored 2024-06-26T07:25:16.9554639Z pass 2024-06-26T07:25:16.9622579Z TIMING: entire_frame_compile:14.15535 backend_compile:0.00078 2024-06-26T07:25:16.9623744Z STATS: call_* op count: 993 | FakeTensor.__torch_dispatch__:4238 | FakeTensorMode.__torch_dispatch__:7209 2024-06-26T07:25:16.9624675Z Dynamo produced 4 graphs covering 993 ops with 7 graph breaks (5 unique) 2024-06-26T07:25:20.8304802Z 2024-06-26T07:25:21.7303585Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:21.7304614Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:21.7305505Z cuda train fbnetv3_b 2024-06-26T07:25:30.7963624Z pass 2024-06-26T07:25:30.8092922Z TIMING: entire_frame_compile:7.71177 backend_compile:0.00027 2024-06-26T07:25:30.8093801Z STATS: call_* op count: 610 | FakeTensor.__torch_dispatch__:1113 | FakeTensorMode.__torch_dispatch__:11628 2024-06-26T07:25:30.8094732Z Dynamo produced 2 graphs covering 610 ops with 6 graph breaks (5 unique) 2024-06-26T07:25:34.5072002Z 2024-06-26T07:25:35.5092070Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:35.5092794Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:25:35.5093504Z cuda train gernet_l 2024-06-26T07:25:42.8984662Z pass 2024-06-26T07:25:42.9208756Z TIMING: entire_frame_compile:5.41255 backend_compile:0.00021 2024-06-26T07:25:42.9209789Z STATS: call_* op count: 410 | FakeTensor.__torch_dispatch__:407 | FakeTensorMode.__torch_dispatch__:6318 2024-06-26T07:25:42.9210706Z Dynamo produced 2 graphs covering 410 ops with 6 graph breaks (5 unique) 2024-06-26T07:25:46.5973733Z 2024-06-26T07:25:47.3536594Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:47.3537069Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:47.3537601Z cuda train ghostnet_100 2024-06-26T07:25:52.1453110Z pass 2024-06-26T07:25:52.1519597Z TIMING: entire_frame_compile:3.77932 backend_compile:0.00015 2024-06-26T07:25:52.1520630Z STATS: call_* op count: 360 | FakeTensor.__torch_dispatch__:1755 | FakeTensorMode.__torch_dispatch__:12053 2024-06-26T07:25:52.1521538Z Dynamo produced 2 graphs covering 360 ops with 6 graph breaks (5 unique) 2024-06-26T07:25:55.7623216Z 2024-06-26T07:25:56.8443645Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:56.8444505Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:25:56.8447373Z cuda train gluon_inception_v3 2024-06-26T07:26:07.3753401Z W0626 07:26:07.374000 139871325041280 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:26:20.3425237Z pass 2024-06-26T07:26:20.3624016Z TIMING: entire_frame_compile:20.83306 backend_compile:0.00104 2024-06-26T07:26:20.3625599Z STATS: call_* op count: 1376 | FakeTensor.__torch_dispatch__:6058 | FakeTensorMode.__torch_dispatch__:10448 2024-06-26T07:26:20.3626558Z Dynamo produced 3 graphs covering 1376 ops with 7 graph breaks (5 unique) 2024-06-26T07:26:24.3757569Z 2024-06-26T07:26:25.1980663Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:25.1981139Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:25.1984560Z cuda train gmixer_24_224 2024-06-26T07:26:30.7593408Z pass 2024-06-26T07:26:30.7783773Z TIMING: entire_frame_compile:3.31148 backend_compile:0.0002 2024-06-26T07:26:30.7784874Z STATS: call_* op count: 684 | FakeTensor.__torch_dispatch__:1892 | FakeTensorMode.__torch_dispatch__:10774 2024-06-26T07:26:30.7785850Z Dynamo produced 2 graphs covering 684 ops with 6 graph breaks (5 unique) 2024-06-26T07:26:34.3647269Z 2024-06-26T07:26:35.0857821Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:35.0858445Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:35.0858888Z cuda train gmlp_s16_224 2024-06-26T07:26:40.5102164Z W0626 07:26:40.509000 140133348913792 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:26:51.5274929Z pass 2024-06-26T07:26:51.5447540Z TIMING: entire_frame_compile:14.05508 backend_compile:0.00102 2024-06-26T07:26:51.5448521Z STATS: call_* op count: 1458 | FakeTensor.__torch_dispatch__:7742 | FakeTensorMode.__torch_dispatch__:10246 2024-06-26T07:26:51.5449857Z Dynamo produced 3 graphs covering 1458 ops with 7 graph breaks (5 unique) 2024-06-26T07:26:55.4008865Z 2024-06-26T07:26:56.9726926Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:56.9727403Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:26:56.9727863Z cuda train hrnet_w18 2024-06-26T07:27:13.3175832Z pass 2024-06-26T07:27:13.3399855Z TIMING: entire_frame_compile:11.55193 backend_compile:0.00034 2024-06-26T07:27:13.3401025Z STATS: call_* op count: 1394 | FakeTensor.__torch_dispatch__:5982 | FakeTensorMode.__torch_dispatch__:41761 2024-06-26T07:27:13.3402176Z Dynamo produced 2 graphs covering 1394 ops with 5 graph breaks (4 unique) 2024-06-26T07:27:17.1373929Z 2024-06-26T07:27:28.1907376Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:27:28.1907916Z loading model: 0it [00:11, ?it/s] 2024-06-26T07:27:28.1908374Z cuda train inception_v3 2024-06-26T07:27:38.8962094Z pass 2024-06-26T07:27:38.9203785Z TIMING: entire_frame_compile:8.27185 backend_compile:0.00026 2024-06-26T07:27:38.9205545Z STATS: call_* op count: 506 | FakeTensor.__torch_dispatch__:662 | FakeTensorMode.__torch_dispatch__:10430 2024-06-26T07:27:38.9207345Z Dynamo produced 2 graphs covering 506 ops with 6 graph breaks (5 unique) 2024-06-26T07:27:42.6242310Z 2024-06-26T07:27:43.9599717Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:27:43.9600380Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:27:43.9600981Z cuda train jx_nest_base 2024-06-26T07:27:54.9876144Z W0626 07:27:54.986000 140624759427712 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:28:06.4654826Z pass 2024-06-26T07:28:06.5182977Z TIMING: entire_frame_compile:16.7317 backend_compile:0.00103 2024-06-26T07:28:06.5184035Z STATS: call_* op count: 1700 | FakeTensor.__torch_dispatch__:7040 | FakeTensorMode.__torch_dispatch__:12821 2024-06-26T07:28:06.5185099Z Dynamo produced 3 graphs covering 1700 ops with 7 graph breaks (5 unique) 2024-06-26T07:28:10.4036878Z 2024-06-26T07:28:11.4806229Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:28:11.4806798Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:28:11.4807235Z cuda train lcnet_050 2024-06-26T07:28:14.2800181Z pass 2024-06-26T07:28:14.2818012Z TIMING: entire_frame_compile:2.36638 backend_compile:0.00014 2024-06-26T07:28:14.2818889Z STATS: call_* op count: 168 | FakeTensor.__torch_dispatch__:302 | FakeTensorMode.__torch_dispatch__:3439 2024-06-26T07:28:14.2819799Z Dynamo produced 2 graphs covering 168 ops with 6 graph breaks (5 unique) 2024-06-26T07:28:17.8768233Z 2024-06-26T07:28:18.6991126Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:28:18.6991718Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:28:18.6992199Z cuda train levit_128 2024-06-26T07:28:23.9815663Z W0626 07:28:23.980000 139702058488448 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:28:31.9435683Z pass 2024-06-26T07:28:31.9477099Z TIMING: entire_frame_compile:12.06226 backend_compile:0.00074 2024-06-26T07:28:31.9478429Z STATS: call_* op count: 1225 | FakeTensor.__torch_dispatch__:5184 | FakeTensorMode.__torch_dispatch__:12697 2024-06-26T07:28:31.9479481Z Dynamo produced 3 graphs covering 1225 ops with 7 graph breaks (5 unique) 2024-06-26T07:28:35.8335847Z 2024-06-26T07:28:36.9141311Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:28:36.9142392Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:28:36.9143083Z cuda train mixer_b16_224 2024-06-26T07:28:42.0193478Z W0626 07:28:42.018000 140127027581568 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:28:47.5959324Z pass 2024-06-26T07:28:47.6485105Z TIMING: entire_frame_compile:7.05562 backend_compile:0.00053 2024-06-26T07:28:47.6486547Z STATS: call_* op count: 720 | FakeTensor.__torch_dispatch__:3831 | FakeTensorMode.__torch_dispatch__:4914 2024-06-26T07:28:47.6487772Z Dynamo produced 3 graphs covering 720 ops with 7 graph breaks (5 unique) 2024-06-26T07:28:48.5204182Z accuracy pass_rate=96.67% 2024-06-26T07:28:48.5207935Z calls_captured gmean=0.00x mean=833.533x 2024-06-26T07:28:48.5209214Z unique_graphs gmean=0.00x mean=2.533x 2024-06-26T07:28:48.5213849Z graph_breaks gmean=0.00x mean=6.267x 2024-06-26T07:28:48.5217837Z unique_graph_breaks gmean=0.00x mean=4.767x 2024-06-26T07:28:48.5221603Z autograd_captures gmean=0.00x mean=0.000x 2024-06-26T07:28:48.5225682Z autograd_compiles gmean=0.00x mean=0.000x 2024-06-26T07:28:48.5229273Z cudagraph_skips gmean=0.00x mean=0.000x 2024-06-26T07:28:49.1066794Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy//dynamo_eager_timm_training.csv 2024-06-26T07:28:49.3434034Z adv_inception_v3 PASS 2024-06-26T07:28:49.3437904Z beit_base_patch16_224 PASS 2024-06-26T07:28:49.3443320Z botnet26t_256 PASS 2024-06-26T07:28:49.3448900Z cait_m36_384 XFAIL 2024-06-26T07:28:49.3454193Z coat_lite_mini PASS 2024-06-26T07:28:49.3459664Z convit_base PASS 2024-06-26T07:28:49.3464721Z convmixer_768_32 PASS 2024-06-26T07:28:49.3470000Z convnext_base PASS 2024-06-26T07:28:49.3474980Z crossvit_9_240 PASS 2024-06-26T07:28:49.3480512Z cspdarknet53 PASS 2024-06-26T07:28:49.3485601Z deit_base_distilled_patch16_224 PASS 2024-06-26T07:28:49.3490736Z dla102 PASS 2024-06-26T07:28:49.3495755Z dm_nfnet_f0 PASS 2024-06-26T07:28:49.3501318Z dpn107 PASS 2024-06-26T07:28:49.3506284Z eca_botnext26ts_256 PASS 2024-06-26T07:28:49.3511287Z eca_halonext26ts PASS 2024-06-26T07:28:49.3516276Z ese_vovnet19b_dw PASS 2024-06-26T07:28:49.3521782Z fbnetc_100 PASS 2024-06-26T07:28:49.3526559Z fbnetv3_b PASS 2024-06-26T07:28:49.3531664Z gernet_l PASS 2024-06-26T07:28:49.3536562Z ghostnet_100 PASS 2024-06-26T07:28:49.3542059Z gluon_inception_v3 PASS 2024-06-26T07:28:49.3547014Z gmixer_24_224 PASS 2024-06-26T07:28:49.3551941Z gmlp_s16_224 PASS 2024-06-26T07:28:49.3557034Z hrnet_w18 PASS 2024-06-26T07:28:49.3562429Z inception_v3 PASS 2024-06-26T07:28:49.3567365Z jx_nest_base PASS 2024-06-26T07:28:49.3572308Z lcnet_050 PASS 2024-06-26T07:28:49.3577274Z levit_128 PASS 2024-06-26T07:28:49.3582670Z mixer_b16_224 PASS 2024-06-26T07:28:49.3959498Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy//dynamo_eager_timm_training.csv 2024-06-26T07:28:49.6357177Z adv_inception_v3 PASS 2024-06-26T07:28:49.6361923Z beit_base_patch16_224 PASS 2024-06-26T07:28:49.6366771Z botnet26t_256 PASS 2024-06-26T07:28:49.6371379Z cait_m36_384 PASS 2024-06-26T07:28:49.6375988Z coat_lite_mini PASS 2024-06-26T07:28:49.6380898Z convit_base PASS 2024-06-26T07:28:49.6385536Z convmixer_768_32 PASS 2024-06-26T07:28:49.6390151Z convnext_base PASS 2024-06-26T07:28:49.6394755Z crossvit_9_240 PASS 2024-06-26T07:28:49.6399518Z cspdarknet53 PASS 2024-06-26T07:28:49.6404284Z deit_base_distilled_patch16_224 PASS 2024-06-26T07:28:49.6408791Z dla102 PASS 2024-06-26T07:28:49.6413390Z dm_nfnet_f0 PASS 2024-06-26T07:28:49.6418119Z dpn107 PASS 2024-06-26T07:28:49.6422729Z eca_botnext26ts_256 PASS 2024-06-26T07:28:49.6427449Z eca_halonext26ts PASS 2024-06-26T07:28:49.6431888Z ese_vovnet19b_dw PASS 2024-06-26T07:28:49.6436487Z fbnetc_100 PASS 2024-06-26T07:28:49.6441139Z fbnetv3_b PASS 2024-06-26T07:28:49.6445769Z gernet_l PASS 2024-06-26T07:28:49.6451022Z ghostnet_100 PASS 2024-06-26T07:28:49.6455609Z gluon_inception_v3 PASS 2024-06-26T07:28:49.6460711Z gmixer_24_224 PASS 2024-06-26T07:28:49.6465315Z gmlp_s16_224 PASS 2024-06-26T07:28:49.6469725Z hrnet_w18 PASS 2024-06-26T07:28:49.6474411Z inception_v3 PASS 2024-06-26T07:28:49.6479541Z jx_nest_base PASS 2024-06-26T07:28:49.6484417Z lcnet_050 PASS 2024-06-26T07:28:49.6488884Z levit_128 PASS 2024-06-26T07:28:49.6493481Z mixer_b16_224 PASS 2024-06-26T07:28:49.6875286Z + cleanup_workspace 2024-06-26T07:28:49.6876319Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-06-26T07:28:49.6877733Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-06-26T07:28:49.6879182Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-06-26T07:28:49.6880056Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-06-26T07:28:49.6880932Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-06-26T07:28:49.6881808Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-06-26T07:28:49.6882527Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-06-26T07:28:50.1242151Z ##[group]Run cat test/**/*_toprint.log || true 2024-06-26T07:28:50.1242686Z cat test/**/*_toprint.log || true 2024-06-26T07:28:50.1250454Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:28:50.1250966Z env: 2024-06-26T07:28:50.1251251Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:50.1251720Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:50.1252476Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:50.1253136Z ##[endgroup] 2024-06-26T07:28:50.1314601Z cat: test/**/*_toprint.log: No such file or directory 2024-06-26T07:28:50.1391854Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-06-26T07:28:50.1392331Z kill "$MONITOR_SCRIPT_PID" 2024-06-26T07:28:50.1400141Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:28:50.1400643Z env: 2024-06-26T07:28:50.1400924Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:50.1401395Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:50.1402138Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:50.1402798Z MONITOR_SCRIPT_PID: 19638 2024-06-26T07:28:50.1403153Z ##[endgroup] 2024-06-26T07:28:50.1588065Z Prepare all required actions 2024-06-26T07:28:50.1588532Z Getting action download info 2024-06-26T07:28:50.3293951Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-06-26T07:28:50.4850956Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-06-26T07:28:50.4851427Z with: 2024-06-26T07:28:50.4851943Z file-suffix: test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688915911 2024-06-26T07:28:50.4852588Z s3-bucket: gha-artifacts 2024-06-26T07:28:50.4852921Z env: 2024-06-26T07:28:50.4853202Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:50.4853654Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:50.4854535Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:50.4855180Z ##[endgroup] 2024-06-26T07:28:50.4896217Z ##[group]Run # Remove any previous test jsons if they exist 2024-06-26T07:28:50.4896923Z # Remove any previous test jsons if they exist 2024-06-26T07:28:50.4897510Z rm -f test-jsons-*.zip 2024-06-26T07:28:50.4898042Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-06-26T07:28:50.4905799Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:28:50.4906312Z env: 2024-06-26T07:28:50.4906588Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:50.4907049Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:50.4907805Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:50.4908700Z FILE_SUFFIX: test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688915911 2024-06-26T07:28:50.4909328Z ##[endgroup] 2024-06-26T07:28:50.5080015Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-06-26T07:28:50.5108630Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-06-26T07:28:50.5109355Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-06-26T07:28:50.5115138Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-06-26T07:28:50.5141108Z ##[group]Run # Remove any previous test reports if they exist 2024-06-26T07:28:50.5141784Z # Remove any previous test reports if they exist 2024-06-26T07:28:50.5142330Z rm -f test-reports-*.zip 2024-06-26T07:28:50.5142936Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-06-26T07:28:50.5150586Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:28:50.5151091Z env: 2024-06-26T07:28:50.5151370Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:50.5151839Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:50.5152598Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:50.5153516Z FILE_SUFFIX: test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688915911 2024-06-26T07:28:50.5154143Z ##[endgroup] 2024-06-26T07:28:50.5308335Z adding: test/test-reports/inference_timm_models.csv (deflated 69%) 2024-06-26T07:28:50.5309139Z adding: test/test-reports/training_timm_models.csv (deflated 66%) 2024-06-26T07:28:50.5312881Z adding: test/test-reports/training_timm_model_graph_breaks.csv (deflated 97%) 2024-06-26T07:28:50.5313826Z adding: test/test-reports/training_timm_model_graph_break_deduped.csv (deflated 73%) 2024-06-26T07:28:50.5339220Z ##[group]Run # Remove any previous usage logs if they exist 2024-06-26T07:28:50.5339862Z # Remove any previous usage logs if they exist 2024-06-26T07:28:50.5340373Z rm -f logs-*.zip 2024-06-26T07:28:50.5341051Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-06-26T07:28:50.5341864Z # so check to see if the file exists first 2024-06-26T07:28:50.5342374Z if [ -f 'usage_log.txt' ]; then 2024-06-26T07:28:50.5342923Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-06-26T07:28:50.5343427Z fi 2024-06-26T07:28:50.5343923Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-06-26T07:28:50.5344508Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-06-26T07:28:50.5345009Z fi 2024-06-26T07:28:50.5352342Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:28:50.5352852Z env: 2024-06-26T07:28:50.5353129Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:50.5353591Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:50.5354332Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:50.5355220Z FILE_SUFFIX: test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688915911 2024-06-26T07:28:50.5355954Z ##[endgroup] 2024-06-26T07:28:50.5395370Z adding: usage_log.txt (deflated 92%) 2024-06-26T07:28:50.5461134Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-06-26T07:28:50.5461915Z # Remove any previous debugging artifacts if they exist 2024-06-26T07:28:50.5462531Z rm -f debug-*.zip 2024-06-26T07:28:50.5462943Z if [ -d 'test/debug' ]; then 2024-06-26T07:28:50.5463455Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-06-26T07:28:50.5463940Z fi 2024-06-26T07:28:50.5470834Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:28:50.5471348Z env: 2024-06-26T07:28:50.5471625Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:50.5472093Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:50.5472843Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:50.5473729Z FILE_SUFFIX: test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688915911 2024-06-26T07:28:50.5474366Z ##[endgroup] 2024-06-26T07:28:50.5548583Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T07:28:50.5549030Z with: 2024-06-26T07:28:50.5549315Z s3-bucket: gha-artifacts 2024-06-26T07:28:50.5549747Z s3-prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:28:50.5550231Z retention-days: 14 2024-06-26T07:28:50.5550562Z if-no-files-found: warn 2024-06-26T07:28:50.5550921Z path: test-jsons-*.zip 2024-06-26T07:28:50.5551269Z name: artifact 2024-06-26T07:28:50.5551566Z region: us-east-1 2024-06-26T07:28:50.5551870Z env: 2024-06-26T07:28:50.5552146Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:50.5552601Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:50.5553348Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:50.5554002Z ##[endgroup] 2024-06-26T07:28:50.8632329Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T07:28:50.8633196Z With the provided path, there will be 1 file uploaded 2024-06-26T07:28:50.8633951Z Uploading to s3 prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:28:50.8659977Z Starting upload of test-jsons-test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688915911.zip 2024-06-26T07:28:50.9912974Z Finished upload of test-jsons-test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688915911.zip 2024-06-26T07:28:51.0041350Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T07:28:51.0041806Z with: 2024-06-26T07:28:51.0042092Z s3-bucket: gha-artifacts 2024-06-26T07:28:51.0042523Z s3-prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:28:51.0042999Z retention-days: 14 2024-06-26T07:28:51.0043340Z if-no-files-found: error 2024-06-26T07:28:51.0043715Z path: test-reports-*.zip 2024-06-26T07:28:51.0044074Z name: artifact 2024-06-26T07:28:51.0044383Z region: us-east-1 2024-06-26T07:28:51.0044691Z env: 2024-06-26T07:28:51.0044970Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:51.0045438Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:51.0046197Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:51.0046850Z ##[endgroup] 2024-06-26T07:28:51.3144824Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T07:28:51.3146126Z With the provided path, there will be 1 file uploaded 2024-06-26T07:28:51.3146795Z Uploading to s3 prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:28:51.3172264Z Starting upload of test-reports-test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688915911.zip 2024-06-26T07:28:51.4248656Z Finished upload of test-reports-test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688915911.zip 2024-06-26T07:28:51.4380205Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T07:28:51.4380665Z with: 2024-06-26T07:28:51.4380970Z s3-bucket: gha-artifacts 2024-06-26T07:28:51.4381415Z s3-prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:28:51.4381911Z retention-days: 14 2024-06-26T07:28:51.4382400Z if-no-files-found: ignore 2024-06-26T07:28:51.4382777Z path: logs-*.zip 2024-06-26T07:28:51.4383111Z name: artifact 2024-06-26T07:28:51.4383428Z region: us-east-1 2024-06-26T07:28:51.4383743Z env: 2024-06-26T07:28:51.4384032Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:51.4384517Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:51.4385272Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:51.4385943Z ##[endgroup] 2024-06-26T07:28:51.7460816Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T07:28:51.7461428Z With the provided path, there will be 1 file uploaded 2024-06-26T07:28:51.7462049Z Uploading to s3 prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:28:51.7488272Z Starting upload of logs-test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688915911.zip 2024-06-26T07:28:51.8595340Z Finished upload of logs-test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688915911.zip 2024-06-26T07:28:51.8723364Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T07:28:51.8723814Z with: 2024-06-26T07:28:51.8724099Z s3-bucket: gha-artifacts 2024-06-26T07:28:51.8724524Z s3-prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:28:51.8724993Z retention-days: 14 2024-06-26T07:28:51.8725348Z if-no-files-found: ignore 2024-06-26T07:28:51.8725711Z path: debug-*.zip 2024-06-26T07:28:51.8738553Z name: artifact 2024-06-26T07:28:51.8738914Z region: us-east-1 2024-06-26T07:28:51.8739225Z env: 2024-06-26T07:28:51.8739498Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:51.8739963Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:51.8740718Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:51.8741365Z ##[endgroup] 2024-06-26T07:28:52.1720025Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-06-26T07:28:52.1923404Z ##[group]Run # shellcheck disable=SC2156 2024-06-26T07:28:52.1923975Z # shellcheck disable=SC2156 2024-06-26T07:28:52.1924797Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-06-26T07:28:52.1932852Z shell: /usr/bin/bash -e {0} 2024-06-26T07:28:52.1933211Z env: 2024-06-26T07:28:52.1933489Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:52.1933951Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:52.1934680Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:52.1935322Z ##[endgroup] 2024-06-26T07:28:52.3546808Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-06-26T07:28:52.3547439Z with: 2024-06-26T07:28:52.3547718Z env: 2024-06-26T07:28:52.3548002Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:52.3548459Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:52.3549207Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:52.3549872Z ##[endgroup] 2024-06-26T07:28:52.3567405Z ##[group]Run set -eou pipefail 2024-06-26T07:28:52.3567824Z set -eou pipefail 2024-06-26T07:28:52.3568185Z  2024-06-26T07:28:52.3568702Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-06-26T07:28:52.3569339Z for _ in $(seq 1440); do 2024-06-26T07:28:52.3569819Z  # Break if no ssh session exists anymore 2024-06-26T07:28:52.3570329Z  if [ "$(who)" = "" ]; then 2024-06-26T07:28:52.3570742Z  break 2024-06-26T07:28:52.3571095Z  fi 2024-06-26T07:28:52.3571392Z  echo "." 2024-06-26T07:28:52.3571730Z  sleep 5 2024-06-26T07:28:52.3572050Z done 2024-06-26T07:28:52.3579492Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:28:52.3579983Z env: 2024-06-26T07:28:52.3580264Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:52.3580722Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:52.3581622Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:52.3582269Z ##[endgroup] 2024-06-26T07:28:52.3603024Z Holding runner for 2 hours until all ssh sessions have logged out 2024-06-26T07:28:52.3639139Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:28:52.3639907Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:28:52.3640502Z # shellcheck disable=SC2046 2024-06-26T07:28:52.3640983Z docker stop $(docker ps -q) || true 2024-06-26T07:28:52.3641464Z # Prune all of the docker images 2024-06-26T07:28:52.3641916Z docker system prune -af 2024-06-26T07:28:52.3649135Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:28:52.3649642Z env: 2024-06-26T07:28:52.3649918Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:28:52.3650388Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:28:52.3651154Z DOCKER_CONTAINER_ID: a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:52.3651808Z ##[endgroup] 2024-06-26T07:28:52.6418695Z a3eb29f592f4 2024-06-26T07:28:53.3799701Z Deleted Containers: 2024-06-26T07:28:53.3800262Z a3eb29f592f4643d5e17ead2956736c4c77f5e6f1942cef2576ff1fdebca3628 2024-06-26T07:28:53.3800720Z 2024-06-26T07:28:58.2963350Z Deleted Images: 2024-06-26T07:28:58.2966078Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:28:58.2969491Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb 2024-06-26T07:28:58.2971110Z deleted: sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd 2024-06-26T07:28:58.2971958Z deleted: sha256:754ead7012f0b7691885a9660f02e80be6cee8c5665486fb995c1fb06adb6e92 2024-06-26T07:28:58.2972808Z deleted: sha256:fbe4496d8830e11be86d8f222522e7956f32f85bfc06b05ea53961f34a58af74 2024-06-26T07:28:58.2973656Z deleted: sha256:1c83814aa1a1a7e0cc998e3479bdab8e8766b975560cbf00690acf39ed899a80 2024-06-26T07:28:58.2974515Z deleted: sha256:e9ea8e61942c7f8ae4aab97b8d102b01d6bb4b6cf2a605808cd34130a17d8a0f 2024-06-26T07:28:58.2975365Z deleted: sha256:acc5fc32588cd95e082a6ce35e48d0f303ae8f96e3ab65c4e375b85e1f3c4a92 2024-06-26T07:28:58.2976195Z deleted: sha256:a93e53400904d29e4185d3be942ea34f19572ea438962788ae4e9b3d9bf4c498 2024-06-26T07:28:58.2977190Z deleted: sha256:c97092b88b5e4c77c68bbc4b51afab64e38719af1ff58013ec49a696cdd40a6e 2024-06-26T07:28:58.2978170Z deleted: sha256:b132889c90d67ffa53655846603e3fcc2372c23f04dbdbc54690a39d304eb89b 2024-06-26T07:28:58.2979064Z deleted: sha256:915d54081ed607710624abe3f2490114c745858831f4fc13396bd3b4b9a121bc 2024-06-26T07:28:58.2979900Z deleted: sha256:bfa8d446c498c87f535aef102695564de1b22c755e6c63944ae2f5073a2b31ec 2024-06-26T07:28:58.2980766Z deleted: sha256:e71460fb363c29e70bbe357d680aaa7ae694f094ef2f2713609158ad90611744 2024-06-26T07:28:58.2981622Z deleted: sha256:84079bb1b4ee1d0119ae3d3e701de460a1324912297c675425c9a36d30fcdb0f 2024-06-26T07:28:58.2982482Z deleted: sha256:d86ac3ca148d4758cb03b493fdd49109d8f1a2c7c85c4b755493c3c4529553e5 2024-06-26T07:28:58.2983335Z deleted: sha256:532b3ab97c44a673a0d5f62ee46fa6a83248a84de8877754a612355be56ed0a3 2024-06-26T07:28:58.2984188Z deleted: sha256:617e2a35f25ae24500c88c0c9b8475f0687fbcbed3d200013b4c99c6d8941aff 2024-06-26T07:28:58.2985237Z deleted: sha256:b88120bcd5a26a660c09c5df13176e0ef098fb2c8cdb57d74ab0796a3d1194d6 2024-06-26T07:28:58.2986283Z deleted: sha256:c016b2e7764735243205fb76fa9a273bb3dd4bf05532288e3a613ce022131ac3 2024-06-26T07:28:58.2987170Z deleted: sha256:bcaca462c390deddd2995de2d245a7495e7d2e1e94f0edad30a573c83d0588fd 2024-06-26T07:28:58.2988011Z deleted: sha256:9e260eba003d5c25d7b410e5c4c3093845d9e46611896840901c689d7fbf8ee6 2024-06-26T07:28:58.2988971Z deleted: sha256:3e62b86f8b911a9b59c62d55f063e71683ae6a01c22418db5d27930712adaad6 2024-06-26T07:28:58.2989810Z deleted: sha256:905920ccb1d88449b8dde686c34a4b2a8b6f59e78ac44916b189ea3593b9fbbb 2024-06-26T07:28:58.2990670Z deleted: sha256:961201237e03acd5061406238dd7a3c1343ac888d7583212cd0ab5ae1b4f0540 2024-06-26T07:28:58.2991495Z deleted: sha256:70047a2aac62159fa28fb4dbd4567ec2ffbab68fe046967e09acb0eb3e5cb9cb 2024-06-26T07:28:58.2992340Z deleted: sha256:1ec680ad781436ae65fa987f5fa1ed032ecd5c82a6e35eea6707de501f62c0f4 2024-06-26T07:28:58.2993194Z deleted: sha256:ef859623a50e85abf6e6c1dbd6188aa765a66cc7b2bd618a1cbe4825b7b2bd91 2024-06-26T07:28:58.2994043Z deleted: sha256:404ade5cacff59773b93ab438bfa64cd3a171d07f6f168bdc4a5d4bfa0b43526 2024-06-26T07:28:58.2994889Z deleted: sha256:5b27de6ee3d9f2f18217bf18dec725f7b4954c0e48c85d35752634c22d62209b 2024-06-26T07:28:58.2995736Z deleted: sha256:498e39c575b3fa069acaac7e46b91b4e54ae3a05f2ada5c4347091f346f25d4c 2024-06-26T07:28:58.2996554Z deleted: sha256:1405919d2694d65669b1f4c842282021afe89208ce86570e96464903733da2d0 2024-06-26T07:28:58.2997446Z deleted: sha256:d5b41e5d70896c0472fc3e6da0516a17d89cd6635dc75010ecddc31ce9690fad 2024-06-26T07:28:58.2998649Z deleted: sha256:18e9379b62b9d0bb40293d28a652eafaee66a499f66f0776bf14e37a513101b7 2024-06-26T07:28:58.2999522Z deleted: sha256:a781f80ed00ff2926a699fd01a309bb0b4fb7d5655eb652c205bb9c7f811035f 2024-06-26T07:28:58.3000383Z deleted: sha256:c0430973b7cf759075330ee29c79bfd1312c91e6033d9dd883b4b1c29743a171 2024-06-26T07:28:58.3001262Z deleted: sha256:e733b3d66efb9342e5a283376c0e1ddffa47a42fecc26e61ecaf0a3472885d97 2024-06-26T07:28:58.3002138Z deleted: sha256:8929c0b2047f74b84c82c39eff9077a92ae1a828cde00c9ab78fb3e0ed5e0283 2024-06-26T07:28:58.3002997Z deleted: sha256:f5b56155fb58a0a60fe6391d3581cf91610ed69d4f0a321257c19944da296a52 2024-06-26T07:28:58.3003860Z deleted: sha256:b54d2cb355238d1aa91e58df93064447b80f53706d1ca62823acc834f5ac54fe 2024-06-26T07:28:58.3004727Z deleted: sha256:e7311285af9eeddb51aa5629aa98386e89f1456ff32843b6aee6443247edf66c 2024-06-26T07:28:58.3005603Z deleted: sha256:67657ceec347da8dc8085d5731633efeac115ba677fa4b7c5966c4260f7abab0 2024-06-26T07:28:58.3006461Z deleted: sha256:b496e50409dbf414930fc9b4437aa0aa4ce27bc7840e050ae459aaffcf9aca86 2024-06-26T07:28:58.3007329Z deleted: sha256:8f5276170cc4a60d36ad7b964a502306b29546254cedcbcb101ea9123497d2c0 2024-06-26T07:28:58.3008176Z deleted: sha256:53d524d1564745e96a7e515462171b73184d9f1ff5454b32fcfbbca39d361467 2024-06-26T07:28:58.3009043Z deleted: sha256:2acbadb00d69b6aca575f04baba4ea26ae2b5c2d85f09945004ad93a78a8eb75 2024-06-26T07:28:58.3010046Z deleted: sha256:02665efeff878aeb603c1edb2e21cb700c171c025d64f5aae1b2b20b12f3964a 2024-06-26T07:28:58.3010943Z deleted: sha256:19bd23e9a51dfe883a8b8d2ff6d66ff1b3476b8912856406bb79b81762aa6d4b 2024-06-26T07:28:58.3011815Z deleted: sha256:73aa8637bf9299d6298dabd858480c274a62117388f2ca1d5e9eb15c6d1e4155 2024-06-26T07:28:58.3012673Z deleted: sha256:5f8968406305f348b1caebc0e035187d6e9965ace7b9d5f2f21114d37c9fe2e1 2024-06-26T07:28:58.3013542Z deleted: sha256:d827104914db0c503cf5c81f13e2456bf4efce42d4cf1f8d5d70838207278bee 2024-06-26T07:28:58.3014416Z deleted: sha256:867b219724f2f1aaf56f42e3e9f03a5d64bdca8a6a72407efd10328aeea5d510 2024-06-26T07:28:58.3015297Z deleted: sha256:b9ff2a7cceb9237b26bcb17728edf8a558eec5a76f63626e84f80541dfc0f1d4 2024-06-26T07:28:58.3016189Z deleted: sha256:be73f509106819007ce2b0f5a1301ac56ae583cb6c1fb69bee93efa0840368f1 2024-06-26T07:28:58.3017063Z deleted: sha256:2a11c07c915c3fdbb4e73096dea73ea950c970d50fe1a5764060bd62b7b7cf2c 2024-06-26T07:28:58.3018003Z deleted: sha256:5e9989622e2abc1c72f8184343507b45d75e4ff971edb68b96cbcd75ae484ff5 2024-06-26T07:28:58.3018913Z deleted: sha256:05ce642bd9cf5a9205bff107df28b1d6cef7d5f6ab3ea944d28c032ec61270df 2024-06-26T07:28:58.3019765Z deleted: sha256:017702617f6c07b73f2088678f4710b794cfe897a213a5cf122a0e2519eae0ac 2024-06-26T07:28:58.3020599Z deleted: sha256:44b3b37bb1ca39dc5cf356de41a23a5c12f605a1992813e408087fb2b0ad6d1c 2024-06-26T07:28:58.3021452Z deleted: sha256:8c387982e9f27db357a1d149dc149cabc0bd04668ad6956387a5072682dada67 2024-06-26T07:28:58.3022375Z deleted: sha256:10d415dca4e3fbb88a747b40317f3c6333c041ba8f73855f67fb9fe981fae5ee 2024-06-26T07:28:58.3023225Z deleted: sha256:6c65469bf0f53dcd625faefe363b8f3136c08388a6cf64d9158e8576e4ee89b1 2024-06-26T07:28:58.3024088Z deleted: sha256:027db5d8b76deeefbd8c34b485ab7ff14ed2a3e6800425c1ce0d77c2e9fdd4ab 2024-06-26T07:28:58.3024936Z deleted: sha256:24941234ad28cf5c8174d7cf18326c10b4e17f2c2137672ea0322df461597dd7 2024-06-26T07:28:58.3025786Z deleted: sha256:240ea2eebfa8519d6c2d88f4c99e66ce4e038d000f93ce8d03726c01d81180ef 2024-06-26T07:28:58.3026643Z deleted: sha256:09e823cb391a69f6a195e3ca8a41624b5f8b34eee14c6eb36277de814a91966e 2024-06-26T07:28:58.3027470Z deleted: sha256:730479d695169a6593bf0ac16b377491dce65175a6d158e012422367d04577fd 2024-06-26T07:28:58.3028281Z deleted: sha256:b0a30f9f7bd5459ff662453b9346a7ba8ea8db1191528345c1953eab4aa2f54e 2024-06-26T07:28:58.3029176Z deleted: sha256:75cefa87effaa3ad361c74642839923101f0f9982f814f3c9e2ed4ad52f2a008 2024-06-26T07:28:58.3030031Z deleted: sha256:6f97db87ba9512e615e871c6982000602982e1e5a5be334a56e7292251e84e00 2024-06-26T07:28:58.3030876Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-06-26T07:28:58.3031717Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-06-26T07:28:58.3032545Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-06-26T07:28:58.3033383Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-06-26T07:28:58.3034240Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-06-26T07:28:58.3035089Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-06-26T07:28:58.3035933Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-06-26T07:28:58.3036802Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-06-26T07:28:58.3037678Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-06-26T07:28:58.3038762Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-06-26T07:28:58.3039604Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-06-26T07:28:58.3040129Z 2024-06-26T07:28:58.3040280Z Total reclaimed space: 36.35GB 2024-06-26T07:28:58.3105530Z Post job cleanup. 2024-06-26T07:28:58.3153979Z Post job cleanup. 2024-06-26T07:28:58.3968815Z [command]/usr/bin/git version 2024-06-26T07:28:58.4009614Z git version 2.40.1 2024-06-26T07:28:58.4049360Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/f55a7d93-ee95-4c84-9c2f-23ac3cfc06aa' before making global git config changes 2024-06-26T07:28:58.4050729Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T07:28:58.4055122Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:28:58.4087737Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T07:28:58.4110072Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T07:28:58.4328582Z Entering 'android/libs/fbjni' 2024-06-26T07:28:58.4363885Z Entering 'third_party/FP16' 2024-06-26T07:28:58.4399864Z Entering 'third_party/FXdiv' 2024-06-26T07:28:58.4435850Z Entering 'third_party/NNPACK' 2024-06-26T07:28:58.4471297Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:28:58.4507609Z Entering 'third_party/XNNPACK' 2024-06-26T07:28:58.4553837Z Entering 'third_party/benchmark' 2024-06-26T07:28:58.4590952Z Entering 'third_party/cpp-httplib' 2024-06-26T07:28:58.4624643Z Entering 'third_party/cpuinfo' 2024-06-26T07:28:58.4659733Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:28:58.4696097Z Entering 'third_party/cutlass' 2024-06-26T07:28:58.4739084Z Entering 'third_party/eigen' 2024-06-26T07:28:58.4776328Z Entering 'third_party/fbgemm' 2024-06-26T07:28:58.4811157Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:28:58.4845133Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:28:58.4880714Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:28:58.4920206Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:28:58.4953359Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:28:58.4989302Z Entering 'third_party/flatbuffers' 2024-06-26T07:28:58.5026911Z Entering 'third_party/fmt' 2024-06-26T07:28:58.5062976Z Entering 'third_party/foxi' 2024-06-26T07:28:58.5098342Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:28:58.5134880Z Entering 'third_party/gloo' 2024-06-26T07:28:58.5171799Z Entering 'third_party/googletest' 2024-06-26T07:28:58.5209482Z Entering 'third_party/ideep' 2024-06-26T07:28:58.5245187Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:28:58.5283360Z Entering 'third_party/ittapi' 2024-06-26T07:28:58.5319363Z Entering 'third_party/kineto' 2024-06-26T07:28:58.5354582Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:28:58.5389063Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:28:58.5425779Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:28:58.5459778Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:28:58.5495556Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:28:58.5529407Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:28:58.5564843Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:28:58.5598203Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:28:58.5634155Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:28:58.5667755Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:28:58.5704385Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:28:58.5738877Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:28:58.5774488Z Entering 'third_party/mimalloc' 2024-06-26T07:28:58.5809206Z Entering 'third_party/nccl/nccl' 2024-06-26T07:28:58.5845503Z Entering 'third_party/nlohmann' 2024-06-26T07:28:58.5881203Z Entering 'third_party/onnx' 2024-06-26T07:28:58.5927453Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:28:58.5966894Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:28:58.6005506Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:28:58.6042231Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:28:58.6075935Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:28:58.6110338Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:28:58.6146589Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:28:58.6180883Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:28:58.6214636Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:28:58.6250685Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:28:58.6282737Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:28:58.6317879Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:28:58.6354834Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:28:58.6405741Z Entering 'third_party/pocketfft' 2024-06-26T07:28:58.6443590Z Entering 'third_party/protobuf' 2024-06-26T07:28:58.6479913Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:28:58.6513434Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:28:58.6549942Z Entering 'third_party/psimd' 2024-06-26T07:28:58.6585603Z Entering 'third_party/pthreadpool' 2024-06-26T07:28:58.6622102Z Entering 'third_party/pybind11' 2024-06-26T07:28:58.6656672Z Entering 'third_party/python-peachpy' 2024-06-26T07:28:58.6704628Z Entering 'third_party/sleef' 2024-06-26T07:28:58.6726372Z Entering 'third_party/tensorpipe' 2024-06-26T07:28:58.6760207Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:28:58.6794779Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:28:58.6827921Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:28:58.6862957Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:28:58.6895639Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:28:58.6948483Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T07:28:58.6970054Z http.https://github.com/.extraheader 2024-06-26T07:28:58.6978984Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-06-26T07:28:58.7007101Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T07:28:58.7214536Z Entering 'android/libs/fbjni' 2024-06-26T07:28:58.7235576Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7260588Z Entering 'third_party/FP16' 2024-06-26T07:28:58.7282043Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7305435Z Entering 'third_party/FXdiv' 2024-06-26T07:28:58.7326790Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7352451Z Entering 'third_party/NNPACK' 2024-06-26T07:28:58.7374934Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7398851Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:28:58.7420763Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7445413Z Entering 'third_party/XNNPACK' 2024-06-26T07:28:58.7467626Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7502971Z Entering 'third_party/benchmark' 2024-06-26T07:28:58.7524136Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7547322Z Entering 'third_party/cpp-httplib' 2024-06-26T07:28:58.7569105Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7591988Z Entering 'third_party/cpuinfo' 2024-06-26T07:28:58.7615059Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7637607Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:28:58.7660204Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7683640Z Entering 'third_party/cutlass' 2024-06-26T07:28:58.7705365Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7733108Z Entering 'third_party/eigen' 2024-06-26T07:28:58.7756045Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7781838Z Entering 'third_party/fbgemm' 2024-06-26T07:28:58.7804036Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7828048Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:28:58.7850023Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7873132Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:28:58.7895210Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7918540Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:28:58.7939527Z http.https://github.com/.extraheader 2024-06-26T07:28:58.7967083Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:28:58.7987451Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8013107Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:28:58.8033480Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8057095Z Entering 'third_party/flatbuffers' 2024-06-26T07:28:58.8081316Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8106911Z Entering 'third_party/fmt' 2024-06-26T07:28:58.8127745Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8148729Z Entering 'third_party/foxi' 2024-06-26T07:28:58.8170505Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8193490Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:28:58.8214443Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8238310Z Entering 'third_party/gloo' 2024-06-26T07:28:58.8260938Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8284958Z Entering 'third_party/googletest' 2024-06-26T07:28:58.8306744Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8329604Z Entering 'third_party/ideep' 2024-06-26T07:28:58.8349829Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8371954Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:28:58.8392454Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8420207Z Entering 'third_party/ittapi' 2024-06-26T07:28:58.8442133Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8464063Z Entering 'third_party/kineto' 2024-06-26T07:28:58.8486429Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8509517Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:28:58.8531983Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8555704Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:28:58.8576774Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8601065Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:28:58.8622831Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8647027Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:28:58.8669378Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8692890Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:28:58.8713306Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8735476Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:28:58.8757221Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8782423Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:28:58.8804115Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8827614Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:28:58.8848584Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8873673Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:28:58.8894550Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8916429Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:28:58.8938474Z http.https://github.com/.extraheader 2024-06-26T07:28:58.8963196Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:28:58.8984962Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9008203Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:28:58.9029906Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9055916Z Entering 'third_party/mimalloc' 2024-06-26T07:28:58.9076883Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9099824Z Entering 'third_party/nccl/nccl' 2024-06-26T07:28:58.9121951Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9144271Z Entering 'third_party/nlohmann' 2024-06-26T07:28:58.9166850Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9192092Z Entering 'third_party/onnx' 2024-06-26T07:28:58.9214166Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9250024Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:28:58.9271718Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9295871Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:28:58.9317618Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9343065Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:28:58.9365391Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9390158Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:28:58.9411825Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9435918Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:28:58.9458136Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9482785Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:28:58.9505517Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9528321Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:28:58.9549550Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9574946Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:28:58.9596386Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9618871Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:28:58.9640238Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9664653Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:28:58.9686355Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9709379Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:28:58.9730541Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9755725Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:28:58.9777087Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9803033Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:28:58.9824447Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9862094Z Entering 'third_party/pocketfft' 2024-06-26T07:28:58.9883387Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9904453Z Entering 'third_party/protobuf' 2024-06-26T07:28:58.9927185Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9952631Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:28:58.9974437Z http.https://github.com/.extraheader 2024-06-26T07:28:58.9997759Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:28:59.0020049Z http.https://github.com/.extraheader 2024-06-26T07:28:59.0046743Z Entering 'third_party/psimd' 2024-06-26T07:28:59.0069133Z http.https://github.com/.extraheader 2024-06-26T07:28:59.0093709Z Entering 'third_party/pthreadpool' 2024-06-26T07:28:59.0116146Z http.https://github.com/.extraheader 2024-06-26T07:28:59.0138680Z Entering 'third_party/pybind11' 2024-06-26T07:28:59.0159894Z http.https://github.com/.extraheader 2024-06-26T07:28:59.0182724Z Entering 'third_party/python-peachpy' 2024-06-26T07:28:59.0204296Z http.https://github.com/.extraheader 2024-06-26T07:28:59.0226391Z Entering 'third_party/sleef' 2024-06-26T07:28:59.0248213Z http.https://github.com/.extraheader 2024-06-26T07:28:59.0270453Z Entering 'third_party/tensorpipe' 2024-06-26T07:28:59.0292917Z http.https://github.com/.extraheader 2024-06-26T07:28:59.0314938Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:28:59.0337159Z http.https://github.com/.extraheader 2024-06-26T07:28:59.0361414Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:28:59.0383099Z http.https://github.com/.extraheader 2024-06-26T07:28:59.0404833Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:28:59.0424726Z http.https://github.com/.extraheader 2024-06-26T07:28:59.0446109Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:28:59.0467181Z http.https://github.com/.extraheader 2024-06-26T07:28:59.0489044Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:28:59.0510691Z http.https://github.com/.extraheader 2024-06-26T07:28:59.0696807Z A job completed hook has been configured by the self-hosted runner administrator 2024-06-26T07:28:59.0715946Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T07:28:59.0722928Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:28:59.0723466Z ##[endgroup] 2024-06-26T07:28:59.6555536Z Cleaning up orphan processes