2024-06-26T07:04:13.5589683Z Current runner version: '2.317.0' 2024-06-26T07:04:13.5595835Z Runner name: 'i-08ac67d1e0a1089db' 2024-06-26T07:04:13.5596597Z Runner group name: 'Default' 2024-06-26T07:04:13.5597490Z Machine name: 'ip-10-0-35-135' 2024-06-26T07:04:13.5602372Z ##[group]GITHUB_TOKEN Permissions 2024-06-26T07:04:13.5604278Z Actions: read 2024-06-26T07:04:13.5604798Z Attestations: read 2024-06-26T07:04:13.5605317Z Checks: read 2024-06-26T07:04:13.5605806Z Contents: read 2024-06-26T07:04:13.5606282Z Deployments: read 2024-06-26T07:04:13.5606796Z Discussions: read 2024-06-26T07:04:13.5607259Z Issues: read 2024-06-26T07:04:13.5607712Z Metadata: read 2024-06-26T07:04:13.5608233Z Packages: read 2024-06-26T07:04:13.5608705Z Pages: read 2024-06-26T07:04:13.5609148Z PullRequests: read 2024-06-26T07:04:13.5609687Z RepositoryProjects: read 2024-06-26T07:04:13.5610263Z SecurityEvents: read 2024-06-26T07:04:13.5610838Z Statuses: read 2024-06-26T07:04:13.5611365Z ##[endgroup] 2024-06-26T07:04:13.5614912Z Secret source: Actions 2024-06-26T07:04:13.5615705Z Prepare workflow directory 2024-06-26T07:04:13.9188725Z Prepare all required actions 2024-06-26T07:04:13.9364053Z Getting action download info 2024-06-26T07:04:14.1152375Z Download action repository 'pytorch/test-infra@main' (SHA:43a2ce341cc31288e9a38b65ce600a7f43021bd5) 2024-06-26T07:04:14.4673486Z Download action repository 'pytorch/pytorch@main' (SHA:c422a9549d2b320b47a9391f6cbcf592b2876c88) 2024-06-26T07:04:17.2797488Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-06-26T07:04:17.4194584Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-06-26T07:04:17.6828339Z Getting action download info 2024-06-26T07:04:17.8071796Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-06-26T07:04:17.9855803Z Getting action download info 2024-06-26T07:04:18.0797987Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-06-26T07:04:18.2143704Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/129470 (b8c4c54d347aa776934c60784e35936878ef18dc) 2024-06-26T07:04:18.2145874Z ##[group] Inputs 2024-06-26T07:04:18.2146347Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:04:18.2153468Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-06-26T07:04:18.2161387Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:18.2162586Z sync-tag: 2024-06-26T07:04:18.2163415Z timeout-minutes: 240 2024-06-26T07:04:18.2163776Z use-gha: 2024-06-26T07:04:18.2164068Z dashboard-tag: 2024-06-26T07:04:18.2164397Z s3-bucket: gha-artifacts 2024-06-26T07:04:18.2164758Z aws-role-to-assume: 2024-06-26T07:04:18.2165105Z ##[endgroup] 2024-06-26T07:04:18.2166139Z Complete job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:04:18.2725113Z A job started hook has been configured by the self-hosted runner administrator 2024-06-26T07:04:18.2862424Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T07:04:18.2872762Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:18.2873333Z ##[endgroup] 2024-06-26T07:04:19.2788578Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-06-26T07:04:19.2789123Z with: 2024-06-26T07:04:19.2789833Z github-secret: *** 2024-06-26T07:04:19.2790781Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-06-26T07:04:19.2791783Z activate-with-label: false 2024-06-26T07:04:19.2792149Z label: with-ssh 2024-06-26T07:04:19.2792470Z remove-existing-keys: true 2024-06-26T07:04:19.2792823Z fail-silently: true 2024-06-26T07:04:19.2793139Z env: 2024-06-26T07:04:19.2793410Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:19.2793758Z ##[endgroup] 2024-06-26T07:04:19.3667717Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-06-26T07:04:19.3668746Z ciflow reference detected, attempting to extract PR number 2024-06-26T07:04:19.6135680Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-06-26T07:04:19.6775930Z No SSH keys found for user pytorch-bot[bot] 2024-06-26T07:04:19.6777021Z Grabbing public ssh keys from https://github.com/leslie-fang-intel.keys 2024-06-26T07:04:19.7503372Z No SSH keys found for user leslie-fang-intel 2024-06-26T07:04:19.7610004Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-06-26T07:04:19.7610549Z with: 2024-06-26T07:04:19.7610957Z submodules: recursive 2024-06-26T07:04:19.7611284Z fetch-depth: 0 2024-06-26T07:04:19.7611571Z env: 2024-06-26T07:04:19.7611835Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:19.7612441Z ##[endgroup] 2024-06-26T07:04:19.7804077Z ##[group]Run retry () { 2024-06-26T07:04:19.7804630Z retry () { 2024-06-26T07:04:19.7805205Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-06-26T07:04:19.7805836Z } 2024-06-26T07:04:19.7806195Z echo "${GITHUB_WORKSPACE}" 2024-06-26T07:04:19.7806664Z if [ -z "${NO_SUDO}" ]; then 2024-06-26T07:04:19.7807280Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T07:04:19.7807792Z else 2024-06-26T07:04:19.7808169Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T07:04:19.7808669Z fi 2024-06-26T07:04:19.7809095Z mkdir "${GITHUB_WORKSPACE}" 2024-06-26T07:04:19.7817539Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:19.7818114Z env: 2024-06-26T07:04:19.7818476Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:19.7818877Z NO_SUDO: 2024-06-26T07:04:19.7819182Z ##[endgroup] 2024-06-26T07:04:19.7871881Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:04:22.0384817Z ##[group]Run malfet/checkout@silent-checkout 2024-06-26T07:04:22.0385252Z with: 2024-06-26T07:04:22.0385566Z ref: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:04:22.0386002Z fetch-depth: 0 2024-06-26T07:04:22.0386314Z submodules: recursive 2024-06-26T07:04:22.0386655Z quiet-checkout: true 2024-06-26T07:04:22.0387124Z repository: pytorch/pytorch 2024-06-26T07:04:22.0387601Z token: *** 2024-06-26T07:04:22.0387888Z ssh-strict: true 2024-06-26T07:04:22.0388208Z persist-credentials: true 2024-06-26T07:04:22.0388552Z clean: true 2024-06-26T07:04:22.0388875Z sparse-checkout-cone-mode: true 2024-06-26T07:04:22.0389262Z lfs: false 2024-06-26T07:04:22.0389553Z set-safe-directory: true 2024-06-26T07:04:22.0389886Z env: 2024-06-26T07:04:22.0390151Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:22.0390487Z ##[endgroup] 2024-06-26T07:04:22.1320209Z Syncing repository: pytorch/pytorch 2024-06-26T07:04:22.1322039Z ##[group]Getting Git version info 2024-06-26T07:04:22.1322813Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T07:04:22.1323670Z [command]/usr/bin/git version 2024-06-26T07:04:22.1324039Z git version 2.40.1 2024-06-26T07:04:22.1325479Z ##[endgroup] 2024-06-26T07:04:22.1337114Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/344f6484-5274-4134-886f-6057c1453219/.gitconfig' 2024-06-26T07:04:22.1338720Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/344f6484-5274-4134-886f-6057c1453219' before making global git config changes 2024-06-26T07:04:22.1339954Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T07:04:22.1341072Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:04:22.1371089Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T07:04:22.1375675Z ##[group]Initializing the repository 2024-06-26T07:04:22.1378540Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:04:22.1399509Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-06-26T07:04:22.1400375Z hint: is subject to change. To configure the initial branch name to use in all 2024-06-26T07:04:22.1401171Z hint: of your new repositories, which will suppress this warning, call: 2024-06-26T07:04:22.1401941Z hint: 2024-06-26T07:04:22.1402376Z hint: git config --global init.defaultBranch 2024-06-26T07:04:22.1402842Z hint: 2024-06-26T07:04:22.1403522Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-06-26T07:04:22.1404369Z hint: 'development'. The just-created branch can be renamed via this command: 2024-06-26T07:04:22.1404979Z hint: 2024-06-26T07:04:22.1405321Z hint: git branch -m 2024-06-26T07:04:22.1406050Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-06-26T07:04:22.1409188Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-06-26T07:04:22.1431279Z ##[endgroup] 2024-06-26T07:04:22.1431859Z ##[group]Disabling automatic garbage collection 2024-06-26T07:04:22.1433998Z [command]/usr/bin/git config --local gc.auto 0 2024-06-26T07:04:22.1459570Z ##[endgroup] 2024-06-26T07:04:22.1460136Z ##[group]Setting up auth 2024-06-26T07:04:22.1465799Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T07:04:22.1491162Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T07:04:22.1683010Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T07:04:22.1708978Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T07:04:22.1903339Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:04:22.1940863Z ##[endgroup] 2024-06-26T07:04:22.1941413Z ##[group]Fetching the repository 2024-06-26T07:04:22.1946960Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-06-26T07:04:24.4999471Z remote: Enumerating objects: 987619 2024-06-26T07:04:24.5105731Z remote: Enumerating objects: 987646 2024-06-26T07:04:24.5106388Z remote: Enumerating objects: 990209, done. 2024-06-26T07:04:24.5113411Z remote: Counting objects: 0% (1/2590) 2024-06-26T07:04:24.5114723Z remote: Counting objects: 1% (26/2590) 2024-06-26T07:04:24.5116020Z remote: Counting objects: 2% (52/2590) 2024-06-26T07:04:24.5116898Z remote: Counting objects: 3% (78/2590) 2024-06-26T07:04:24.5117407Z remote: Counting objects: 4% (104/2590) 2024-06-26T07:04:24.5117921Z remote: Counting objects: 5% (130/2590) 2024-06-26T07:04:24.5118431Z remote: Counting objects: 6% (156/2590) 2024-06-26T07:04:24.5118931Z remote: Counting objects: 7% (182/2590) 2024-06-26T07:04:24.5119424Z remote: Counting objects: 8% (208/2590) 2024-06-26T07:04:24.5119978Z remote: Counting objects: 9% (234/2590) 2024-06-26T07:04:24.5121667Z remote: Counting objects: 10% (259/2590) 2024-06-26T07:04:24.5122312Z remote: Counting objects: 11% (285/2590) 2024-06-26T07:04:24.5122821Z remote: Counting objects: 12% (311/2590) 2024-06-26T07:04:24.5123323Z remote: Counting objects: 13% (337/2590) 2024-06-26T07:04:24.5129528Z remote: Counting objects: 14% (363/2590) 2024-06-26T07:04:24.5130149Z remote: Counting objects: 15% (389/2590) 2024-06-26T07:04:24.5130737Z remote: Counting objects: 16% (415/2590) 2024-06-26T07:04:24.5131243Z remote: Counting objects: 17% (441/2590) 2024-06-26T07:04:24.5131749Z remote: Counting objects: 18% (467/2590) 2024-06-26T07:04:24.5132418Z remote: Counting objects: 19% (493/2590) 2024-06-26T07:04:24.5132921Z remote: Counting objects: 20% (518/2590) 2024-06-26T07:04:24.5133410Z remote: Counting objects: 21% (544/2590) 2024-06-26T07:04:24.5137487Z remote: Counting objects: 22% (570/2590) 2024-06-26T07:04:24.5138117Z remote: Counting objects: 23% (596/2590) 2024-06-26T07:04:24.5138716Z remote: Counting objects: 24% (622/2590) 2024-06-26T07:04:24.5139415Z remote: Counting objects: 25% (648/2590) 2024-06-26T07:04:24.5139927Z remote: Counting objects: 26% (674/2590) 2024-06-26T07:04:24.5140428Z remote: Counting objects: 27% (700/2590) 2024-06-26T07:04:24.5140922Z remote: Counting objects: 28% (726/2590) 2024-06-26T07:04:24.5141409Z remote: Counting objects: 29% (752/2590) 2024-06-26T07:04:24.5141911Z remote: Counting objects: 30% (777/2590) 2024-06-26T07:04:24.5142489Z remote: Counting objects: 31% (803/2590) 2024-06-26T07:04:24.5143151Z remote: Counting objects: 32% (829/2590) 2024-06-26T07:04:24.5143652Z remote: Counting objects: 33% (855/2590) 2024-06-26T07:04:24.5144206Z remote: Counting objects: 34% (881/2590) 2024-06-26T07:04:24.5144899Z remote: Counting objects: 35% (907/2590) 2024-06-26T07:04:24.5145410Z remote: Counting objects: 36% (933/2590) 2024-06-26T07:04:24.5145909Z remote: Counting objects: 37% (959/2590) 2024-06-26T07:04:24.5146420Z remote: Counting objects: 38% (985/2590) 2024-06-26T07:04:24.5146983Z remote: Counting objects: 39% (1011/2590) 2024-06-26T07:04:24.5147514Z remote: Counting objects: 40% (1036/2590) 2024-06-26T07:04:24.5148042Z remote: Counting objects: 41% (1062/2590) 2024-06-26T07:04:24.5148571Z remote: Counting objects: 42% (1088/2590) 2024-06-26T07:04:24.5149098Z remote: Counting objects: 43% (1114/2590) 2024-06-26T07:04:24.5149620Z remote: Counting objects: 44% (1140/2590) 2024-06-26T07:04:24.5150135Z remote: Counting objects: 45% (1166/2590) 2024-06-26T07:04:24.5150658Z remote: Counting objects: 46% (1192/2590) 2024-06-26T07:04:24.5155250Z remote: Counting objects: 47% (1218/2590) 2024-06-26T07:04:24.5156270Z remote: Counting objects: 48% (1244/2590) 2024-06-26T07:04:24.5156991Z remote: Counting objects: 49% (1270/2590) 2024-06-26T07:04:24.5157538Z remote: Counting objects: 50% (1295/2590) 2024-06-26T07:04:24.5158183Z remote: Counting objects: 51% (1321/2590) 2024-06-26T07:04:24.5158930Z remote: Counting objects: 52% (1347/2590) 2024-06-26T07:04:24.5159606Z remote: Counting objects: 53% (1373/2590) 2024-06-26T07:04:24.5160294Z remote: Counting objects: 54% (1399/2590) 2024-06-26T07:04:24.5160933Z remote: Counting objects: 55% (1425/2590) 2024-06-26T07:04:24.5161660Z remote: Counting objects: 56% (1451/2590) 2024-06-26T07:04:24.5162177Z remote: Counting objects: 57% (1477/2590) 2024-06-26T07:04:24.5162694Z remote: Counting objects: 58% (1503/2590) 2024-06-26T07:04:24.5163209Z remote: Counting objects: 59% (1529/2590) 2024-06-26T07:04:24.5163877Z remote: Counting objects: 60% (1554/2590) 2024-06-26T07:04:24.5164468Z remote: Counting objects: 61% (1580/2590) 2024-06-26T07:04:24.5164984Z remote: Counting objects: 62% (1606/2590) 2024-06-26T07:04:24.5165518Z remote: Counting objects: 63% (1632/2590) 2024-06-26T07:04:24.5166031Z remote: Counting objects: 64% (1658/2590) 2024-06-26T07:04:24.5166548Z remote: Counting objects: 65% (1684/2590) 2024-06-26T07:04:24.5167056Z remote: Counting objects: 66% (1710/2590) 2024-06-26T07:04:24.5167570Z remote: Counting objects: 67% (1736/2590) 2024-06-26T07:04:24.5168082Z remote: Counting objects: 68% (1762/2590) 2024-06-26T07:04:24.5168595Z remote: Counting objects: 69% (1788/2590) 2024-06-26T07:04:24.5169102Z remote: Counting objects: 70% (1813/2590) 2024-06-26T07:04:24.5169613Z remote: Counting objects: 71% (1839/2590) 2024-06-26T07:04:24.5170123Z remote: Counting objects: 72% (1865/2590) 2024-06-26T07:04:24.5170760Z remote: Counting objects: 73% (1891/2590) 2024-06-26T07:04:24.5171272Z remote: Counting objects: 74% (1917/2590) 2024-06-26T07:04:24.5171790Z remote: Counting objects: 75% (1943/2590) 2024-06-26T07:04:24.5172596Z remote: Counting objects: 76% (1969/2590) 2024-06-26T07:04:24.5173122Z remote: Counting objects: 77% (1995/2590) 2024-06-26T07:04:24.5173631Z remote: Counting objects: 78% (2021/2590) 2024-06-26T07:04:24.5174145Z remote: Counting objects: 79% (2047/2590) 2024-06-26T07:04:24.5174662Z remote: Counting objects: 80% (2072/2590) 2024-06-26T07:04:24.5175174Z remote: Counting objects: 81% (2098/2590) 2024-06-26T07:04:24.5175681Z remote: Counting objects: 82% (2124/2590) 2024-06-26T07:04:24.5176194Z remote: Counting objects: 83% (2150/2590) 2024-06-26T07:04:24.5176769Z remote: Counting objects: 84% (2176/2590) 2024-06-26T07:04:24.5177287Z remote: Counting objects: 85% (2202/2590) 2024-06-26T07:04:24.5177806Z remote: Counting objects: 86% (2228/2590) 2024-06-26T07:04:24.5178313Z remote: Counting objects: 87% (2254/2590) 2024-06-26T07:04:24.5178826Z remote: Counting objects: 88% (2280/2590) 2024-06-26T07:04:24.5179347Z remote: Counting objects: 89% (2306/2590) 2024-06-26T07:04:24.5179862Z remote: Counting objects: 90% (2331/2590) 2024-06-26T07:04:24.5180369Z remote: Counting objects: 91% (2357/2590) 2024-06-26T07:04:24.5180879Z remote: Counting objects: 92% (2383/2590) 2024-06-26T07:04:24.5181389Z remote: Counting objects: 93% (2409/2590) 2024-06-26T07:04:24.5181895Z remote: Counting objects: 94% (2435/2590) 2024-06-26T07:04:24.5182412Z remote: Counting objects: 95% (2461/2590) 2024-06-26T07:04:24.5182925Z remote: Counting objects: 96% (2487/2590) 2024-06-26T07:04:24.5183437Z remote: Counting objects: 97% (2513/2590) 2024-06-26T07:04:24.5183948Z remote: Counting objects: 98% (2539/2590) 2024-06-26T07:04:24.5184536Z remote: Counting objects: 99% (2565/2590) 2024-06-26T07:04:24.5185051Z remote: Counting objects: 100% (2590/2590) 2024-06-26T07:04:24.5185606Z remote: Counting objects: 100% (2590/2590), done. 2024-06-26T07:04:24.5301840Z remote: Compressing objects: 0% (1/1068) 2024-06-26T07:04:24.5718045Z remote: Compressing objects: 1% (11/1068) 2024-06-26T07:04:24.6153345Z remote: Compressing objects: 2% (22/1068) 2024-06-26T07:04:24.6527715Z remote: Compressing objects: 3% (33/1068) 2024-06-26T07:04:24.6755773Z remote: Compressing objects: 4% (43/1068) 2024-06-26T07:04:24.7035725Z remote: Compressing objects: 5% (54/1068) 2024-06-26T07:04:24.7231371Z remote: Compressing objects: 6% (65/1068) 2024-06-26T07:04:24.7618524Z remote: Compressing objects: 7% (75/1068) 2024-06-26T07:04:24.8487930Z remote: Compressing objects: 8% (86/1068) 2024-06-26T07:04:24.9260399Z remote: Compressing objects: 9% (97/1068) 2024-06-26T07:04:24.9511741Z remote: Compressing objects: 10% (107/1068) 2024-06-26T07:04:24.9691629Z remote: Compressing objects: 11% (118/1068) 2024-06-26T07:04:24.9794076Z remote: Compressing objects: 12% (129/1068) 2024-06-26T07:04:24.9794626Z remote: Compressing objects: 13% (139/1068) 2024-06-26T07:04:24.9918462Z remote: Compressing objects: 14% (150/1068) 2024-06-26T07:04:24.9919000Z remote: Compressing objects: 15% (161/1068) 2024-06-26T07:04:24.9981057Z remote: Compressing objects: 16% (171/1068) 2024-06-26T07:04:24.9981698Z remote: Compressing objects: 17% (182/1068) 2024-06-26T07:04:24.9982243Z remote: Compressing objects: 18% (193/1068) 2024-06-26T07:04:24.9995943Z remote: Compressing objects: 19% (203/1068) 2024-06-26T07:04:25.0018419Z remote: Compressing objects: 20% (214/1068) 2024-06-26T07:04:25.0033083Z remote: Compressing objects: 21% (225/1068) 2024-06-26T07:04:25.0049340Z remote: Compressing objects: 22% (235/1068) 2024-06-26T07:04:25.0081869Z remote: Compressing objects: 23% (246/1068) 2024-06-26T07:04:25.0091922Z remote: Compressing objects: 24% (257/1068) 2024-06-26T07:04:25.0094998Z remote: Compressing objects: 25% (267/1068) 2024-06-26T07:04:25.0099701Z remote: Compressing objects: 26% (278/1068) 2024-06-26T07:04:25.0105416Z remote: Compressing objects: 27% (289/1068) 2024-06-26T07:04:25.0115331Z remote: Compressing objects: 28% (300/1068) 2024-06-26T07:04:25.0120222Z remote: Compressing objects: 29% (310/1068) 2024-06-26T07:04:25.0125884Z remote: Compressing objects: 30% (321/1068) 2024-06-26T07:04:25.0130501Z remote: Compressing objects: 31% (332/1068) 2024-06-26T07:04:25.0137178Z remote: Compressing objects: 32% (342/1068) 2024-06-26T07:04:25.0145943Z remote: Compressing objects: 33% (353/1068) 2024-06-26T07:04:25.0159964Z remote: Compressing objects: 34% (364/1068) 2024-06-26T07:04:25.0175024Z remote: Compressing objects: 35% (374/1068) 2024-06-26T07:04:25.0195489Z remote: Compressing objects: 36% (385/1068) 2024-06-26T07:04:25.0209675Z remote: Compressing objects: 37% (396/1068) 2024-06-26T07:04:25.0230878Z remote: Compressing objects: 38% (406/1068) 2024-06-26T07:04:25.0241057Z remote: Compressing objects: 39% (417/1068) 2024-06-26T07:04:25.0255421Z remote: Compressing objects: 40% (428/1068) 2024-06-26T07:04:25.0275558Z remote: Compressing objects: 41% (438/1068) 2024-06-26T07:04:25.0290330Z remote: Compressing objects: 42% (449/1068) 2024-06-26T07:04:25.0305379Z remote: Compressing objects: 43% (460/1068) 2024-06-26T07:04:25.0310299Z remote: Compressing objects: 44% (470/1068) 2024-06-26T07:04:25.0314762Z remote: Compressing objects: 45% (481/1068) 2024-06-26T07:04:25.0321981Z remote: Compressing objects: 46% (492/1068) 2024-06-26T07:04:25.0344696Z remote: Compressing objects: 47% (502/1068) 2024-06-26T07:04:25.0352092Z remote: Compressing objects: 48% (513/1068) 2024-06-26T07:04:25.0358492Z remote: Compressing objects: 49% (524/1068) 2024-06-26T07:04:25.0368045Z remote: Compressing objects: 50% (534/1068) 2024-06-26T07:04:25.0382102Z remote: Compressing objects: 51% (545/1068) 2024-06-26T07:04:25.0396725Z remote: Compressing objects: 52% (556/1068) 2024-06-26T07:04:25.0405317Z remote: Compressing objects: 53% (567/1068) 2024-06-26T07:04:25.0420329Z remote: Compressing objects: 54% (577/1068) 2024-06-26T07:04:25.0428624Z remote: Compressing objects: 55% (588/1068) 2024-06-26T07:04:25.0440112Z remote: Compressing objects: 56% (599/1068) 2024-06-26T07:04:25.0455265Z remote: Compressing objects: 57% (609/1068) 2024-06-26T07:04:25.0472089Z remote: Compressing objects: 58% (620/1068) 2024-06-26T07:04:25.0476173Z remote: Compressing objects: 59% (631/1068) 2024-06-26T07:04:25.0487297Z remote: Compressing objects: 60% (641/1068) 2024-06-26T07:04:25.0499939Z remote: Compressing objects: 61% (652/1068) 2024-06-26T07:04:25.0508316Z remote: Compressing objects: 62% (663/1068) 2024-06-26T07:04:25.0517298Z remote: Compressing objects: 63% (673/1068) 2024-06-26T07:04:25.0525426Z remote: Compressing objects: 64% (684/1068) 2024-06-26T07:04:25.0534212Z remote: Compressing objects: 65% (695/1068) 2024-06-26T07:04:25.0536860Z remote: Compressing objects: 66% (705/1068) 2024-06-26T07:04:25.0545465Z remote: Compressing objects: 67% (716/1068) 2024-06-26T07:04:25.0548729Z remote: Compressing objects: 68% (727/1068) 2024-06-26T07:04:25.0551842Z remote: Compressing objects: 69% (737/1068) 2024-06-26T07:04:25.0555078Z remote: Compressing objects: 70% (748/1068) 2024-06-26T07:04:25.0557968Z remote: Compressing objects: 71% (759/1068) 2024-06-26T07:04:25.0561959Z remote: Compressing objects: 72% (769/1068) 2024-06-26T07:04:25.0565091Z remote: Compressing objects: 73% (780/1068) 2024-06-26T07:04:25.0565796Z remote: Compressing objects: 74% (791/1068) 2024-06-26T07:04:25.0566553Z remote: Compressing objects: 75% (801/1068) 2024-06-26T07:04:25.0567330Z remote: Compressing objects: 76% (812/1068) 2024-06-26T07:04:25.0567919Z remote: Compressing objects: 77% (823/1068) 2024-06-26T07:04:25.0568451Z remote: Compressing objects: 78% (834/1068) 2024-06-26T07:04:25.0568989Z remote: Compressing objects: 79% (844/1068) 2024-06-26T07:04:25.0569511Z remote: Compressing objects: 80% (855/1068) 2024-06-26T07:04:25.0574362Z remote: Compressing objects: 81% (866/1068) 2024-06-26T07:04:25.0577748Z remote: Compressing objects: 82% (876/1068) 2024-06-26T07:04:25.0578291Z remote: Compressing objects: 83% (887/1068) 2024-06-26T07:04:25.0580767Z remote: Compressing objects: 84% (898/1068) 2024-06-26T07:04:25.0581399Z remote: Compressing objects: 85% (908/1068) 2024-06-26T07:04:25.0581928Z remote: Compressing objects: 86% (919/1068) 2024-06-26T07:04:25.0586099Z remote: Compressing objects: 87% (930/1068) 2024-06-26T07:04:25.0590538Z remote: Compressing objects: 88% (940/1068) 2024-06-26T07:04:25.0591159Z remote: Compressing objects: 89% (951/1068) 2024-06-26T07:04:25.0591760Z remote: Compressing objects: 90% (962/1068) 2024-06-26T07:04:25.0592452Z remote: Compressing objects: 91% (972/1068) 2024-06-26T07:04:25.0592985Z remote: Compressing objects: 92% (983/1068) 2024-06-26T07:04:25.0593509Z remote: Compressing objects: 93% (994/1068) 2024-06-26T07:04:25.0594049Z remote: Compressing objects: 94% (1004/1068) 2024-06-26T07:04:25.0594743Z remote: Compressing objects: 95% (1015/1068) 2024-06-26T07:04:25.0595351Z remote: Compressing objects: 96% (1026/1068) 2024-06-26T07:04:25.0596029Z remote: Compressing objects: 97% (1036/1068) 2024-06-26T07:04:25.0596573Z remote: Compressing objects: 98% (1047/1068) 2024-06-26T07:04:25.0598393Z remote: Compressing objects: 99% (1058/1068) 2024-06-26T07:04:25.0598947Z remote: Compressing objects: 100% (1068/1068) 2024-06-26T07:04:25.0599522Z remote: Compressing objects: 100% (1068/1068), done. 2024-06-26T07:04:45.1547052Z remote: Total 990209 (delta 2055), reused 1861 (delta 1521), pack-reused 987619 2024-06-26T07:05:08.0279492Z [command]/usr/bin/git rev-parse --verify --quiet b8c4c54d347aa776934c60784e35936878ef18dc^{object} 2024-06-26T07:05:08.0299546Z b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:05:08.0303340Z ##[endgroup] 2024-06-26T07:05:08.0303923Z ##[group]Determining the checkout info 2024-06-26T07:05:08.0306302Z ##[endgroup] 2024-06-26T07:05:08.0306856Z ##[group]Checking out the ref 2024-06-26T07:05:08.0307760Z [command]/usr/bin/git checkout --quiet --force b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:05:09.2572619Z ##[endgroup] 2024-06-26T07:05:09.2573243Z ##[group]Setting up auth for fetching submodules 2024-06-26T07:05:09.2575492Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:05:09.2619375Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-06-26T07:05:09.2640179Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-06-26T07:05:09.2660075Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-06-26T07:05:09.2678077Z ##[endgroup] 2024-06-26T07:05:09.2678612Z ##[group]Fetching submodules 2024-06-26T07:05:09.2681808Z [command]/usr/bin/git submodule sync --recursive 2024-06-26T07:05:09.2890580Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-06-26T07:05:09.3096489Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-06-26T07:05:09.3097952Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-06-26T07:05:09.3099637Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-06-26T07:05:09.3101260Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-06-26T07:05:09.3103571Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-06-26T07:05:09.3105344Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-06-26T07:05:09.3107956Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-06-26T07:05:09.3110356Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-06-26T07:05:09.3112768Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-06-26T07:05:09.3115385Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-06-26T07:05:09.3117903Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-06-26T07:05:09.3120568Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-06-26T07:05:09.3123336Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-06-26T07:05:09.3126254Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-06-26T07:05:09.3129087Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-06-26T07:05:09.3132888Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-06-26T07:05:09.3135693Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:05:09.3138493Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-06-26T07:05:09.3141778Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-06-26T07:05:09.3144751Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-06-26T07:05:09.3148163Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-06-26T07:05:09.3151460Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-06-26T07:05:09.3155003Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-06-26T07:05:09.3158434Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-06-26T07:05:09.3161987Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-06-26T07:05:09.3165578Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-06-26T07:05:09.3169498Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-06-26T07:05:09.3173479Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-06-26T07:05:09.3177237Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-06-26T07:05:09.3181152Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-06-26T07:05:09.3185049Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-06-26T07:05:09.3188865Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-06-26T07:05:09.3192952Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-06-26T07:05:09.3196909Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-06-26T07:05:09.3201172Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-06-26T07:05:09.3220906Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-06-26T07:05:09.5590872Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-06-26T07:05:09.7216281Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-06-26T07:05:09.8895271Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-06-26T07:05:10.1364218Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-06-26T07:05:12.2000729Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-06-26T07:05:21.7506861Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-06-26T07:05:22.1337262Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-06-26T07:05:22.6148120Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-06-26T07:05:23.1568600Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-06-26T07:05:24.2831945Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-06-26T07:05:25.9943946Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-06-26T07:05:31.7625793Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-06-26T07:05:33.0292880Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-06-26T07:05:34.6297394Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-06-26T07:05:35.8858014Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-06-26T07:05:36.0263467Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-06-26T07:05:36.3839529Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-06-26T07:05:36.6574298Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-06-26T07:05:37.6195511Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-06-26T07:05:37.9340006Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-06-26T07:05:38.1635637Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-06-26T07:05:39.6406679Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-06-26T07:05:40.3440461Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-06-26T07:05:41.0277164Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-06-26T07:05:47.8362741Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-06-26T07:05:50.1442549Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-06-26T07:05:55.6927707Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-06-26T07:05:55.8707707Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-06-26T07:06:03.8505719Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-06-26T07:06:04.0447761Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-06-26T07:06:04.2605593Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-06-26T07:06:05.4553690Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-06-26T07:06:05.7205560Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-06-26T07:06:06.3086210Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-06-26T07:06:06.7331175Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-06-26T07:06:06.7415757Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-06-26T07:06:06.7474260Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-06-26T07:06:06.7654215Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-06-26T07:06:06.7957847Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-06-26T07:06:07.5701963Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-06-26T07:06:07.5878246Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-06-26T07:06:07.6236232Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-06-26T07:06:07.7081236Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-06-26T07:06:07.7349993Z Submodule path 'third_party/cudnn_frontend': checked out 'aa3abd4bc689d6412979c7f55f9cd132848c9c6a' 2024-06-26T07:06:08.1656352Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-06-26T07:06:08.3775748Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-06-26T07:06:08.4414233Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-06-26T07:06:08.4426221Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:06:08.4429464Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:06:08.4430975Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:06:08.4432486Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:06:08.4434151Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:06:08.4451069Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-06-26T07:06:09.3437405Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-06-26T07:06:09.9532003Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-06-26T07:06:11.6448047Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-06-26T07:06:12.6212441Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-06-26T07:06:12.9423860Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-06-26T07:06:13.0262354Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-06-26T07:06:13.3711866Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-06-26T07:06:13.4259331Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-06-26T07:06:13.4350040Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-06-26T07:06:13.5260935Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-06-26T07:06:13.5541703Z Submodule path 'third_party/fmt': checked out 'e69e5f977d458f2650bb346dadf2ad30c5320281' 2024-06-26T07:06:13.5607957Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-06-26T07:06:13.5915055Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-06-26T07:06:13.6115063Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-06-26T07:06:13.6494133Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:06:13.6592110Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-06-26T07:06:13.6602507Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-06-26T07:06:13.6618598Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-06-26T07:06:25.4464723Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-06-26T07:06:25.4601591Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-06-26T07:06:25.5379259Z Submodule path 'third_party/kineto': checked out '8681ff11e1fa54da39023076c5c43eddd87b7a8a' 2024-06-26T07:06:25.5392183Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:06:25.5393863Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:06:25.5395536Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:06:25.5411119Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-06-26T07:06:26.0543097Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-06-26T07:06:27.3329231Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-06-26T07:06:28.4068249Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-06-26T07:06:28.4078378Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:06:28.4080363Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:06:28.4082279Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:06:28.4084265Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:06:28.4086443Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:06:28.4088514Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:06:28.4090558Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:06:28.4092898Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:06:28.4110933Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-06-26T07:06:29.4688543Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-06-26T07:06:29.8550406Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-06-26T07:06:31.1272358Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-06-26T07:06:31.3928998Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-06-26T07:06:31.9044054Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-06-26T07:06:32.9087368Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-06-26T07:06:38.5207080Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-06-26T07:06:38.8540869Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-06-26T07:06:38.8675932Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-06-26T07:06:38.8965792Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-06-26T07:06:38.9061008Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-06-26T07:06:38.9070625Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:06:38.9086948Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-06-26T07:06:39.1891711Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-06-26T07:06:39.2035543Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-06-26T07:06:39.2368267Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-06-26T07:06:39.3159283Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-06-26T07:06:39.3275188Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-06-26T07:06:39.3570456Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out 'a33701196adfad74917046096bf5a2aa0ab0bb50' 2024-06-26T07:06:39.4063150Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-06-26T07:06:39.4366517Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-06-26T07:06:39.4563023Z Submodule path 'third_party/nccl/nccl': checked out '48bb7fec7953112ff37499a272317f6663f8f600' 2024-06-26T07:06:39.5423401Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-06-26T07:06:39.8147741Z Submodule path 'third_party/onnx': checked out '990217f043af7222348ca8f0301e17fa7b841781' 2024-06-26T07:06:39.8170212Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-06-26T07:06:39.8172386Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-06-26T07:06:39.8189104Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-06-26T07:06:40.2692017Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-06-26T07:06:41.4110690Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-06-26T07:06:41.4387829Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-06-26T07:06:41.4911126Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-06-26T07:06:41.4922460Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:06:41.4931538Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:06:41.4933372Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:06:41.4934999Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:06:41.4936889Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:06:41.4938906Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:06:41.4940798Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:06:41.4942404Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:06:41.4956970Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-06-26T07:06:41.9269926Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-06-26T07:06:42.9511362Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-06-26T07:06:43.2359663Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-06-26T07:06:49.4811533Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-06-26T07:06:49.7879620Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-06-26T07:06:49.9624621Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-06-26T07:06:50.2207672Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-06-26T07:06:57.7769801Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-06-26T07:06:57.8108961Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-06-26T07:06:57.8232312Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-06-26T07:06:57.9082309Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-06-26T07:06:57.9171757Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-06-26T07:06:57.9276391Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-06-26T07:06:57.9388223Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-06-26T07:06:57.9398499Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:06:57.9400487Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:06:57.9417698Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-06-26T07:06:59.6866051Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-06-26T07:07:00.8550656Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-06-26T07:07:00.8915289Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:07:01.2485509Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-06-26T07:07:01.2570271Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-06-26T07:07:01.4740950Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-06-26T07:07:01.4757256Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:01.4759133Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:01.4777811Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-06-26T07:07:01.8637312Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-06-26T07:07:02.8734580Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-06-26T07:07:02.9358371Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-06-26T07:07:02.9418525Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-06-26T07:07:02.9505393Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-06-26T07:07:02.9782336Z Submodule path 'third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b' 2024-06-26T07:07:03.0000075Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-06-26T07:07:03.0336722Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-06-26T07:07:03.0540882Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-06-26T07:07:03.0552454Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:03.0554466Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:03.0557191Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:03.0558800Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:03.0574315Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-06-26T07:07:03.9704462Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-06-26T07:07:04.1815735Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-06-26T07:07:05.2209274Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-06-26T07:07:07.4891320Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-06-26T07:07:07.5005490Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-06-26T07:07:07.5568296Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-06-26T07:07:07.5798425Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-06-26T07:07:07.5809190Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:07.5826510Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-06-26T07:07:07.7647780Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-06-26T07:07:07.7670087Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-06-26T07:07:07.7878612Z Entering 'android/libs/fbjni' 2024-06-26T07:07:07.7907087Z Entering 'third_party/FP16' 2024-06-26T07:07:07.7935231Z Entering 'third_party/FXdiv' 2024-06-26T07:07:07.7960139Z Entering 'third_party/NNPACK' 2024-06-26T07:07:07.7990488Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:07.8018380Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:07.8057182Z Entering 'third_party/benchmark' 2024-06-26T07:07:07.8086487Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:07.8115083Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:07.8141231Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:07.8169076Z Entering 'third_party/cutlass' 2024-06-26T07:07:07.8203833Z Entering 'third_party/eigen' 2024-06-26T07:07:07.8232974Z Entering 'third_party/fbgemm' 2024-06-26T07:07:07.8261842Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:07.8287411Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:07.8315715Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:07.8346530Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:07.8374721Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:07.8401499Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:07.8430806Z Entering 'third_party/fmt' 2024-06-26T07:07:07.8458946Z Entering 'third_party/foxi' 2024-06-26T07:07:07.8485809Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:07.8516960Z Entering 'third_party/gloo' 2024-06-26T07:07:07.8542886Z Entering 'third_party/googletest' 2024-06-26T07:07:07.8572762Z Entering 'third_party/ideep' 2024-06-26T07:07:07.8599638Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:07.8633100Z Entering 'third_party/ittapi' 2024-06-26T07:07:07.8661576Z Entering 'third_party/kineto' 2024-06-26T07:07:07.8689774Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:07.8718128Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:07.8745201Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:07.8773449Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:07.8800654Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:07.8826999Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:07.8853413Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:07.8884391Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:07.8911981Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:07.8939892Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:07.8968197Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:07.8996392Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:07.9023258Z Entering 'third_party/mimalloc' 2024-06-26T07:07:07.9052917Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:07.9081568Z Entering 'third_party/nlohmann' 2024-06-26T07:07:07.9108994Z Entering 'third_party/onnx' 2024-06-26T07:07:07.9148626Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:07.9177134Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:07.9207561Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:07.9238451Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:07.9264851Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:07.9291195Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:07.9316996Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:07.9344621Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:07.9372542Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:07.9399327Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:07.9424823Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:07.9454593Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:07.9482608Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:07.9524812Z Entering 'third_party/pocketfft' 2024-06-26T07:07:07.9554194Z Entering 'third_party/protobuf' 2024-06-26T07:07:07.9584568Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:07.9609944Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:07.9638783Z Entering 'third_party/psimd' 2024-06-26T07:07:07.9665997Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:07.9694212Z Entering 'third_party/pybind11' 2024-06-26T07:07:07.9720665Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:07.9746646Z Entering 'third_party/sleef' 2024-06-26T07:07:07.9775158Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:07.9801855Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:07.9828129Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:07.9853732Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:07.9880891Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:07.9907848Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:07.9945446Z ##[endgroup] 2024-06-26T07:07:07.9947474Z ##[group]Persisting credentials for submodules 2024-06-26T07:07:07.9951590Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-06-26T07:07:08.0157391Z Entering 'android/libs/fbjni' 2024-06-26T07:07:08.0193971Z Entering 'third_party/FP16' 2024-06-26T07:07:08.0225715Z Entering 'third_party/FXdiv' 2024-06-26T07:07:08.0261568Z Entering 'third_party/NNPACK' 2024-06-26T07:07:08.0297010Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:08.0333543Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:08.0378445Z Entering 'third_party/benchmark' 2024-06-26T07:07:08.0411742Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:08.0446302Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:08.0481998Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:08.0518122Z Entering 'third_party/cutlass' 2024-06-26T07:07:08.0559182Z Entering 'third_party/eigen' 2024-06-26T07:07:08.0597625Z Entering 'third_party/fbgemm' 2024-06-26T07:07:08.0634083Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:08.0668121Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:08.0703057Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:08.0742159Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:08.0777105Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:08.0811259Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:08.0849154Z Entering 'third_party/fmt' 2024-06-26T07:07:08.0883309Z Entering 'third_party/foxi' 2024-06-26T07:07:08.0919452Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:08.0956035Z Entering 'third_party/gloo' 2024-06-26T07:07:08.0990420Z Entering 'third_party/googletest' 2024-06-26T07:07:08.1025943Z Entering 'third_party/ideep' 2024-06-26T07:07:08.1059494Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:08.1098994Z Entering 'third_party/ittapi' 2024-06-26T07:07:08.1131452Z Entering 'third_party/kineto' 2024-06-26T07:07:08.1167024Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:08.1201920Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:08.1238261Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:08.1274669Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:08.1311156Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:08.1346128Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:08.1379920Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:08.1412459Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:08.1446469Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:08.1481636Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:08.1519021Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:08.1552186Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:08.1587383Z Entering 'third_party/mimalloc' 2024-06-26T07:07:08.1622412Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:08.1657002Z Entering 'third_party/nlohmann' 2024-06-26T07:07:08.1693359Z Entering 'third_party/onnx' 2024-06-26T07:07:08.1739541Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:08.1773914Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:08.1808688Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:08.1843076Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:08.1879157Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:08.1911890Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:08.1945944Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:08.1978839Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:08.2010099Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:08.2044025Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:08.2076966Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:08.2113241Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:08.2149628Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:08.2196469Z Entering 'third_party/pocketfft' 2024-06-26T07:07:08.2233920Z Entering 'third_party/protobuf' 2024-06-26T07:07:08.2270342Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:08.2305636Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:08.2341261Z Entering 'third_party/psimd' 2024-06-26T07:07:08.2373831Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:08.2408271Z Entering 'third_party/pybind11' 2024-06-26T07:07:08.2440017Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:08.2474848Z Entering 'third_party/sleef' 2024-06-26T07:07:08.2509193Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:08.2545077Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:08.2579401Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:08.2611658Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:08.2644869Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:08.2679067Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:08.2727140Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-06-26T07:07:08.2935649Z Entering 'android/libs/fbjni' 2024-06-26T07:07:08.2969839Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-06-26T07:07:08.2983815Z Entering 'third_party/FP16' 2024-06-26T07:07:08.3016560Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-06-26T07:07:08.3030816Z Entering 'third_party/FXdiv' 2024-06-26T07:07:08.3063391Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-06-26T07:07:08.3076950Z Entering 'third_party/NNPACK' 2024-06-26T07:07:08.3107665Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-06-26T07:07:08.3122330Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:08.3154778Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-06-26T07:07:08.3166942Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:08.3198436Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-06-26T07:07:08.3223028Z Entering 'third_party/benchmark' 2024-06-26T07:07:08.3255350Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:08.3267690Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:08.3299599Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-06-26T07:07:08.3311486Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:08.3345033Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:07:08.3357355Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:08.3390190Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-06-26T07:07:08.3402990Z Entering 'third_party/cutlass' 2024-06-26T07:07:08.3434963Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:07:08.3452814Z Entering 'third_party/eigen' 2024-06-26T07:07:08.3482889Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-06-26T07:07:08.3496051Z Entering 'third_party/fbgemm' 2024-06-26T07:07:08.3526921Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-06-26T07:07:08.3541315Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:08.3571667Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-06-26T07:07:08.3584078Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:08.3615448Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:07:08.3628030Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:08.3658308Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:07:08.3674823Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:08.3706256Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:08.3718287Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:08.3750731Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-06-26T07:07:08.3763586Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:08.3796015Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-06-26T07:07:08.3809416Z Entering 'third_party/fmt' 2024-06-26T07:07:08.3840769Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:07:08.3855958Z Entering 'third_party/foxi' 2024-06-26T07:07:08.3886626Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-06-26T07:07:08.3899637Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:08.3930533Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-06-26T07:07:08.3943047Z Entering 'third_party/gloo' 2024-06-26T07:07:08.3974651Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-06-26T07:07:08.3987442Z Entering 'third_party/googletest' 2024-06-26T07:07:08.4017759Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:08.4030089Z Entering 'third_party/ideep' 2024-06-26T07:07:08.4061473Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-06-26T07:07:08.4072642Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:08.4103178Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-06-26T07:07:08.4121129Z Entering 'third_party/ittapi' 2024-06-26T07:07:08.4153260Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-06-26T07:07:08.4164562Z Entering 'third_party/kineto' 2024-06-26T07:07:08.4196564Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-06-26T07:07:08.4207332Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:08.4239640Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-06-26T07:07:08.4250709Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:08.4282971Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-06-26T07:07:08.4295803Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:08.4326912Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-06-26T07:07:08.4340220Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:08.4370890Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:07:08.4382681Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:08.4414355Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-06-26T07:07:08.4426204Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:08.4457304Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-06-26T07:07:08.4471277Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:08.4503220Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-06-26T07:07:08.4515512Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:08.4547544Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:08.4560302Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:08.4592961Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-06-26T07:07:08.4606642Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:08.4639237Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-06-26T07:07:08.4653654Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:08.4685117Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-06-26T07:07:08.4697623Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:08.4729485Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-06-26T07:07:08.4742308Z Entering 'third_party/mimalloc' 2024-06-26T07:07:08.4774363Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-06-26T07:07:08.4787680Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:08.4820194Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-06-26T07:07:08.4833118Z Entering 'third_party/nlohmann' 2024-06-26T07:07:08.4864634Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-06-26T07:07:08.4879579Z Entering 'third_party/onnx' 2024-06-26T07:07:08.4911630Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-06-26T07:07:08.4937199Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:08.4968485Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:08.4983551Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:08.5014491Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:07:08.5030259Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:08.5061835Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-06-26T07:07:08.5075288Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:08.5106627Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:08.5118546Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:08.5148488Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:08.5160200Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:08.5190916Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-06-26T07:07:08.5203183Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:08.5233869Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-06-26T07:07:08.5247356Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:08.5278744Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-06-26T07:07:08.5289507Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:08.5321334Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-06-26T07:07:08.5334334Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:08.5364623Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-06-26T07:07:08.5377187Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:08.5408460Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-06-26T07:07:08.5423020Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:08.5454912Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-06-26T07:07:08.5468926Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:08.5501020Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-06-26T07:07:08.5529500Z Entering 'third_party/pocketfft' 2024-06-26T07:07:08.5560437Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-06-26T07:07:08.5573461Z Entering 'third_party/protobuf' 2024-06-26T07:07:08.5604896Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-06-26T07:07:08.5620462Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:08.5648978Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:08.5660463Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:08.5690892Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:08.5704277Z Entering 'third_party/psimd' 2024-06-26T07:07:08.5735151Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-06-26T07:07:08.5748708Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:08.5779799Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-06-26T07:07:08.5792042Z Entering 'third_party/pybind11' 2024-06-26T07:07:08.5824225Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:07:08.5835472Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:08.5866609Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-06-26T07:07:08.5878851Z Entering 'third_party/sleef' 2024-06-26T07:07:08.5909250Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-06-26T07:07:08.5920549Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:08.5953756Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-06-26T07:07:08.5965403Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:08.5996808Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:08.6008093Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:08.6040289Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-06-26T07:07:08.6053536Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:08.6084133Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-06-26T07:07:08.6098515Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:08.6129685Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:07:08.6140562Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:08.6173453Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-06-26T07:07:08.6930321Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-06-26T07:07:08.7130285Z Entering 'android/libs/fbjni' 2024-06-26T07:07:08.7157150Z Entering 'third_party/FP16' 2024-06-26T07:07:08.7183919Z Entering 'third_party/FXdiv' 2024-06-26T07:07:08.7209621Z Entering 'third_party/NNPACK' 2024-06-26T07:07:08.7236060Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:08.7261907Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:08.7302431Z Entering 'third_party/benchmark' 2024-06-26T07:07:08.7336117Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:08.7364889Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:08.7390670Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:08.7419544Z Entering 'third_party/cutlass' 2024-06-26T07:07:08.7453509Z Entering 'third_party/eigen' 2024-06-26T07:07:08.7483590Z Entering 'third_party/fbgemm' 2024-06-26T07:07:08.7512170Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:08.7539058Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:08.7567387Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:08.7599888Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:08.7626903Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:08.7654533Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:08.7683289Z Entering 'third_party/fmt' 2024-06-26T07:07:08.7713527Z Entering 'third_party/foxi' 2024-06-26T07:07:08.7738918Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:08.7766774Z Entering 'third_party/gloo' 2024-06-26T07:07:08.7796043Z Entering 'third_party/googletest' 2024-06-26T07:07:08.7822826Z Entering 'third_party/ideep' 2024-06-26T07:07:08.7848894Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:08.7879642Z Entering 'third_party/ittapi' 2024-06-26T07:07:08.7905550Z Entering 'third_party/kineto' 2024-06-26T07:07:08.7933763Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:08.7960654Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:08.7988484Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:08.8014501Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:08.8042219Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:08.8067824Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:08.8095870Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:08.8121608Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:08.8148223Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:08.8174811Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:08.8201217Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:08.8228936Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:08.8257413Z Entering 'third_party/mimalloc' 2024-06-26T07:07:08.8286587Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:08.8316476Z Entering 'third_party/nlohmann' 2024-06-26T07:07:08.8343369Z Entering 'third_party/onnx' 2024-06-26T07:07:08.8383564Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:08.8410810Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:08.8441858Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:08.8470789Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:08.8497532Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:08.8524000Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:08.8551160Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:08.8579313Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:08.8606923Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:08.8634593Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:08.8660425Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:08.8689421Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:08.8716525Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:08.8757869Z Entering 'third_party/pocketfft' 2024-06-26T07:07:08.8787708Z Entering 'third_party/protobuf' 2024-06-26T07:07:08.8820073Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:08.8847762Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:08.8877853Z Entering 'third_party/psimd' 2024-06-26T07:07:08.8907797Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:08.8935053Z Entering 'third_party/pybind11' 2024-06-26T07:07:08.8963634Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:08.8990648Z Entering 'third_party/sleef' 2024-06-26T07:07:08.9018946Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:08.9046886Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:08.9074699Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:08.9100799Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:08.9125897Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:08.9152984Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:08.9193835Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-06-26T07:07:08.9398426Z Entering 'android/libs/fbjni' 2024-06-26T07:07:08.9427042Z Entering 'third_party/FP16' 2024-06-26T07:07:08.9458311Z Entering 'third_party/FXdiv' 2024-06-26T07:07:08.9485417Z Entering 'third_party/NNPACK' 2024-06-26T07:07:08.9514127Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:08.9542437Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:08.9580133Z Entering 'third_party/benchmark' 2024-06-26T07:07:08.9606093Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:08.9635637Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:08.9663172Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:08.9691306Z Entering 'third_party/cutlass' 2024-06-26T07:07:08.9726282Z Entering 'third_party/eigen' 2024-06-26T07:07:08.9756731Z Entering 'third_party/fbgemm' 2024-06-26T07:07:08.9784407Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:08.9809406Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:08.9835769Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:08.9864860Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:08.9891398Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:08.9920633Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:08.9950006Z Entering 'third_party/fmt' 2024-06-26T07:07:08.9978018Z Entering 'third_party/foxi' 2024-06-26T07:07:09.0006204Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:09.0035172Z Entering 'third_party/gloo' 2024-06-26T07:07:09.0063487Z Entering 'third_party/googletest' 2024-06-26T07:07:09.0093313Z Entering 'third_party/ideep' 2024-06-26T07:07:09.0121058Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:09.0151792Z Entering 'third_party/ittapi' 2024-06-26T07:07:09.0177896Z Entering 'third_party/kineto' 2024-06-26T07:07:09.0205192Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:09.0233288Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:09.0260123Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:09.0288257Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:09.0315313Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:09.0339799Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:09.0366520Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:09.0394442Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:09.0421013Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:09.0448143Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:09.0478264Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:09.0504259Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:09.0530108Z Entering 'third_party/mimalloc' 2024-06-26T07:07:09.0560836Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:09.0588105Z Entering 'third_party/nlohmann' 2024-06-26T07:07:09.0614568Z Entering 'third_party/onnx' 2024-06-26T07:07:09.0658099Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:09.0685595Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:09.0716539Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:09.0743896Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:09.0769009Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:09.0797968Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:09.0824431Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:09.0852942Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:09.0879162Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:09.0906638Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:09.0932785Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:09.0961100Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:09.0989899Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:09.1031429Z Entering 'third_party/pocketfft' 2024-06-26T07:07:09.1063691Z Entering 'third_party/protobuf' 2024-06-26T07:07:09.1093420Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:09.1120141Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:09.1149290Z Entering 'third_party/psimd' 2024-06-26T07:07:09.1176651Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:09.1204956Z Entering 'third_party/pybind11' 2024-06-26T07:07:09.1231702Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:09.1260132Z Entering 'third_party/sleef' 2024-06-26T07:07:09.1286869Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:09.1315975Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:09.1341923Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:09.1369130Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:09.1395203Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:09.1422507Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:09.1461500Z ##[endgroup] 2024-06-26T07:07:09.1493937Z [command]/usr/bin/git log -1 --format='%H' 2024-06-26T07:07:09.1515684Z 'b8c4c54d347aa776934c60784e35936878ef18dc' 2024-06-26T07:07:09.1695234Z Prepare all required actions 2024-06-26T07:07:09.1695686Z Getting action download info 2024-06-26T07:07:09.3036743Z ##[group]Run ./.github/actions/setup-linux 2024-06-26T07:07:09.3037167Z env: 2024-06-26T07:07:09.3037444Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:09.3037783Z ##[endgroup] 2024-06-26T07:07:09.3113443Z ##[group]Run set -euo pipefail 2024-06-26T07:07:09.3113885Z set -euo pipefail 2024-06-26T07:07:09.3114275Z function get_ec2_metadata() { 2024-06-26T07:07:09.3114927Z  # Pulled from instance metadata endpoint for EC2 2024-06-26T07:07:09.3115824Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-06-26T07:07:09.3116678Z  category=$1 2024-06-26T07:07:09.3117205Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-06-26T07:07:09.3117848Z  runner_name_str=i-08ac67d1e0a1089db 2024-06-26T07:07:09.3118360Z  if [[ -f /.inarc ]]; then 2024-06-26T07:07:09.3118849Z  echo "ARC Runner, no info on ec2 metadata" 2024-06-26T07:07:09.3119480Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-06-26T07:07:09.3120287Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-06-26T07:07:09.3120896Z  else 2024-06-26T07:07:09.3121389Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-06-26T07:07:09.3121964Z  fi 2024-06-26T07:07:09.3122251Z } 2024-06-26T07:07:09.3122616Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-06-26T07:07:09.3123192Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-06-26T07:07:09.3123849Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-06-26T07:07:09.3124421Z echo "system info $(uname -a)" 2024-06-26T07:07:09.3132631Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:09.3133132Z env: 2024-06-26T07:07:09.3133527Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:09.3134030Z ##[endgroup] 2024-06-26T07:07:09.3204831Z ami-id: ami-0ce0c36d7a00b20e2 2024-06-26T07:07:09.3251138Z instance-id: i-08ac67d1e0a1089db 2024-06-26T07:07:09.3294397Z instance-type: g5.4xlarge 2024-06-26T07:07:09.3300828Z system info Linux ip-10-0-35-135.ec2.internal 4.14.336-257.562.amzn2.x86_64 #1 SMP Sat Feb 24 09:50:35 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-06-26T07:07:09.3321285Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:07:09.3322205Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:07:09.3330066Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:09.3330566Z env: 2024-06-26T07:07:09.3331019Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:09.3331369Z ##[endgroup] 2024-06-26T07:07:09.3391880Z ##[group]Run if systemctl is-active --quiet docker; then 2024-06-26T07:07:09.3392616Z if systemctl is-active --quiet docker; then 2024-06-26T07:07:09.3393162Z  echo "Docker daemon is running..."; 2024-06-26T07:07:09.3393607Z else 2024-06-26T07:07:09.3394106Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-06-26T07:07:09.3394689Z fi 2024-06-26T07:07:09.3401724Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:09.3402219Z env: 2024-06-26T07:07:09.3402506Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:09.3402857Z ##[endgroup] 2024-06-26T07:07:09.3438260Z Docker daemon is running... 2024-06-26T07:07:09.3479649Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:07:09.3480198Z with: 2024-06-26T07:07:09.3480459Z shell: bash 2024-06-26T07:07:09.3480746Z timeout_minutes: 5 2024-06-26T07:07:09.3481060Z max_attempts: 3 2024-06-26T07:07:09.3481367Z retry_wait_seconds: 30 2024-06-26T07:07:09.3482885Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-06-26T07:07:09.3484666Z polling_interval_seconds: 1 2024-06-26T07:07:09.3485033Z warning_on_retry: true 2024-06-26T07:07:09.3485374Z continue_on_error: false 2024-06-26T07:07:09.3485699Z env: 2024-06-26T07:07:09.3485956Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:09.3486306Z AWS_RETRY_MODE: standard 2024-06-26T07:07:09.3486647Z AWS_MAX_ATTEMPTS: 5 2024-06-26T07:07:09.3486984Z AWS_DEFAULT_REGION: us-east-1 2024-06-26T07:07:09.3487341Z ##[endgroup] 2024-06-26T07:07:10.2212597Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:07:10.2213441Z Configure a credential helper to remove this warning. See 2024-06-26T07:07:10.2214280Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:07:10.2214824Z 2024-06-26T07:07:10.2214963Z Login Succeeded 2024-06-26T07:07:10.3986344Z Command completed after 1 attempt(s). 2024-06-26T07:07:10.4026946Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:07:10.4027667Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:07:10.4028330Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:07:10.4036076Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:10.4036576Z env: 2024-06-26T07:07:10.4036850Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:10.4037198Z ##[endgroup] 2024-06-26T07:07:10.4095357Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:07:10.4096126Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:07:10.4096719Z # shellcheck disable=SC2046 2024-06-26T07:07:10.4097185Z docker stop $(docker ps -q) || true 2024-06-26T07:07:10.4097694Z # Prune all of the docker images 2024-06-26T07:07:10.4098146Z docker system prune -af 2024-06-26T07:07:10.4105407Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:10.4105904Z env: 2024-06-26T07:07:10.4106172Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:10.4106515Z ##[endgroup] 2024-06-26T07:07:10.4351676Z "docker stop" requires at least 1 argument. 2024-06-26T07:07:10.4352587Z See 'docker stop --help'. 2024-06-26T07:07:10.4352941Z 2024-06-26T07:07:10.4353246Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-06-26T07:07:10.4353803Z 2024-06-26T07:07:10.4354005Z Stop one or more running containers 2024-06-26T07:07:10.4503859Z Total reclaimed space: 0B 2024-06-26T07:07:10.4533649Z ##[group]Run set +e 2024-06-26T07:07:10.4534013Z set +e 2024-06-26T07:07:10.4534316Z set -x 2024-06-26T07:07:10.4534614Z  2024-06-26T07:07:10.4534931Z PT_DOMAIN=download.pytorch.org 2024-06-26T07:07:10.4535852Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-06-26T07:07:10.4536936Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-06-26T07:07:10.4537687Z # one is returned at random 2024-06-26T07:07:10.4538237Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-06-26T07:07:10.4538748Z  2024-06-26T07:07:10.4539069Z if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:07:10.4539687Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-06-26T07:07:10.4540454Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-06-26T07:07:10.4541014Z  2024-06-26T07:07:10.4541342Z  if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:07:10.4541897Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-06-26T07:07:10.4542407Z  exit 1 2024-06-26T07:07:10.4542728Z  fi 2024-06-26T07:07:10.4543021Z fi 2024-06-26T07:07:10.4543406Z  2024-06-26T07:07:10.4543765Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-06-26T07:07:10.4544287Z  # Clean up any old records first 2024-06-26T07:07:10.4544799Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-06-26T07:07:10.4545262Z fi 2024-06-26T07:07:10.4545535Z  2024-06-26T07:07:10.4545971Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-06-26T07:07:10.4546525Z cat /etc/hosts 2024-06-26T07:07:10.4554236Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:10.4554723Z env: 2024-06-26T07:07:10.4554997Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:10.4555346Z ##[endgroup] 2024-06-26T07:07:10.4575854Z + PT_DOMAIN=download.pytorch.org 2024-06-26T07:07:10.4579905Z ++ dig -4 +short download.pytorch.org 2024-06-26T07:07:10.4580352Z ++ tail -n1 2024-06-26T07:07:10.4683663Z + RESOLVED_IP=18.67.76.121 2024-06-26T07:07:10.4684091Z + '[' -z 18.67.76.121 ']' 2024-06-26T07:07:10.4684523Z + grep -r download.pytorch.org /etc/hosts 2024-06-26T07:07:10.4689669Z 18.160.10.22 download.pytorch.org 2024-06-26T07:07:10.4690186Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-06-26T07:07:10.4784620Z + echo '18.67.76.121 download.pytorch.org' 2024-06-26T07:07:10.4785105Z + sudo tee -a /etc/hosts 2024-06-26T07:07:10.4964932Z 18.67.76.121 download.pytorch.org 2024-06-26T07:07:10.4978142Z + cat /etc/hosts 2024-06-26T07:07:10.4982877Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-06-26T07:07:10.4995879Z ::1 localhost6 localhost6.localdomain6 2024-06-26T07:07:10.4996386Z 18.67.76.121 download.pytorch.org 2024-06-26T07:07:10.5145528Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-06-26T07:07:10.5181111Z with: 2024-06-26T07:07:10.5182245Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:10.5183503Z docker-build-dir: .ci/docker 2024-06-26T07:07:10.5183874Z working-directory: . 2024-06-26T07:07:10.5184343Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:10.5184959Z force-push: false 2024-06-26T07:07:10.5185264Z env: 2024-06-26T07:07:10.5185531Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:10.5185870Z ##[endgroup] 2024-06-26T07:07:10.5203396Z ##[group]Run set -ex 2024-06-26T07:07:10.5203757Z set -ex 2024-06-26T07:07:10.5204050Z  2024-06-26T07:07:10.5204606Z # If the docker build directory or the build script doesn't exist, the action will 2024-06-26T07:07:10.5205578Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-06-26T07:07:10.5206385Z # job could then download the pre-built image as usual 2024-06-26T07:07:10.5207130Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-06-26T07:07:10.5207917Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:10.5208552Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:10.5209108Z  2024-06-26T07:07:10.5209628Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-06-26T07:07:10.5210250Z  exit 0 2024-06-26T07:07:10.5210546Z else 2024-06-26T07:07:10.5211019Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:10.5211466Z fi 2024-06-26T07:07:10.5211742Z  2024-06-26T07:07:10.5212413Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-06-26T07:07:10.5213273Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-06-26T07:07:10.5214054Z  # use it as it is, but first let's extract the tag 2024-06-26T07:07:10.5214778Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-06-26T07:07:10.5215509Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:10.5216208Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:10.5216760Z else 2024-06-26T07:07:10.5217196Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-06-26T07:07:10.5217863Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:10.5218766Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:10.5219517Z fi 2024-06-26T07:07:10.5227059Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:10.5227552Z env: 2024-06-26T07:07:10.5227826Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:10.5228175Z REPO_NAME: pytorch 2024-06-26T07:07:10.5229297Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:10.5230508Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:07:10.5231005Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:10.5231519Z ##[endgroup] 2024-06-26T07:07:10.5254717Z + [[ ! -d .ci/docker ]] 2024-06-26T07:07:10.5255152Z + [[ ! -f .ci/docker/build.sh ]] 2024-06-26T07:07:10.5255546Z + echo skip=false 2024-06-26T07:07:10.5257538Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-06-26T07:07:10.5260099Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:10.5261297Z ++ awk -F '[:,]' '{print $2}' 2024-06-26T07:07:10.5266767Z + DOCKER_TAG=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:10.5267417Z + echo docker-tag=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:10.5269248Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:10.5295639Z ##[group]Run set +e 2024-06-26T07:07:10.5296012Z set +e 2024-06-26T07:07:10.5296317Z set -x 2024-06-26T07:07:10.5296606Z  2024-06-26T07:07:10.5296884Z login() { 2024-06-26T07:07:10.5297554Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:07:10.5298278Z } 2024-06-26T07:07:10.5298553Z  2024-06-26T07:07:10.5298835Z retry () { 2024-06-26T07:07:10.5299228Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:07:10.5299804Z } 2024-06-26T07:07:10.5300077Z  2024-06-26T07:07:10.5300400Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:07:10.5300819Z  2024-06-26T07:07:10.5301287Z # Check if image already exists, if it does then skip building it 2024-06-26T07:07:10.5302048Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-06-26T07:07:10.5302553Z  exit 0 2024-06-26T07:07:10.5302861Z fi 2024-06-26T07:07:10.5303135Z  2024-06-26T07:07:10.5303635Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-06-26T07:07:10.5304467Z # be empty. The default action would be to continue rebuild the image 2024-06-26T07:07:10.5305210Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-06-26T07:07:10.5305881Z  # if we're on the base branch then use the parent commit 2024-06-26T07:07:10.5306483Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-06-26T07:07:10.5306927Z else 2024-06-26T07:07:10.5307398Z  # otherwise we're on a PR, so use the most recent base commit 2024-06-26T07:07:10.5308084Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-06-26T07:07:10.5308603Z fi 2024-06-26T07:07:10.5308892Z  2024-06-26T07:07:10.5309213Z if [[ -z "${MERGE_BASE}" ]]; then 2024-06-26T07:07:10.5309709Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:10.5310164Z  2024-06-26T07:07:10.5310815Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-06-26T07:07:10.5311571Z  exit 0 2024-06-26T07:07:10.5311908Z fi 2024-06-26T07:07:10.5312195Z  2024-06-26T07:07:10.5312635Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-06-26T07:07:10.5313606Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-06-26T07:07:10.5314410Z  exit 1 2024-06-26T07:07:10.5314723Z fi 2024-06-26T07:07:10.5315006Z  2024-06-26T07:07:10.5315496Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-06-26T07:07:10.5316440Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-06-26T07:07:10.5317318Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-06-26T07:07:10.5318285Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-06-26T07:07:10.5319374Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-06-26T07:07:10.5320152Z fi 2024-06-26T07:07:10.5320432Z  2024-06-26T07:07:10.5320796Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:10.5328476Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:10.5328980Z env: 2024-06-26T07:07:10.5329248Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:10.5329610Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:07:10.5330070Z BASE_REVISION: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:07:10.5331439Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:10.5332954Z DOCKER_TAG: 91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:10.5333556Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:10.5334075Z ##[endgroup] 2024-06-26T07:07:10.5353474Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:10.5354143Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:10.5355004Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:07:10.5355892Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:10.9347794Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:07:10.9348885Z Configure a credential helper to remove this warning. See 2024-06-26T07:07:10.9349911Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:07:10.9350471Z 2024-06-26T07:07:10.9350596Z Login Succeeded 2024-06-26T07:07:10.9361201Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:11.1578336Z { 2024-06-26T07:07:11.1578712Z "schemaVersion": 2, 2024-06-26T07:07:11.1579361Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-06-26T07:07:11.1579941Z "config": { 2024-06-26T07:07:11.1580386Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-06-26T07:07:11.1580930Z "size": 48681, 2024-06-26T07:07:11.1581466Z "digest": "sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd" 2024-06-26T07:07:11.1582214Z }, 2024-06-26T07:07:11.1582541Z "layers": [ 2024-06-26T07:07:11.1582858Z { 2024-06-26T07:07:11.1583293Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1583850Z "size": 28580681, 2024-06-26T07:07:11.1584423Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-06-26T07:07:11.1585278Z }, 2024-06-26T07:07:11.1585601Z { 2024-06-26T07:07:11.1586184Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1586942Z "size": 7943451, 2024-06-26T07:07:11.1587689Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-06-26T07:07:11.1588534Z }, 2024-06-26T07:07:11.1588851Z { 2024-06-26T07:07:11.1589432Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1590182Z "size": 55728572, 2024-06-26T07:07:11.1590931Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-06-26T07:07:11.1591813Z }, 2024-06-26T07:07:11.1592549Z + exit 0 2024-06-26T07:07:11.1592902Z { 2024-06-26T07:07:11.1593466Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1594191Z "size": 186, 2024-06-26T07:07:11.1594913Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-06-26T07:07:11.1595633Z }, 2024-06-26T07:07:11.1595873Z { 2024-06-26T07:07:11.1596301Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1596847Z "size": 6886, 2024-06-26T07:07:11.1597383Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-06-26T07:07:11.1598194Z }, 2024-06-26T07:07:11.1598448Z { 2024-06-26T07:07:11.1598878Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1599429Z "size": 1329001756, 2024-06-26T07:07:11.1599999Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-06-26T07:07:11.1600636Z }, 2024-06-26T07:07:11.1600883Z { 2024-06-26T07:07:11.1601312Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1601855Z "size": 62501, 2024-06-26T07:07:11.1602444Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-06-26T07:07:11.1603052Z }, 2024-06-26T07:07:11.1603291Z { 2024-06-26T07:07:11.1603720Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1604275Z "size": 1684, 2024-06-26T07:07:11.1604803Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-06-26T07:07:11.1605399Z }, 2024-06-26T07:07:11.1605647Z { 2024-06-26T07:07:11.1606077Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1606625Z "size": 1523, 2024-06-26T07:07:11.1607169Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-06-26T07:07:11.1607906Z }, 2024-06-26T07:07:11.1608151Z { 2024-06-26T07:07:11.1608568Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1609115Z "size": 2528295403, 2024-06-26T07:07:11.1609674Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-06-26T07:07:11.1610276Z }, 2024-06-26T07:07:11.1610513Z { 2024-06-26T07:07:11.1611085Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1611634Z "size": 86016, 2024-06-26T07:07:11.1612464Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-06-26T07:07:11.1613079Z }, 2024-06-26T07:07:11.1613329Z { 2024-06-26T07:07:11.1613758Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1614309Z "size": 1823, 2024-06-26T07:07:11.1614829Z "digest": "sha256:309b872da805fcdf74137a887464876fa0615dc8b24199307119eb65fc2bfb49" 2024-06-26T07:07:11.1615437Z }, 2024-06-26T07:07:11.1615683Z { 2024-06-26T07:07:11.1616099Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1616650Z "size": 246737542, 2024-06-26T07:07:11.1617197Z "digest": "sha256:e82932cb002c06a782077b8e5965f8733eb8563406a23cab260d9d8c4a994f7d" 2024-06-26T07:07:11.1617797Z }, 2024-06-26T07:07:11.1618036Z { 2024-06-26T07:07:11.1618464Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1619013Z "size": 545, 2024-06-26T07:07:11.1619549Z "digest": "sha256:ca9dd73e3e9378225ebdcfd12199ef44649e34fe55003cfbc8afe2e8a4527c5b" 2024-06-26T07:07:11.1620152Z }, 2024-06-26T07:07:11.1620395Z { 2024-06-26T07:07:11.1620824Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1621368Z "size": 1283, 2024-06-26T07:07:11.1621884Z "digest": "sha256:16d76a79d5d6616e228537785ec83a869f68a402520bbc6e826201621ca6cb98" 2024-06-26T07:07:11.1622477Z }, 2024-06-26T07:07:11.1622724Z { 2024-06-26T07:07:11.1623141Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1623699Z "size": 485, 2024-06-26T07:07:11.1624247Z "digest": "sha256:4bdf41c9ecbe1651e0986ccbea4663a0556af27b7145a679cfa45c35dc96b891" 2024-06-26T07:07:11.1624870Z }, 2024-06-26T07:07:11.1625108Z { 2024-06-26T07:07:11.1625533Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1626084Z "size": 91710937, 2024-06-26T07:07:11.1626652Z "digest": "sha256:f4863801e30cdbcd1d4e6cb1cbbcde7dbcaf6ae317ca80429295741ce60ac31d" 2024-06-26T07:07:11.1627269Z }, 2024-06-26T07:07:11.1627520Z { 2024-06-26T07:07:11.1627949Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1628594Z "size": 3233, 2024-06-26T07:07:11.1629149Z "digest": "sha256:d37dd9ed8948a0d02e1198bbedadbca36477a0980dd553196385aab0c025581d" 2024-06-26T07:07:11.1629767Z }, 2024-06-26T07:07:11.1630014Z { 2024-06-26T07:07:11.1630440Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1630989Z "size": 1905, 2024-06-26T07:07:11.1631535Z "digest": "sha256:cf2f34f225dbed17c1a9a8b9ea4ad030aba797774a716dc0a82fe212324aa751" 2024-06-26T07:07:11.1632198Z }, 2024-06-26T07:07:11.1632435Z { 2024-06-26T07:07:11.1632857Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1633399Z "size": 702, 2024-06-26T07:07:11.1633934Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:07:11.1634536Z }, 2024-06-26T07:07:11.1634787Z { 2024-06-26T07:07:11.1635214Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1635752Z "size": 2784789002, 2024-06-26T07:07:11.1636311Z "digest": "sha256:3d914950f8addfc4d657a2365c76e1143eb80b710cd3f6e9ae05640217fc6347" 2024-06-26T07:07:11.1636918Z }, 2024-06-26T07:07:11.1637166Z { 2024-06-26T07:07:11.1637583Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1638219Z "size": 381, 2024-06-26T07:07:11.1638747Z "digest": "sha256:96d0e2032184c57e62a652aaffcf0241f0d0c7a23e894ca0ad192c4a64a36583" 2024-06-26T07:07:11.1639351Z }, 2024-06-26T07:07:11.1639589Z { 2024-06-26T07:07:11.1640015Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1640569Z "size": 11879, 2024-06-26T07:07:11.1641094Z "digest": "sha256:06e12ce59790cb8b091e5d619c2a62a28329008ee184a8b9a562e3db12471f30" 2024-06-26T07:07:11.1641690Z }, 2024-06-26T07:07:11.1641934Z { 2024-06-26T07:07:11.1642415Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1642953Z "size": 803, 2024-06-26T07:07:11.1643501Z "digest": "sha256:ed712f63cde7070bdec2a6179c4c35e752c6991ddbc69fe9f2724a9903250ceb" 2024-06-26T07:07:11.1644112Z }, 2024-06-26T07:07:11.1644354Z { 2024-06-26T07:07:11.1644771Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1645327Z "size": 106, 2024-06-26T07:07:11.1645852Z "digest": "sha256:a97a71ef8a4e66234dfc2098834f8395ac5630240e60164d1a251fdb0115ba85" 2024-06-26T07:07:11.1646453Z }, 2024-06-26T07:07:11.1646690Z { 2024-06-26T07:07:11.1647118Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1647663Z "size": 503, 2024-06-26T07:07:11.1648183Z "digest": "sha256:25cd0c482ad612f936b35b4f220c580e16be982226aac8801a6a849a6bfdc221" 2024-06-26T07:07:11.1648787Z }, 2024-06-26T07:07:11.1649038Z { 2024-06-26T07:07:11.1649465Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1650002Z "size": 121477177, 2024-06-26T07:07:11.1650560Z "digest": "sha256:9bf94bc456584365fda21d0ed83b7cc690b08a71727b71c4c1e667e15028a386" 2024-06-26T07:07:11.1651247Z }, 2024-06-26T07:07:11.1651492Z { 2024-06-26T07:07:11.1651925Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1652702Z "size": 109, 2024-06-26T07:07:11.1653244Z "digest": "sha256:de45a14c9b227e4b0c9d45d42223b50a71c03522ddd9e2ca0d9517835206c823" 2024-06-26T07:07:11.1653843Z }, 2024-06-26T07:07:11.1654090Z { 2024-06-26T07:07:11.1654513Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1655057Z "size": 491, 2024-06-26T07:07:11.1655574Z "digest": "sha256:e99241e77209b0b35a0fb59a61c0821fc161190178af7e4af45e6eeaa043375e" 2024-06-26T07:07:11.1656170Z }, 2024-06-26T07:07:11.1656414Z { 2024-06-26T07:07:11.1656839Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1657372Z "size": 291, 2024-06-26T07:07:11.1657909Z "digest": "sha256:35e8f4e2a42f9df0c969dedd2199cf71719cf5ecfa676dfa014bbdeb4fdb295f" 2024-06-26T07:07:11.1658624Z }, 2024-06-26T07:07:11.1658875Z { 2024-06-26T07:07:11.1659292Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1659848Z "size": 101, 2024-06-26T07:07:11.1660377Z "digest": "sha256:a823ffa4c541394b52953391dc0a6867e26c37fe4a71fa022a2cc3f07f0ba70d" 2024-06-26T07:07:11.1660971Z }, 2024-06-26T07:07:11.1661219Z { 2024-06-26T07:07:11.1661642Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1662192Z "size": 1473, 2024-06-26T07:07:11.1662707Z "digest": "sha256:44ba272eae2064603605a6238b3f35cc6578859d660068e702a047a9cd8ab2fd" 2024-06-26T07:07:11.1663300Z }, 2024-06-26T07:07:11.1663546Z { 2024-06-26T07:07:11.1663968Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1664508Z "size": 446710781, 2024-06-26T07:07:11.1665062Z "digest": "sha256:da02d278e5240c285bc855ce4e42cd6121d5b903787f5c14f03e575ad1268650" 2024-06-26T07:07:11.1665666Z }, 2024-06-26T07:07:11.1665908Z { 2024-06-26T07:07:11.1666335Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1666885Z "size": 161, 2024-06-26T07:07:11.1667396Z "digest": "sha256:73a0ce681c295135456e190578733273276b38b4dd5babbb39298f384d43b1c6" 2024-06-26T07:07:11.1668062Z }, 2024-06-26T07:07:11.1668313Z { 2024-06-26T07:07:11.1668733Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1669275Z "size": 564, 2024-06-26T07:07:11.1669801Z "digest": "sha256:49737bd5ae709dbf7956e21fc8b05ba009e9e6c9b130b0126a73bd3d3f5efc78" 2024-06-26T07:07:11.1670407Z }, 2024-06-26T07:07:11.1670651Z { 2024-06-26T07:07:11.1671071Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1671608Z "size": 35874600, 2024-06-26T07:07:11.1672215Z "digest": "sha256:9cd4c745f8e4f9d5c9bb8050923575afad21f5b7852975a4a1981bc9eff282c5" 2024-06-26T07:07:11.1672816Z }, 2024-06-26T07:07:11.1673055Z { 2024-06-26T07:07:11.1673487Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1674025Z "size": 104, 2024-06-26T07:07:11.1674536Z "digest": "sha256:b2950a0365345d4c17a48627c905fd8930b3b3154c727f0526eb9ca547102bb7" 2024-06-26T07:07:11.1675126Z }, 2024-06-26T07:07:11.1675374Z { 2024-06-26T07:07:11.1675795Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1676399Z "size": 425, 2024-06-26T07:07:11.1676920Z "digest": "sha256:25e19d81d9fc081962ff4cbc09e098f4035419822712c19a410f5282300dfb9e" 2024-06-26T07:07:11.1677515Z }, 2024-06-26T07:07:11.1677751Z { 2024-06-26T07:07:11.1678173Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1678716Z "size": 20262084, 2024-06-26T07:07:11.1679262Z "digest": "sha256:60156b40fcbec2c25f0170b0a82fa8cd9303e0daae8bb1811fd0fc6562e88e8e" 2024-06-26T07:07:11.1679873Z }, 2024-06-26T07:07:11.1680122Z { 2024-06-26T07:07:11.1680553Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1681097Z "size": 439, 2024-06-26T07:07:11.1681613Z "digest": "sha256:7a868239227fd19b30a19927bbc8f058d247403fe5b343c55f4b4b8987918911" 2024-06-26T07:07:11.1682226Z }, 2024-06-26T07:07:11.1682503Z { 2024-06-26T07:07:11.1682918Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1683461Z "size": 702, 2024-06-26T07:07:11.1683995Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:07:11.1684601Z }, 2024-06-26T07:07:11.1684837Z { 2024-06-26T07:07:11.1685260Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1685803Z "size": 143, 2024-06-26T07:07:11.1686316Z "digest": "sha256:83911d7c7ecf11b5932bb7449975c52e7f737a5b867691935e1ccb85ceb25b52" 2024-06-26T07:07:11.1686911Z }, 2024-06-26T07:07:11.1687157Z { 2024-06-26T07:07:11.1687577Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1688196Z "size": 135, 2024-06-26T07:07:11.1688736Z "digest": "sha256:b3c5d66c679f58c71febdd502889bd8cd3d36e5001754cdf1e30eabe21135cce" 2024-06-26T07:07:11.1689350Z }, 2024-06-26T07:07:11.1689593Z { 2024-06-26T07:07:11.1690005Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1690558Z "size": 4911495247, 2024-06-26T07:07:11.1691174Z "digest": "sha256:84b2c7a899820f6acdb22148c5e8346d8b26c33269fd01842c9573515251e1e1" 2024-06-26T07:07:11.1691783Z }, 2024-06-26T07:07:11.1692021Z { 2024-06-26T07:07:11.1692623Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1693174Z "size": 190, 2024-06-26T07:07:11.1693714Z "digest": "sha256:c1068d03a811d8dead5da9e82fdc8869205fc4ee9cb92d4f7a4acfbb5ba83950" 2024-06-26T07:07:11.1694332Z }, 2024-06-26T07:07:11.1694577Z { 2024-06-26T07:07:11.1694999Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1695537Z "size": 565, 2024-06-26T07:07:11.1696064Z "digest": "sha256:671b4a301d81074fca17d36851f81801552c4ecb2f90ff3412aa9ab6a1a91371" 2024-06-26T07:07:11.1696665Z }, 2024-06-26T07:07:11.1696909Z { 2024-06-26T07:07:11.1697327Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1697974Z "size": 43163095, 2024-06-26T07:07:11.1698528Z "digest": "sha256:67760cf5ec46ab58d3ca98637fd9b26cb47a67446bfd0db176fe5e023606aa0d" 2024-06-26T07:07:11.1699133Z }, 2024-06-26T07:07:11.1699378Z { 2024-06-26T07:07:11.1699803Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1700346Z "size": 106, 2024-06-26T07:07:11.1700875Z "digest": "sha256:ddc9740bcea2b301e99b50525059aea8d9c7121a1c8eab566561a8968438a8c2" 2024-06-26T07:07:11.1701479Z }, 2024-06-26T07:07:11.1701723Z { 2024-06-26T07:07:11.1702146Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1702712Z "size": 1213, 2024-06-26T07:07:11.1703245Z "digest": "sha256:fb2e977685609fcc01de170ba7f50a573e73e6520e47253717e92a28952c5ee8" 2024-06-26T07:07:11.1703843Z }, 2024-06-26T07:07:11.1704087Z { 2024-06-26T07:07:11.1704504Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1705054Z "size": 702, 2024-06-26T07:07:11.1705588Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:07:11.1706194Z }, 2024-06-26T07:07:11.1706438Z { 2024-06-26T07:07:11.1706859Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1707399Z "size": 138, 2024-06-26T07:07:11.1707929Z "digest": "sha256:b599c5a8c7cb461cbd7b3c98b99960b8eb04506e05e79db1cc0cfadb14a47e91" 2024-06-26T07:07:11.1708541Z }, 2024-06-26T07:07:11.1708785Z { 2024-06-26T07:07:11.1709207Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1709744Z "size": 120, 2024-06-26T07:07:11.1710284Z "digest": "sha256:e08e88c1366827455ee7ddbf92b6ae615ceb3cbd3fb11cd03ddd31bd3fa94a9c" 2024-06-26T07:07:11.1710898Z }, 2024-06-26T07:07:11.1711136Z { 2024-06-26T07:07:11.1711558Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1712108Z "size": 1904157495, 2024-06-26T07:07:11.1712673Z "digest": "sha256:12ba7431ff38de8cf6af3052a2a52e79f4e712b8c99f5bafde3c20e63b1f570f" 2024-06-26T07:07:11.1713276Z }, 2024-06-26T07:07:11.1713519Z { 2024-06-26T07:07:11.1713942Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1714488Z "size": 172, 2024-06-26T07:07:11.1715019Z "digest": "sha256:cf7c2cb896cba6a594028cc0fd246e127a2f4481d48905a3aa12c5f5872332d4" 2024-06-26T07:07:11.1715630Z }, 2024-06-26T07:07:11.1715875Z { 2024-06-26T07:07:11.1716309Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1716850Z "size": 908, 2024-06-26T07:07:11.1717394Z "digest": "sha256:3a778ddfac2af5b3a11f49ddf2f3f1ad1731f74ba691f434ab69966d1e2a189f" 2024-06-26T07:07:11.1718095Z }, 2024-06-26T07:07:11.1718340Z { 2024-06-26T07:07:11.1718766Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1719312Z "size": 702, 2024-06-26T07:07:11.1719847Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:07:11.1720460Z }, 2024-06-26T07:07:11.1720708Z { 2024-06-26T07:07:11.1721130Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1721675Z "size": 134, 2024-06-26T07:07:11.1722194Z "digest": "sha256:236c8f0245f4b40616a4322eaf1e1b3e178d5990db1fcaba39622ea4e79c06a1" 2024-06-26T07:07:11.1722796Z }, 2024-06-26T07:07:11.1723044Z { 2024-06-26T07:07:11.1723462Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1724005Z "size": 32, 2024-06-26T07:07:11.1724540Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:11.1725153Z }, 2024-06-26T07:07:11.1725397Z { 2024-06-26T07:07:11.1725822Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1726367Z "size": 156, 2024-06-26T07:07:11.1726902Z "digest": "sha256:dda1ce302b9a9df0c8f1145d2e3eaa03c96909bb45a792451f1fcb555b06738d" 2024-06-26T07:07:11.1727564Z }, 2024-06-26T07:07:11.1727808Z { 2024-06-26T07:07:11.1728232Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1728779Z "size": 1841, 2024-06-26T07:07:11.1729310Z "digest": "sha256:e797fe86caef19319e03f8dca92cf96838ab3bb3644f4447c6dfc04230118afa" 2024-06-26T07:07:11.1729924Z }, 2024-06-26T07:07:11.1730174Z { 2024-06-26T07:07:11.1730591Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1731270Z "size": 7529784, 2024-06-26T07:07:11.1731815Z "digest": "sha256:967190076fc0ab2bb2fc0d74e58edd3d662a537d702ff3f8b6e876230154a16f" 2024-06-26T07:07:11.1732611Z }, 2024-06-26T07:07:11.1732850Z { 2024-06-26T07:07:11.1733280Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1733824Z "size": 165, 2024-06-26T07:07:11.1734350Z "digest": "sha256:4fb0a83f06855534ef82b80dcf7c685303edb2bcbd446196066dc2ad2dd71281" 2024-06-26T07:07:11.1734951Z }, 2024-06-26T07:07:11.1735194Z { 2024-06-26T07:07:11.1735619Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1736154Z "size": 7944, 2024-06-26T07:07:11.1736690Z "digest": "sha256:68af2ec4e30a33f26947ece9f3a31b7d3b04c2bca5865d4b48618457832a1da2" 2024-06-26T07:07:11.1737298Z }, 2024-06-26T07:07:11.1737545Z { 2024-06-26T07:07:11.1737960Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1738503Z "size": 8070, 2024-06-26T07:07:11.1739027Z "digest": "sha256:cc91933e985a044c6df53caa15488d2285896f5e991f38ec068c253ceec53f14" 2024-06-26T07:07:11.1739625Z }, 2024-06-26T07:07:11.1739862Z { 2024-06-26T07:07:11.1740286Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1740831Z "size": 303, 2024-06-26T07:07:11.1741362Z "digest": "sha256:9f3187916e7dcad33cc572f8f0e949d2f507b887f99cfd6acefd14826b44960d" 2024-06-26T07:07:11.1741965Z }, 2024-06-26T07:07:11.1742208Z { 2024-06-26T07:07:11.1742634Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1743170Z "size": 7629071, 2024-06-26T07:07:11.1743689Z "digest": "sha256:52608399485738d10d779254946c9e1d536010dd45593c69bf1eb9f62f85dc24" 2024-06-26T07:07:11.1744272Z }, 2024-06-26T07:07:11.1744515Z { 2024-06-26T07:07:11.1744926Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1745469Z "size": 108, 2024-06-26T07:07:11.1745998Z "digest": "sha256:05a6f6dcc1c00bac6660923a22adc7d7c34416718f8c89daf9b3351d9365f694" 2024-06-26T07:07:11.1746600Z }, 2024-06-26T07:07:11.1746836Z { 2024-06-26T07:07:11.1747258Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1747896Z "size": 54145775, 2024-06-26T07:07:11.1748449Z "digest": "sha256:6bc9dc8315b09b27c976bfba9700767ba412d72fd0528b18bbe8f8e62005ec22" 2024-06-26T07:07:11.1749060Z }, 2024-06-26T07:07:11.1749303Z { 2024-06-26T07:07:11.1749726Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1750269Z "size": 474, 2024-06-26T07:07:11.1750798Z "digest": "sha256:1b0e27c35f07b3fbf0aeb548d1949bf1d60a420136da21789d94d196c891d981" 2024-06-26T07:07:11.1751406Z }, 2024-06-26T07:07:11.1751647Z { 2024-06-26T07:07:11.1752065Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1752608Z "size": 1374858747, 2024-06-26T07:07:11.1753154Z "digest": "sha256:95237e7311a1c5ce6937db7196c866632f44e8788514a5c0eb4a3f04da4017e6" 2024-06-26T07:07:11.1753749Z }, 2024-06-26T07:07:11.1753984Z { 2024-06-26T07:07:11.1754406Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1754943Z "size": 106, 2024-06-26T07:07:11.1755486Z "digest": "sha256:789eff6c51aada70bf75eb1acafed63c0d839b768638001500616ff6e53b854a" 2024-06-26T07:07:11.1756105Z }, 2024-06-26T07:07:11.1756351Z { 2024-06-26T07:07:11.1756770Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1757375Z "size": 558, 2024-06-26T07:07:11.1757915Z "digest": "sha256:de0f8a8c45c6077b3c5ba3b84d7f07de98353fd618dad9a1e2f368ffe00493c7" 2024-06-26T07:07:11.1758529Z }, 2024-06-26T07:07:11.1758775Z { 2024-06-26T07:07:11.1759193Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1759738Z "size": 46248513, 2024-06-26T07:07:11.1760274Z "digest": "sha256:2060028f74888d218bebdd10d723f66c33ee1c708935c4ffd4654d34912ac4dc" 2024-06-26T07:07:11.1760874Z }, 2024-06-26T07:07:11.1761110Z { 2024-06-26T07:07:11.1761532Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1762100Z "size": 111, 2024-06-26T07:07:11.1762649Z "digest": "sha256:e412c7f3b2ded440992955bd92512345cbece8a8894cd3592f714ebc758ad1a0" 2024-06-26T07:07:11.1763251Z }, 2024-06-26T07:07:11.1763493Z { 2024-06-26T07:07:11.1763916Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1764456Z "size": 32, 2024-06-26T07:07:11.1764989Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:11.1765597Z }, 2024-06-26T07:07:11.1765838Z { 2024-06-26T07:07:11.1766252Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1766798Z "size": 32, 2024-06-26T07:07:11.1767327Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:11.1767925Z }, 2024-06-26T07:07:11.1768168Z { 2024-06-26T07:07:11.1768593Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1769135Z "size": 32, 2024-06-26T07:07:11.1769659Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:11.1770271Z }, 2024-06-26T07:07:11.1770515Z { 2024-06-26T07:07:11.1771045Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:11.1771589Z "size": 32, 2024-06-26T07:07:11.1772294Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:11.1772932Z } 2024-06-26T07:07:11.1773171Z ] 2024-06-26T07:07:11.1773416Z } 2024-06-26T07:07:11.1887312Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:07:11.1887793Z tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:07:11.1888341Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-06-26T07:07:11.1896718Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:11.1897224Z env: 2024-06-26T07:07:11.1897507Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:11.1898659Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:11.1899825Z ##[endgroup] 2024-06-26T07:07:11.1922014Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:11.1970345Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-06-26T07:07:11.1971013Z with: 2024-06-26T07:07:11.1972212Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:11.1973526Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:11.1974040Z env: 2024-06-26T07:07:11.1974318Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:11.1974668Z ##[endgroup] 2024-06-26T07:07:11.1989685Z ##[group]Run set -x 2024-06-26T07:07:11.1990021Z set -x 2024-06-26T07:07:11.1990324Z set +e 2024-06-26T07:07:11.1990622Z  2024-06-26T07:07:11.1990896Z login() { 2024-06-26T07:07:11.1991556Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:07:11.1992270Z } 2024-06-26T07:07:11.1992550Z  2024-06-26T07:07:11.1992987Z retry () { 2024-06-26T07:07:11.1993381Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:07:11.1993822Z } 2024-06-26T07:07:11.1994098Z  2024-06-26T07:07:11.1994410Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:07:11.1994821Z  2024-06-26T07:07:11.1995091Z set -e 2024-06-26T07:07:11.1995561Z # ignore output since only exit code is used for conditional 2024-06-26T07:07:11.1996263Z # only pull docker image if it's not available locally 2024-06-26T07:07:11.1997018Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-06-26T07:07:11.1997704Z  retry docker pull "${DOCKER_IMAGE}" 2024-06-26T07:07:11.1998140Z fi 2024-06-26T07:07:11.2004770Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:11.2005259Z env: 2024-06-26T07:07:11.2005531Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:11.2006669Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:11.2007977Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:11.2008493Z ##[endgroup] 2024-06-26T07:07:11.2028651Z + set +e 2024-06-26T07:07:11.2029326Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:11.2030041Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:11.2030864Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:07:11.2031658Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:11.6081471Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:07:11.6082308Z Configure a credential helper to remove this warning. See 2024-06-26T07:07:11.6083131Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:07:11.6083682Z 2024-06-26T07:07:11.6083817Z Login Succeeded 2024-06-26T07:07:11.6093033Z + set -e 2024-06-26T07:07:11.6094600Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:11.6213291Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:11.6215442Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:11.8528312Z 91382da70d5719cd7007b6b80b71d2f48398f6b7: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-06-26T07:07:11.8529632Z 7a2c55901189: Pulling fs layer 2024-06-26T07:07:11.8530207Z 224fe954d725: Pulling fs layer 2024-06-26T07:07:11.8531149Z 75722010b82e: Pulling fs layer 2024-06-26T07:07:11.8531700Z d527cbbb87e3: Pulling fs layer 2024-06-26T07:07:11.8532468Z b57676e46aee: Pulling fs layer 2024-06-26T07:07:11.8535105Z a8c1e85b5e14: Pulling fs layer 2024-06-26T07:07:11.8535632Z a41a8d1c11c8: Pulling fs layer 2024-06-26T07:07:11.8536096Z 0c1227890755: Pulling fs layer 2024-06-26T07:07:11.8536652Z d8d1234baab3: Pulling fs layer 2024-06-26T07:07:11.8537176Z 7ed32bc8e469: Pulling fs layer 2024-06-26T07:07:11.8537581Z ec1e7978c1fe: Pulling fs layer 2024-06-26T07:07:11.8537949Z 309b872da805: Pulling fs layer 2024-06-26T07:07:11.8538324Z e82932cb002c: Pulling fs layer 2024-06-26T07:07:11.8538716Z ca9dd73e3e93: Pulling fs layer 2024-06-26T07:07:11.8539248Z 16d76a79d5d6: Pulling fs layer 2024-06-26T07:07:11.8539707Z 4bdf41c9ecbe: Pulling fs layer 2024-06-26T07:07:11.8540139Z f4863801e30c: Pulling fs layer 2024-06-26T07:07:11.8540515Z d37dd9ed8948: Pulling fs layer 2024-06-26T07:07:11.8540886Z cf2f34f225db: Pulling fs layer 2024-06-26T07:07:11.8541308Z df546d7b3e34: Pulling fs layer 2024-06-26T07:07:11.8542084Z 3d914950f8ad: Pulling fs layer 2024-06-26T07:07:11.8542703Z 96d0e2032184: Pulling fs layer 2024-06-26T07:07:11.8543297Z 06e12ce59790: Pulling fs layer 2024-06-26T07:07:11.8543842Z ed712f63cde7: Pulling fs layer 2024-06-26T07:07:11.8544386Z a97a71ef8a4e: Pulling fs layer 2024-06-26T07:07:11.8544952Z 25cd0c482ad6: Pulling fs layer 2024-06-26T07:07:11.8545482Z d527cbbb87e3: Waiting 2024-06-26T07:07:11.8545922Z 9bf94bc45658: Pulling fs layer 2024-06-26T07:07:11.8546301Z de45a14c9b22: Pulling fs layer 2024-06-26T07:07:11.8546670Z e99241e77209: Pulling fs layer 2024-06-26T07:07:11.8547058Z 35e8f4e2a42f: Pulling fs layer 2024-06-26T07:07:11.8547495Z a823ffa4c541: Pulling fs layer 2024-06-26T07:07:11.8547910Z 44ba272eae20: Pulling fs layer 2024-06-26T07:07:11.8548278Z da02d278e524: Pulling fs layer 2024-06-26T07:07:11.8548699Z 73a0ce681c29: Pulling fs layer 2024-06-26T07:07:11.8551960Z 49737bd5ae70: Pulling fs layer 2024-06-26T07:07:11.8552515Z 9cd4c745f8e4: Pulling fs layer 2024-06-26T07:07:11.8553025Z b2950a036534: Pulling fs layer 2024-06-26T07:07:11.8553554Z 25e19d81d9fc: Pulling fs layer 2024-06-26T07:07:11.8554041Z 60156b40fcbe: Pulling fs layer 2024-06-26T07:07:11.8554482Z 7a868239227f: Pulling fs layer 2024-06-26T07:07:11.8554977Z b57676e46aee: Waiting 2024-06-26T07:07:11.8555382Z a8c1e85b5e14: Waiting 2024-06-26T07:07:11.8555836Z 83911d7c7ecf: Pulling fs layer 2024-06-26T07:07:11.8556238Z b3c5d66c679f: Pulling fs layer 2024-06-26T07:07:11.8556583Z 96d0e2032184: Waiting 2024-06-26T07:07:11.8556910Z 84b2c7a89982: Pulling fs layer 2024-06-26T07:07:11.8557332Z c1068d03a811: Pulling fs layer 2024-06-26T07:07:11.8557723Z a41a8d1c11c8: Waiting 2024-06-26T07:07:11.8558046Z 671b4a301d81: Pulling fs layer 2024-06-26T07:07:11.8558421Z 16d76a79d5d6: Waiting 2024-06-26T07:07:11.8558894Z 67760cf5ec46: Pulling fs layer 2024-06-26T07:07:11.8559391Z ddc9740bcea2: Pulling fs layer 2024-06-26T07:07:11.8559874Z 0c1227890755: Waiting 2024-06-26T07:07:11.8560338Z fb2e97768560: Pulling fs layer 2024-06-26T07:07:11.8560847Z 4bdf41c9ecbe: Waiting 2024-06-26T07:07:11.8561298Z 06e12ce59790: Waiting 2024-06-26T07:07:11.8561764Z b599c5a8c7cb: Pulling fs layer 2024-06-26T07:07:11.8562272Z e08e88c13668: Pulling fs layer 2024-06-26T07:07:11.8562734Z 12ba7431ff38: Pulling fs layer 2024-06-26T07:07:11.8563249Z cf7c2cb896cb: Pulling fs layer 2024-06-26T07:07:11.8563758Z 3a778ddfac2a: Pulling fs layer 2024-06-26T07:07:11.8564264Z 236c8f0245f4: Pulling fs layer 2024-06-26T07:07:11.8564738Z a823ffa4c541: Waiting 2024-06-26T07:07:11.8565084Z 4f4fb700ef54: Pulling fs layer 2024-06-26T07:07:11.8565455Z dda1ce302b9a: Pulling fs layer 2024-06-26T07:07:11.8565835Z e797fe86caef: Pulling fs layer 2024-06-26T07:07:11.8566249Z d8d1234baab3: Waiting 2024-06-26T07:07:11.8566578Z 967190076fc0: Pulling fs layer 2024-06-26T07:07:11.8566929Z 7ed32bc8e469: Waiting 2024-06-26T07:07:11.8567279Z 4fb0a83f0685: Pulling fs layer 2024-06-26T07:07:11.8567625Z 44ba272eae20: Waiting 2024-06-26T07:07:11.8568104Z 68af2ec4e30a: Pulling fs layer 2024-06-26T07:07:11.8568484Z cc91933e985a: Pulling fs layer 2024-06-26T07:07:11.8568848Z 9f3187916e7d: Pulling fs layer 2024-06-26T07:07:11.8569190Z ec1e7978c1fe: Waiting 2024-06-26T07:07:11.8569517Z 526083994857: Pulling fs layer 2024-06-26T07:07:11.8569863Z de45a14c9b22: Waiting 2024-06-26T07:07:11.8570183Z 05a6f6dcc1c0: Pulling fs layer 2024-06-26T07:07:11.8570553Z 6bc9dc8315b0: Pulling fs layer 2024-06-26T07:07:11.8571066Z 309b872da805: Waiting 2024-06-26T07:07:11.8571438Z 1b0e27c35f07: Pulling fs layer 2024-06-26T07:07:11.8571796Z 95237e7311a1: Pulling fs layer 2024-06-26T07:07:11.8572419Z 789eff6c51aa: Pulling fs layer 2024-06-26T07:07:11.8572804Z de0f8a8c45c6: Pulling fs layer 2024-06-26T07:07:11.8573155Z 35e8f4e2a42f: Waiting 2024-06-26T07:07:11.8573502Z a97a71ef8a4e: Waiting 2024-06-26T07:07:11.8573830Z 2060028f7488: Pulling fs layer 2024-06-26T07:07:11.8574197Z e412c7f3b2de: Pulling fs layer 2024-06-26T07:07:11.8574541Z 25cd0c482ad6: Waiting 2024-06-26T07:07:11.8574859Z f4863801e30c: Waiting 2024-06-26T07:07:11.8575287Z 68af2ec4e30a: Waiting 2024-06-26T07:07:11.8575597Z 3d914950f8ad: Waiting 2024-06-26T07:07:11.8575900Z d37dd9ed8948: Waiting 2024-06-26T07:07:11.8576209Z cc91933e985a: Waiting 2024-06-26T07:07:11.8576515Z 7a868239227f: Waiting 2024-06-26T07:07:11.8576858Z e99241e77209: Waiting 2024-06-26T07:07:11.8577168Z 9bf94bc45658: Waiting 2024-06-26T07:07:11.8577475Z cf2f34f225db: Waiting 2024-06-26T07:07:11.8577783Z 9f3187916e7d: Waiting 2024-06-26T07:07:11.8578083Z df546d7b3e34: Waiting 2024-06-26T07:07:11.8578402Z cf7c2cb896cb: Waiting 2024-06-26T07:07:11.8578708Z 2060028f7488: Waiting 2024-06-26T07:07:11.8579014Z da02d278e524: Waiting 2024-06-26T07:07:11.8579312Z 526083994857: Waiting 2024-06-26T07:07:11.8579609Z 95237e7311a1: Waiting 2024-06-26T07:07:11.8579921Z 05a6f6dcc1c0: Waiting 2024-06-26T07:07:11.8580343Z 6bc9dc8315b0: Waiting 2024-06-26T07:07:11.8580780Z 1b0e27c35f07: Waiting 2024-06-26T07:07:11.8581244Z e412c7f3b2de: Waiting 2024-06-26T07:07:11.8581713Z 4f4fb700ef54: Waiting 2024-06-26T07:07:11.8582170Z e797fe86caef: Waiting 2024-06-26T07:07:11.8582619Z 3a778ddfac2a: Waiting 2024-06-26T07:07:11.8583067Z ddc9740bcea2: Waiting 2024-06-26T07:07:11.8583511Z de0f8a8c45c6: Waiting 2024-06-26T07:07:11.8583955Z ca9dd73e3e93: Waiting 2024-06-26T07:07:11.8584393Z b3c5d66c679f: Waiting 2024-06-26T07:07:11.8584830Z 967190076fc0: Waiting 2024-06-26T07:07:11.8585252Z fb2e97768560: Waiting 2024-06-26T07:07:11.8585686Z 84b2c7a89982: Waiting 2024-06-26T07:07:11.8586126Z b2950a036534: Waiting 2024-06-26T07:07:11.8586569Z 4fb0a83f0685: Waiting 2024-06-26T07:07:11.8587003Z 25e19d81d9fc: Waiting 2024-06-26T07:07:11.8587451Z b599c5a8c7cb: Waiting 2024-06-26T07:07:11.8587894Z dda1ce302b9a: Waiting 2024-06-26T07:07:11.8588326Z 49737bd5ae70: Waiting 2024-06-26T07:07:11.8588758Z 12ba7431ff38: Waiting 2024-06-26T07:07:11.8589194Z 60156b40fcbe: Waiting 2024-06-26T07:07:11.8589636Z 9cd4c745f8e4: Waiting 2024-06-26T07:07:11.8590067Z 671b4a301d81: Waiting 2024-06-26T07:07:11.8590514Z 236c8f0245f4: Waiting 2024-06-26T07:07:11.8590967Z e08e88c13668: Waiting 2024-06-26T07:07:11.9807373Z 224fe954d725: Verifying Checksum 2024-06-26T07:07:11.9807842Z 224fe954d725: Download complete 2024-06-26T07:07:12.0611711Z d527cbbb87e3: Verifying Checksum 2024-06-26T07:07:12.1396911Z b57676e46aee: Download complete 2024-06-26T07:07:12.1943480Z 7a2c55901189: Verifying Checksum 2024-06-26T07:07:12.1943932Z 7a2c55901189: Download complete 2024-06-26T07:07:12.2737081Z a41a8d1c11c8: Verifying Checksum 2024-06-26T07:07:12.2737564Z a41a8d1c11c8: Download complete 2024-06-26T07:07:12.3446775Z 0c1227890755: Download complete 2024-06-26T07:07:12.4216973Z d8d1234baab3: Download complete 2024-06-26T07:07:12.4714746Z 75722010b82e: Verifying Checksum 2024-06-26T07:07:12.4715212Z 75722010b82e: Download complete 2024-06-26T07:07:12.5501935Z ec1e7978c1fe: Download complete 2024-06-26T07:07:12.6130756Z 309b872da805: Download complete 2024-06-26T07:07:13.2042079Z 7a2c55901189: Pull complete 2024-06-26T07:07:13.5005033Z 224fe954d725: Pull complete 2024-06-26T07:07:14.4341183Z 75722010b82e: Pull complete 2024-06-26T07:07:14.4695610Z d527cbbb87e3: Pull complete 2024-06-26T07:07:14.5010098Z b57676e46aee: Pull complete 2024-06-26T07:07:15.1589323Z e82932cb002c: Download complete 2024-06-26T07:07:15.2351383Z ca9dd73e3e93: Download complete 2024-06-26T07:07:15.3137079Z 16d76a79d5d6: Download complete 2024-06-26T07:07:15.4013103Z 4bdf41c9ecbe: Verifying Checksum 2024-06-26T07:07:15.4013566Z 4bdf41c9ecbe: Download complete 2024-06-26T07:07:16.3738868Z f4863801e30c: Verifying Checksum 2024-06-26T07:07:16.3739455Z f4863801e30c: Download complete 2024-06-26T07:07:16.4724659Z d37dd9ed8948: Verifying Checksum 2024-06-26T07:07:16.4725210Z d37dd9ed8948: Download complete 2024-06-26T07:07:16.5441282Z cf2f34f225db: Download complete 2024-06-26T07:07:16.6158940Z df546d7b3e34: Verifying Checksum 2024-06-26T07:07:16.6159625Z df546d7b3e34: Download complete 2024-06-26T07:07:25.4863164Z a8c1e85b5e14: Verifying Checksum 2024-06-26T07:07:25.4863699Z a8c1e85b5e14: Download complete 2024-06-26T07:07:25.5606318Z 96d0e2032184: Verifying Checksum 2024-06-26T07:07:25.5606763Z 96d0e2032184: Download complete 2024-06-26T07:07:25.6390722Z 06e12ce59790: Download complete 2024-06-26T07:07:25.7077131Z ed712f63cde7: Verifying Checksum 2024-06-26T07:07:25.7077642Z ed712f63cde7: Download complete 2024-06-26T07:07:25.7804118Z a97a71ef8a4e: Download complete 2024-06-26T07:07:25.8539834Z 25cd0c482ad6: Verifying Checksum 2024-06-26T07:07:25.8540446Z 25cd0c482ad6: Download complete 2024-06-26T07:07:27.1311562Z 9bf94bc45658: Verifying Checksum 2024-06-26T07:07:27.1312254Z 9bf94bc45658: Download complete 2024-06-26T07:07:27.2086631Z de45a14c9b22: Verifying Checksum 2024-06-26T07:07:27.2087067Z de45a14c9b22: Download complete 2024-06-26T07:07:27.2942428Z e99241e77209: Download complete 2024-06-26T07:07:27.3733341Z 35e8f4e2a42f: Download complete 2024-06-26T07:07:27.4525053Z a823ffa4c541: Download complete 2024-06-26T07:07:27.5182044Z 44ba272eae20: Verifying Checksum 2024-06-26T07:07:27.5182496Z 44ba272eae20: Download complete 2024-06-26T07:07:32.0354274Z da02d278e524: Verifying Checksum 2024-06-26T07:07:32.0354759Z da02d278e524: Download complete 2024-06-26T07:07:32.1052569Z 73a0ce681c29: Verifying Checksum 2024-06-26T07:07:32.1053243Z 73a0ce681c29: Download complete 2024-06-26T07:07:32.1807399Z 49737bd5ae70: Verifying Checksum 2024-06-26T07:07:32.1807951Z 49737bd5ae70: Download complete 2024-06-26T07:07:32.5976309Z 9cd4c745f8e4: Verifying Checksum 2024-06-26T07:07:32.5976768Z 9cd4c745f8e4: Download complete 2024-06-26T07:07:32.6798905Z b2950a036534: Download complete 2024-06-26T07:07:32.7527191Z 25e19d81d9fc: Download complete 2024-06-26T07:07:33.0549425Z 60156b40fcbe: Verifying Checksum 2024-06-26T07:07:33.0551368Z 60156b40fcbe: Download complete 2024-06-26T07:07:33.1273445Z 7a868239227f: Verifying Checksum 2024-06-26T07:07:33.1274114Z 7a868239227f: Download complete 2024-06-26T07:07:33.2202296Z 83911d7c7ecf: Download complete 2024-06-26T07:07:33.2916940Z b3c5d66c679f: Download complete 2024-06-26T07:07:37.7580149Z 7ed32bc8e469: Verifying Checksum 2024-06-26T07:07:37.7580663Z 7ed32bc8e469: Download complete 2024-06-26T07:07:37.8365918Z c1068d03a811: Download complete 2024-06-26T07:07:37.9127263Z 671b4a301d81: Verifying Checksum 2024-06-26T07:07:37.9127763Z 671b4a301d81: Download complete 2024-06-26T07:07:38.0220971Z a8c1e85b5e14: Pull complete 2024-06-26T07:07:38.2585767Z a41a8d1c11c8: Pull complete 2024-06-26T07:07:38.4025624Z 67760cf5ec46: Verifying Checksum 2024-06-26T07:07:38.4026116Z 67760cf5ec46: Download complete 2024-06-26T07:07:38.4751246Z ddc9740bcea2: Verifying Checksum 2024-06-26T07:07:38.4751749Z ddc9740bcea2: Download complete 2024-06-26T07:07:38.5113747Z 0c1227890755: Pull complete 2024-06-26T07:07:38.5494588Z fb2e97768560: Verifying Checksum 2024-06-26T07:07:38.5495018Z fb2e97768560: Download complete 2024-06-26T07:07:38.6320055Z b599c5a8c7cb: Verifying Checksum 2024-06-26T07:07:38.6320574Z b599c5a8c7cb: Download complete 2024-06-26T07:07:38.7434346Z e08e88c13668: Verifying Checksum 2024-06-26T07:07:38.7434885Z e08e88c13668: Download complete 2024-06-26T07:07:38.7736110Z d8d1234baab3: Pull complete 2024-06-26T07:07:44.5106634Z 3d914950f8ad: Verifying Checksum 2024-06-26T07:07:44.5107099Z 3d914950f8ad: Download complete 2024-06-26T07:07:44.5872491Z cf7c2cb896cb: Verifying Checksum 2024-06-26T07:07:44.5873093Z cf7c2cb896cb: Download complete 2024-06-26T07:07:44.6761213Z 3a778ddfac2a: Verifying Checksum 2024-06-26T07:07:44.6761685Z 3a778ddfac2a: Download complete 2024-06-26T07:07:44.7471303Z 236c8f0245f4: Verifying Checksum 2024-06-26T07:07:44.7471756Z 236c8f0245f4: Download complete 2024-06-26T07:07:44.7563312Z 4f4fb700ef54: Download complete 2024-06-26T07:07:44.8272206Z dda1ce302b9a: Verifying Checksum 2024-06-26T07:07:44.8272757Z dda1ce302b9a: Download complete 2024-06-26T07:07:44.9011185Z e797fe86caef: Download complete 2024-06-26T07:07:45.0303163Z 967190076fc0: Verifying Checksum 2024-06-26T07:07:45.0303623Z 967190076fc0: Download complete 2024-06-26T07:07:45.1110668Z 4fb0a83f0685: Verifying Checksum 2024-06-26T07:07:45.1111351Z 4fb0a83f0685: Download complete 2024-06-26T07:07:45.1932964Z 68af2ec4e30a: Verifying Checksum 2024-06-26T07:07:45.1933405Z 68af2ec4e30a: Download complete 2024-06-26T07:07:45.2736428Z cc91933e985a: Download complete 2024-06-26T07:07:45.3550311Z 9f3187916e7d: Download complete 2024-06-26T07:07:45.4851503Z 526083994857: Verifying Checksum 2024-06-26T07:07:45.4852036Z 526083994857: Download complete 2024-06-26T07:07:45.5679283Z 05a6f6dcc1c0: Verifying Checksum 2024-06-26T07:07:45.5679806Z 05a6f6dcc1c0: Download complete 2024-06-26T07:07:46.1695283Z 6bc9dc8315b0: Verifying Checksum 2024-06-26T07:07:46.1695755Z 6bc9dc8315b0: Download complete 2024-06-26T07:07:46.2611313Z 1b0e27c35f07: Verifying Checksum 2024-06-26T07:07:46.2611888Z 1b0e27c35f07: Download complete 2024-06-26T07:08:10.0611343Z 12ba7431ff38: Verifying Checksum 2024-06-26T07:08:10.0611970Z 12ba7431ff38: Download complete 2024-06-26T07:08:10.1331500Z 789eff6c51aa: Verifying Checksum 2024-06-26T07:08:10.1331998Z 789eff6c51aa: Download complete 2024-06-26T07:08:10.1993470Z de0f8a8c45c6: Download complete 2024-06-26T07:08:11.5480027Z 2060028f7488: Verifying Checksum 2024-06-26T07:08:11.5480488Z 2060028f7488: Download complete 2024-06-26T07:08:11.6134627Z e412c7f3b2de: Verifying Checksum 2024-06-26T07:08:11.6135069Z e412c7f3b2de: Download complete 2024-06-26T07:08:14.8686194Z 95237e7311a1: Verifying Checksum 2024-06-26T07:08:14.8687051Z 95237e7311a1: Download complete 2024-06-26T07:08:37.2742954Z 84b2c7a89982: Verifying Checksum 2024-06-26T07:08:37.2743420Z 84b2c7a89982: Download complete 2024-06-26T07:08:54.8351903Z 7ed32bc8e469: Pull complete 2024-06-26T07:08:55.0557707Z ec1e7978c1fe: Pull complete 2024-06-26T07:08:55.2980780Z 309b872da805: Pull complete 2024-06-26T07:09:02.6492525Z e82932cb002c: Pull complete 2024-06-26T07:09:02.8684637Z ca9dd73e3e93: Pull complete 2024-06-26T07:09:03.0928606Z 16d76a79d5d6: Pull complete 2024-06-26T07:09:03.3275414Z 4bdf41c9ecbe: Pull complete 2024-06-26T07:09:05.7336153Z f4863801e30c: Pull complete 2024-06-26T07:09:05.9691933Z d37dd9ed8948: Pull complete 2024-06-26T07:09:06.2149336Z cf2f34f225db: Pull complete 2024-06-26T07:09:06.4395569Z df546d7b3e34: Pull complete 2024-06-26T07:09:55.7283017Z 3d914950f8ad: Pull complete 2024-06-26T07:09:55.8327005Z 96d0e2032184: Pull complete 2024-06-26T07:09:55.9420877Z 06e12ce59790: Pull complete 2024-06-26T07:09:56.0642401Z ed712f63cde7: Pull complete 2024-06-26T07:09:56.1592040Z a97a71ef8a4e: Pull complete 2024-06-26T07:09:56.3542314Z 25cd0c482ad6: Pull complete 2024-06-26T07:09:59.9346571Z 9bf94bc45658: Pull complete 2024-06-26T07:10:00.1612802Z de45a14c9b22: Pull complete 2024-06-26T07:10:00.3972469Z e99241e77209: Pull complete 2024-06-26T07:10:00.5855095Z 35e8f4e2a42f: Pull complete 2024-06-26T07:10:00.7243316Z a823ffa4c541: Pull complete 2024-06-26T07:10:00.8621694Z 44ba272eae20: Pull complete 2024-06-26T07:10:09.0085444Z da02d278e524: Pull complete 2024-06-26T07:10:09.1694943Z 73a0ce681c29: Pull complete 2024-06-26T07:10:09.3938252Z 49737bd5ae70: Pull complete 2024-06-26T07:10:10.3575180Z 9cd4c745f8e4: Pull complete 2024-06-26T07:10:10.5730232Z b2950a036534: Pull complete 2024-06-26T07:10:10.7354106Z 25e19d81d9fc: Pull complete 2024-06-26T07:10:11.1782960Z 60156b40fcbe: Pull complete 2024-06-26T07:10:11.3421947Z 7a868239227f: Pull complete 2024-06-26T07:10:11.7770174Z 83911d7c7ecf: Pull complete 2024-06-26T07:10:12.0128885Z b3c5d66c679f: Pull complete 2024-06-26T07:10:50.8616282Z 84b2c7a89982: Pull complete 2024-06-26T07:10:51.0958527Z c1068d03a811: Pull complete 2024-06-26T07:10:51.1734593Z 671b4a301d81: Pull complete 2024-06-26T07:10:53.1141622Z 67760cf5ec46: Pull complete 2024-06-26T07:10:53.1615716Z ddc9740bcea2: Pull complete 2024-06-26T07:10:53.1977239Z fb2e97768560: Pull complete 2024-06-26T07:10:53.3009456Z b599c5a8c7cb: Pull complete 2024-06-26T07:10:53.3448428Z e08e88c13668: Pull complete 2024-06-26T07:11:26.9961900Z 12ba7431ff38: Pull complete 2024-06-26T07:11:27.1878088Z cf7c2cb896cb: Pull complete 2024-06-26T07:11:27.3699341Z 3a778ddfac2a: Pull complete 2024-06-26T07:11:27.8159047Z 236c8f0245f4: Pull complete 2024-06-26T07:11:27.9630913Z 4f4fb700ef54: Pull complete 2024-06-26T07:11:28.1491098Z dda1ce302b9a: Pull complete 2024-06-26T07:11:28.3930297Z e797fe86caef: Pull complete 2024-06-26T07:11:28.7774485Z 967190076fc0: Pull complete 2024-06-26T07:11:28.9978661Z 4fb0a83f0685: Pull complete 2024-06-26T07:11:29.2322868Z 68af2ec4e30a: Pull complete 2024-06-26T07:11:29.4658384Z cc91933e985a: Pull complete 2024-06-26T07:11:29.6851532Z 9f3187916e7d: Pull complete 2024-06-26T07:11:30.7076953Z 526083994857: Pull complete 2024-06-26T07:11:30.8722461Z 05a6f6dcc1c0: Pull complete 2024-06-26T07:11:32.9463536Z 6bc9dc8315b0: Pull complete 2024-06-26T07:11:33.1699236Z 1b0e27c35f07: Pull complete 2024-06-26T07:11:46.7084086Z 95237e7311a1: Pull complete 2024-06-26T07:11:46.7691453Z 789eff6c51aa: Pull complete 2024-06-26T07:11:46.8346902Z de0f8a8c45c6: Pull complete 2024-06-26T07:11:47.5552350Z 2060028f7488: Pull complete 2024-06-26T07:11:47.5966777Z e412c7f3b2de: Pull complete 2024-06-26T07:11:47.9304916Z Digest: sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb 2024-06-26T07:11:47.9333096Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:11:47.9363111Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:11:47.9414710Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:11:47.9415651Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:11:47.9423851Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:47.9424348Z env: 2024-06-26T07:11:47.9424624Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:47.9424968Z ##[endgroup] 2024-06-26T07:11:47.9581635Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-06-26T07:11:47.9582181Z with: 2024-06-26T07:11:47.9582467Z driver-version: 550.54.15 2024-06-26T07:11:47.9582828Z env: 2024-06-26T07:11:47.9583099Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:47.9583446Z ##[endgroup] 2024-06-26T07:11:47.9622211Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:11:47.9622762Z with: 2024-06-26T07:11:47.9623037Z timeout_minutes: 10 2024-06-26T07:11:47.9623353Z max_attempts: 3 2024-06-26T07:11:47.9654900Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" 2024-06-26T07:11:47.9686525Z retry_wait_seconds: 10 2024-06-26T07:11:47.9686880Z polling_interval_seconds: 1 2024-06-26T07:11:47.9687242Z warning_on_retry: true 2024-06-26T07:11:47.9687594Z continue_on_error: false 2024-06-26T07:11:47.9687973Z env: 2024-06-26T07:11:47.9688238Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:47.9688644Z DRIVER_VERSION: 550.54.15 2024-06-26T07:11:47.9688989Z ##[endgroup] 2024-06-26T07:11:48.0216151Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-06-26T07:11:48.0216888Z + pre_install_nvidia_driver_amzn2 2024-06-26T07:11:48.0217424Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-06-26T07:11:48.3242976Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:11:48.3629106Z No Match for argument: nvidia-driver-latest-dkms 2024-06-26T07:11:48.3865851Z No Packages marked for removal 2024-06-26T07:11:48.3982006Z + install_nvidia_driver_common 2024-06-26T07:11:48.3984971Z + echo 'Before installing NVIDIA driver' 2024-06-26T07:11:48.3985671Z + lspci 2024-06-26T07:11:48.3987738Z Before installing NVIDIA driver 2024-06-26T07:11:48.4066755Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-06-26T07:11:48.4067895Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T07:11:48.4068863Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-06-26T07:11:48.4069687Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-06-26T07:11:48.4070444Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 2024-06-26T07:11:48.4071207Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T07:11:48.4071938Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1) 2024-06-26T07:11:48.4072732Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-06-26T07:11:48.4073325Z + lsmod 2024-06-26T07:11:48.4084954Z Module Size Used by 2024-06-26T07:11:48.4085593Z xt_nat 16384 0 2024-06-26T07:11:48.4086020Z nvidia_modeset 1339392 0 2024-06-26T07:11:48.4086409Z nvidia_uvm 4599808 0 2024-06-26T07:11:48.4086863Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:11:48.4087334Z drm 421888 1 nvidia 2024-06-26T07:11:48.4088086Z i2c_core 77824 2 nvidia,drm 2024-06-26T07:11:48.4088658Z veth 16384 0 2024-06-26T07:11:48.4089078Z backlight 16384 1 nvidia_modeset 2024-06-26T07:11:48.4089510Z xt_conntrack 16384 1 2024-06-26T07:11:48.4089890Z ipt_MASQUERADE 16384 1 2024-06-26T07:11:48.4090328Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T07:11:48.4090920Z nf_conntrack_netlink 49152 0 2024-06-26T07:11:48.4091543Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T07:11:48.4092508Z xfrm_user 45056 1 2024-06-26T07:11:48.4093034Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:11:48.4093455Z iptable_nat 16384 1 2024-06-26T07:11:48.4093827Z nf_conntrack_ipv4 16384 3 2024-06-26T07:11:48.4094282Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T07:11:48.4094996Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T07:11:48.4095806Z nf_nat 32768 3 nf_nat_masquerade_ipv4,nf_nat_ipv4,xt_nat 2024-06-26T07:11:48.4097187Z nf_conntrack 155648 8 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,xt_nat,nf_conntrack_netlink 2024-06-26T07:11:48.4098240Z xt_addrtype 16384 2 2024-06-26T07:11:48.4098826Z iptable_filter 16384 1 2024-06-26T07:11:48.4099348Z br_netfilter 24576 0 2024-06-26T07:11:48.4099765Z bridge 172032 1 br_netfilter 2024-06-26T07:11:48.4100208Z stp 16384 1 bridge 2024-06-26T07:11:48.4100634Z llc 16384 2 bridge,stp 2024-06-26T07:11:48.4101054Z overlay 86016 0 2024-06-26T07:11:48.4101427Z sunrpc 393216 1 2024-06-26T07:11:48.4101808Z dm_mirror 28672 0 2024-06-26T07:11:48.4102206Z dm_region_hash 20480 1 dm_mirror 2024-06-26T07:11:48.4102705Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T07:11:48.4103222Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T07:11:48.4103828Z dax 69632 1 dm_mod 2024-06-26T07:11:48.4104352Z crc32_pclmul 16384 0 2024-06-26T07:11:48.4104737Z ghash_clmulni_intel 16384 0 2024-06-26T07:11:48.4105164Z pcbc 16384 0 2024-06-26T07:11:48.4105545Z mousedev 24576 0 2024-06-26T07:11:48.4105913Z aesni_intel 188416 0 2024-06-26T07:11:48.4106297Z aes_x86_64 20480 1 aesni_intel 2024-06-26T07:11:48.4106746Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:11:48.4107191Z glue_helper 16384 1 aesni_intel 2024-06-26T07:11:48.4107746Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T07:11:48.4108268Z evdev 20480 3 2024-06-26T07:11:48.4108637Z psmouse 32768 0 2024-06-26T07:11:48.4109004Z button 16384 0 2024-06-26T07:11:48.4109364Z ena 139264 0 2024-06-26T07:11:48.4109718Z ptp 20480 1 ena 2024-06-26T07:11:48.4110125Z pps_core 20480 1 ptp 2024-06-26T07:11:48.4110528Z crc32c_intel 24576 0 2024-06-26T07:11:48.4110896Z autofs4 49152 2 2024-06-26T07:11:48.4111249Z + modinfo nvidia 2024-06-26T07:11:48.4111896Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T07:11:48.4112595Z alias: char-major-195-* 2024-06-26T07:11:48.4112969Z version: 550.54.15 2024-06-26T07:11:48.4113310Z supported: external 2024-06-26T07:11:48.4113654Z license: NVIDIA 2024-06-26T07:11:48.4114036Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:11:48.4114514Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:11:48.4114970Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T07:11:48.4115435Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:11:48.4115931Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:11:48.4116410Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:11:48.4116990Z depends: i2c-core,drm 2024-06-26T07:11:48.4117365Z retpoline: Y 2024-06-26T07:11:48.4117702Z name: nvidia 2024-06-26T07:11:48.4118267Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T07:11:48.4118924Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:11:48.4119578Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:11:48.4120177Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:11:48.4120616Z parm: NVreg_RmLogonRC:int 2024-06-26T07:11:48.4121056Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:11:48.4121514Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:11:48.4121941Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:11:48.4122379Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:11:48.4122913Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:11:48.4123487Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:11:48.4123961Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:11:48.4124390Z parm: NVreg_EnableMSI:int 2024-06-26T07:11:48.4124815Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:11:48.4125267Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:11:48.4125792Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:11:48.4126376Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:11:48.4126934Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:11:48.4127541Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:11:48.4128127Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:11:48.4128732Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:11:48.4129318Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:11:48.4129815Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:11:48.4130345Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:11:48.4131084Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:11:48.4131577Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:11:48.4132041Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:11:48.4132758Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:11:48.4133247Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:11:48.4133691Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:11:48.4134179Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:11:48.4134696Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:11:48.4135170Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:11:48.4135654Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:11:48.4136143Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:11:48.4136626Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:11:48.4137137Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:11:48.4137615Z parm: NVreg_RmMsg:charp 2024-06-26T07:11:48.4138027Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:11:48.4138503Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:11:48.4138973Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:11:48.4139430Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:11:48.4139897Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:11:48.4140375Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:11:48.4140829Z parm: rm_firmware_active:charp 2024-06-26T07:11:48.4141250Z + HAS_NVIDIA_DRIVER=0 2024-06-26T07:11:48.4141618Z ++ command -v nvidia-smi 2024-06-26T07:11:48.4142014Z + '[' -x /usr/bin/nvidia-smi ']' 2024-06-26T07:11:48.4142384Z + set +e 2024-06-26T07:11:48.4142882Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-06-26T07:11:50.4154026Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-06-26T07:11:50.4154640Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:11:50.4155248Z + '[' 0 -ne 0 ']' 2024-06-26T07:11:50.4155985Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-06-26T07:11:50.4156403Z + HAS_NVIDIA_DRIVER=1 2024-06-26T07:11:50.4157191Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-06-26T07:11:50.4158178Z + set -e 2024-06-26T07:11:50.4158624Z + '[' 1 -eq 0 ']' 2024-06-26T07:11:50.4159395Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-06-26T07:11:50.4160365Z + post_install_nvidia_driver_common 2024-06-26T07:11:50.4160952Z + sudo modprobe nvidia 2024-06-26T07:11:50.4255422Z + echo 'After installing NVIDIA driver' 2024-06-26T07:11:50.4256064Z + lspci 2024-06-26T07:11:50.4256491Z After installing NVIDIA driver 2024-06-26T07:11:50.4334635Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-06-26T07:11:50.4335664Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T07:11:50.4336616Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-06-26T07:11:50.4337424Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-06-26T07:11:50.4338194Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 2024-06-26T07:11:50.4338942Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T07:11:50.4339667Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1) 2024-06-26T07:11:50.4340449Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-06-26T07:11:50.4341035Z + lsmod 2024-06-26T07:11:50.4348543Z Module Size Used by 2024-06-26T07:11:50.4349037Z xt_nat 16384 0 2024-06-26T07:11:50.4349485Z nvidia_modeset 1339392 0 2024-06-26T07:11:50.4349975Z nvidia_uvm 4599808 0 2024-06-26T07:11:50.4350598Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:11:50.4351270Z drm 421888 1 nvidia 2024-06-26T07:11:50.4351804Z i2c_core 77824 2 nvidia,drm 2024-06-26T07:11:50.4352442Z veth 16384 0 2024-06-26T07:11:50.4352868Z backlight 16384 1 nvidia_modeset 2024-06-26T07:11:50.4353334Z xt_conntrack 16384 1 2024-06-26T07:11:50.4353735Z ipt_MASQUERADE 16384 1 2024-06-26T07:11:50.4354185Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T07:11:50.4354679Z nf_conntrack_netlink 49152 0 2024-06-26T07:11:50.4355145Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T07:11:50.4355643Z xfrm_user 45056 1 2024-06-26T07:11:50.4356032Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:11:50.4356496Z iptable_nat 16384 1 2024-06-26T07:11:50.4357060Z nf_conntrack_ipv4 16384 3 2024-06-26T07:11:50.4357654Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T07:11:50.4358372Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T07:11:50.4359125Z nf_nat 32768 3 nf_nat_masquerade_ipv4,nf_nat_ipv4,xt_nat 2024-06-26T07:11:50.4360455Z nf_conntrack 155648 8 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,xt_nat,nf_conntrack_netlink 2024-06-26T07:11:50.4361321Z xt_addrtype 16384 2 2024-06-26T07:11:50.4361696Z iptable_filter 16384 1 2024-06-26T07:11:50.4362072Z br_netfilter 24576 0 2024-06-26T07:11:50.4362478Z bridge 172032 1 br_netfilter 2024-06-26T07:11:50.4362917Z stp 16384 1 bridge 2024-06-26T07:11:50.4363341Z llc 16384 2 bridge,stp 2024-06-26T07:11:50.4363757Z overlay 86016 0 2024-06-26T07:11:50.4364122Z sunrpc 393216 1 2024-06-26T07:11:50.4364491Z dm_mirror 28672 0 2024-06-26T07:11:50.4364882Z dm_region_hash 20480 1 dm_mirror 2024-06-26T07:11:50.4365360Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T07:11:50.4365912Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T07:11:50.4366354Z dax 69632 1 dm_mod 2024-06-26T07:11:50.4366985Z crc32_pclmul 16384 0 2024-06-26T07:11:50.4367380Z ghash_clmulni_intel 16384 0 2024-06-26T07:11:50.4367755Z pcbc 16384 0 2024-06-26T07:11:50.4368116Z mousedev 24576 0 2024-06-26T07:11:50.4368476Z aesni_intel 188416 0 2024-06-26T07:11:50.4368865Z aes_x86_64 20480 1 aesni_intel 2024-06-26T07:11:50.4369306Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:11:50.4369760Z glue_helper 16384 1 aesni_intel 2024-06-26T07:11:50.4370297Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T07:11:50.4370949Z evdev 20480 3 2024-06-26T07:11:50.4371319Z psmouse 32768 0 2024-06-26T07:11:50.4371681Z button 16384 0 2024-06-26T07:11:50.4372031Z ena 139264 0 2024-06-26T07:11:50.4372676Z ptp 20480 1 ena 2024-06-26T07:11:50.4373070Z pps_core 20480 1 ptp 2024-06-26T07:11:50.4373466Z crc32c_intel 24576 0 2024-06-26T07:11:50.4373836Z autofs4 49152 2 2024-06-26T07:11:50.4374194Z + modinfo nvidia 2024-06-26T07:11:50.4374852Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T07:11:50.4375536Z alias: char-major-195-* 2024-06-26T07:11:50.4375912Z version: 550.54.15 2024-06-26T07:11:50.4376249Z supported: external 2024-06-26T07:11:50.4376586Z license: NVIDIA 2024-06-26T07:11:50.4376952Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:11:50.4377429Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:11:50.4377882Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T07:11:50.4378337Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:11:50.4378836Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:11:50.4379322Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:11:50.4379808Z depends: i2c-core,drm 2024-06-26T07:11:50.4380171Z retpoline: Y 2024-06-26T07:11:50.4380584Z name: nvidia 2024-06-26T07:11:50.4381136Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T07:11:50.4381784Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:11:50.4382414Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:11:50.4383001Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:11:50.4383439Z parm: NVreg_RmLogonRC:int 2024-06-26T07:11:50.4383866Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:11:50.4384310Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:11:50.4384735Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:11:50.4385165Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:11:50.4385681Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:11:50.4386227Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:11:50.4386698Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:11:50.4387137Z parm: NVreg_EnableMSI:int 2024-06-26T07:11:50.4387548Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:11:50.4388006Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:11:50.4388526Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:11:50.4389158Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:11:50.4389707Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:11:50.4390313Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:11:50.4390904Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:11:50.4391512Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:11:50.4392090Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:11:50.4392582Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:11:50.4393106Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:11:50.4393733Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:11:50.4394233Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:11:50.4394702Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:11:50.4395186Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:11:50.4395652Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:11:50.4396090Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:11:50.4396591Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:11:50.4397113Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:11:50.4397587Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:11:50.4398065Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:11:50.4398565Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:11:50.4399050Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:11:50.4399549Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:11:50.4400021Z parm: NVreg_RmMsg:charp 2024-06-26T07:11:50.4400447Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:11:50.4400928Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:11:50.4401392Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:11:50.4401849Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:11:50.4402332Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:11:50.4402807Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:11:50.4403255Z parm: rm_firmware_active:charp 2024-06-26T07:11:50.4403671Z + set +e 2024-06-26T07:11:50.4403968Z + nvidia-smi 2024-06-26T07:11:51.9914684Z Wed Jun 26 07:11:51 2024 2024-06-26T07:11:51.9915634Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:11:51.9916465Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-06-26T07:11:51.9917268Z |-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:11:51.9918095Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-06-26T07:11:51.9919243Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-06-26T07:11:51.9919940Z | | | MIG M. | 2024-06-26T07:11:51.9920540Z |=========================================+========================+======================| 2024-06-26T07:11:52.0017109Z | 0 NVIDIA A10G Off | 00000000:00:1E.0 Off | 0 | 2024-06-26T07:11:52.0017831Z | 0% 34C P0 59W / 300W | 0MiB / 23028MiB | 0% Default | 2024-06-26T07:11:52.0018473Z | | | N/A | 2024-06-26T07:11:52.0019141Z +-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:11:52.0019845Z 2024-06-26T07:11:52.0020517Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:11:52.0021142Z | Processes: | 2024-06-26T07:11:52.0021845Z | GPU GI CI PID Type Process name GPU Memory | 2024-06-26T07:11:52.0022541Z | ID ID Usage | 2024-06-26T07:11:52.0023112Z |=========================================================================================| 2024-06-26T07:11:52.0023752Z | No running processes found | 2024-06-26T07:11:52.0024500Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:11:52.6209061Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-06-26T07:11:54.1914304Z NVIDIA A10G 2024-06-26T07:11:54.6328454Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:11:54.6329050Z + '[' 0 -eq 0 ']' 2024-06-26T07:11:54.6329440Z + echo 'INFO: Ignoring allowed status 0' 2024-06-26T07:11:54.6329872Z + set -e 2024-06-26T07:11:54.6330180Z INFO: Ignoring allowed status 0 2024-06-26T07:11:54.6331731Z == Installing nvidia container toolkit for amzn2 == 2024-06-26T07:11:54.6334437Z + sudo yum install -y yum-utils 2024-06-26T07:11:54.9342796Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:11:56.5735857Z Package yum-utils-1.1.31-46.amzn2.0.1.noarch already installed and latest version 2024-06-26T07:11:56.5736538Z Nothing to do 2024-06-26T07:11:56.7259099Z + [[ amzn2 == \a\m\z\n\2\0\2\3 ]] 2024-06-26T07:11:56.7259970Z + YUM_REPO_URL=https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:11:56.7261019Z + sudo yum-config-manager --add-repo https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:11:57.0464390Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:11:57.0718765Z adding repo from: https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:11:57.0719964Z grabbing file https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T07:11:57.0720928Z repo saved to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T07:11:57.0835726Z + sudo yum install -y nvidia-docker2 2024-06-26T07:11:57.3806388Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:11:58.8347224Z Package nvidia-docker2-2.13.0-1.noarch already installed and latest version 2024-06-26T07:11:58.8347905Z Nothing to do 2024-06-26T07:11:58.9842162Z + sudo systemctl restart docker 2024-06-26T07:12:39.0746225Z Command completed after 1 attempt(s). 2024-06-26T07:12:39.0803441Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:12:39.0804211Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:12:39.0804904Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-06-26T07:12:39.0805710Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:12:39.0813863Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:39.0814368Z env: 2024-06-26T07:12:39.0814648Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:39.0815103Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:39.0815578Z ##[endgroup] 2024-06-26T07:12:39.2985783Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T07:12:39.3135871Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.7/site-packages (5.9.1) 2024-06-26T07:12:39.3233814Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.7/site-packages (11.525.84) 2024-06-26T07:12:39.4135199Z Prepare all required actions 2024-06-26T07:12:39.4135698Z Getting action download info 2024-06-26T07:12:39.5097637Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-06-26T07:12:39.7600517Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-06-26T07:12:39.8644694Z ##[group]Run ./.github/actions/download-build-artifacts 2024-06-26T07:12:39.8645172Z with: 2024-06-26T07:12:39.8645502Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:12:39.8645949Z s3-bucket: gha-artifacts 2024-06-26T07:12:39.8646275Z env: 2024-06-26T07:12:39.8646543Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:39.8646988Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:39.8647461Z ##[endgroup] 2024-06-26T07:12:39.8678719Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:12:39.8679163Z with: 2024-06-26T07:12:39.8679524Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:12:39.8679986Z s3-bucket: gha-artifacts 2024-06-26T07:12:39.8680381Z region: us-east-1 2024-06-26T07:12:39.8680682Z env: 2024-06-26T07:12:39.8680951Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:39.8681420Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:39.8681901Z ##[endgroup] 2024-06-26T07:12:40.2933917Z (node:12465) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:12:40.2934618Z 2024-06-26T07:12:40.2934892Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:12:40.2935608Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:12:40.2936490Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:12:40.3803798Z Found 1 objects with prefix pytorch/pytorch/9673646806/linux-focal-cuda12.1-py3.10-gcc9-sm86/ 2024-06-26T07:12:40.3804886Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:12:47.9020765Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:12:47.9026582Z Artifact download has finished successfully 2024-06-26T07:12:47.9182102Z ##[group]Run unzip -o artifacts.zip 2024-06-26T07:12:47.9182544Z unzip -o artifacts.zip 2024-06-26T07:12:47.9190125Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:47.9190615Z env: 2024-06-26T07:12:47.9190893Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:47.9191348Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:47.9191832Z ##[endgroup] 2024-06-26T07:12:47.9219850Z Archive: artifacts.zip 2024-06-26T07:12:47.9221388Z creating: dist/ 2024-06-26T07:12:49.8970990Z inflating: dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:12:49.8971856Z creating: build/custom_test_artifacts/ 2024-06-26T07:12:49.8972852Z creating: build/custom_test_artifacts/custom-op-build/ 2024-06-26T07:12:49.8973658Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-06-26T07:12:49.8974508Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:12:49.8979423Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:12:49.8980459Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-06-26T07:12:49.8981398Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:12:49.8982450Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:12:49.8983514Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:12:49.8984772Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:12:49.8986020Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:12:49.8987231Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:12:49.8988475Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:12:49.8989972Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:12:49.8991342Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:12:49.8992806Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:12:49.8993956Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:12:49.8995121Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:12:49.8996297Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:12:49.8997323Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:12:49.8998329Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:12:49.9031834Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:12:49.9069596Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:12:49.9071044Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:12:49.9113871Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:12:49.9115743Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:12:49.9117738Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:12:49.9119657Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:12:49.9121591Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:12:49.9123319Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:12:49.9124864Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:12:49.9126239Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:12:49.9127583Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:12:49.9128854Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:12:49.9130225Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:12:49.9131531Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:12:49.9133151Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:12:49.9134338Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:12:49.9135547Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:12:49.9182490Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:12:49.9240331Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:12:49.9241734Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:12:49.9242836Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:12:49.9243813Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:12:49.9244823Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-06-26T07:12:49.9245779Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-06-26T07:12:49.9247132Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-06-26T07:12:49.9248377Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-06-26T07:12:49.9249491Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-06-26T07:12:49.9250526Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-06-26T07:12:49.9251687Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-06-26T07:12:49.9253175Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-06-26T07:12:49.9254271Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-06-26T07:12:49.9255356Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-06-26T07:12:49.9256422Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-06-26T07:12:49.9268561Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-06-26T07:12:49.9396895Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-06-26T07:12:49.9398176Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-06-26T07:12:49.9399517Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-06-26T07:12:49.9401194Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-06-26T07:12:49.9402399Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-06-26T07:12:49.9403490Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-06-26T07:12:49.9404632Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-06-26T07:12:49.9405764Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-06-26T07:12:49.9406909Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-06-26T07:12:49.9408038Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-06-26T07:12:49.9409283Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-06-26T07:12:49.9421707Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-06-26T07:12:49.9501187Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-06-26T07:12:49.9502556Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:12:49.9503744Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:12:49.9504948Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-06-26T07:12:49.9505866Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-06-26T07:12:49.9507024Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-06-26T07:12:49.9508217Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-06-26T07:12:49.9509070Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-06-26T07:12:49.9509856Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-06-26T07:12:49.9510656Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-06-26T07:12:49.9616314Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-06-26T07:12:49.9677363Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-06-26T07:12:49.9679064Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-06-26T07:12:49.9680471Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-06-26T07:12:49.9682080Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:12:49.9685171Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:12:49.9687186Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-06-26T07:12:49.9688379Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:12:49.9689496Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:12:49.9690465Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:12:49.9691838Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:12:49.9693395Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:12:49.9694679Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:12:49.9695767Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:12:49.9697340Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:12:49.9698502Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:12:49.9699632Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:12:49.9700762Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:12:49.9701905Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:12:49.9703059Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:12:49.9704090Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:12:49.9705082Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:12:49.9736186Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:12:49.9773914Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:12:49.9775363Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:12:49.9817761Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:12:49.9819567Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:12:49.9821589Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:12:49.9823570Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:12:49.9825304Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:12:49.9826929Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:12:49.9828314Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:12:49.9829683Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:12:49.9831023Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:12:49.9832294Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:12:49.9833519Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:12:49.9834729Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:12:49.9835944Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:12:49.9837120Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:12:49.9838322Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:12:49.9884264Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:12:49.9942094Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:12:49.9943713Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:12:49.9945018Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:12:49.9946182Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:12:49.9947471Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-06-26T07:12:49.9948528Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-06-26T07:12:49.9949582Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-06-26T07:12:49.9950765Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-06-26T07:12:49.9951907Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-06-26T07:12:49.9952977Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-06-26T07:12:49.9954200Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-06-26T07:12:49.9955297Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-06-26T07:12:49.9956410Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-06-26T07:12:49.9957513Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-06-26T07:12:49.9958604Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-06-26T07:12:49.9967957Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-06-26T07:12:50.0029413Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-06-26T07:12:50.0030886Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:12:50.0032194Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:12:50.0033206Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-06-26T07:12:50.0034439Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-06-26T07:12:50.0035456Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-06-26T07:12:50.0036345Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-06-26T07:12:50.0037230Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-06-26T07:12:50.0037993Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-06-26T07:12:50.0038790Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-06-26T07:12:50.0083703Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-06-26T07:12:50.0085570Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-06-26T07:12:50.0086799Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-06-26T07:12:50.0087675Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:12:50.0090307Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:12:50.0091595Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-06-26T07:12:50.0093047Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:12:50.0094375Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:12:50.0095673Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:12:50.0096837Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:12:50.0098296Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:12:50.0099573Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:12:50.0100971Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:12:50.0102378Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:12:50.0103886Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:12:50.0105081Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:12:50.0122764Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:12:50.0124072Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:12:50.0125501Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:12:50.0126618Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:12:50.0127695Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:12:50.0142319Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:12:50.0180025Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:12:50.0224273Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:12:50.0226139Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:12:50.0227630Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:12:50.0229130Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:12:50.0230660Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:12:50.0232140Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:12:50.0243509Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:12:50.0244997Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:12:50.0246465Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:12:50.0247873Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:12:50.0249221Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:12:50.0250523Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:12:50.0251883Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:12:50.0253625Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:12:50.0255057Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:12:50.0256519Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:12:50.0307093Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:12:50.0365858Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:12:50.0367406Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:12:50.0368448Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:12:50.0369366Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:12:50.0370339Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-06-26T07:12:50.0371422Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-06-26T07:12:50.0373537Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-06-26T07:12:50.0375006Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-06-26T07:12:50.0376220Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-06-26T07:12:50.0377355Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-06-26T07:12:50.0378529Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-06-26T07:12:50.0379940Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-06-26T07:12:50.0381121Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-06-26T07:12:50.0382310Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-06-26T07:12:50.0383477Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-06-26T07:12:50.0384724Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-06-26T07:12:50.0496151Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-06-26T07:12:50.0497637Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-06-26T07:12:50.0498830Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-06-26T07:12:50.0500154Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-06-26T07:12:50.0501414Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-06-26T07:12:50.0502609Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-06-26T07:12:50.0503829Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-06-26T07:12:50.0505073Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-06-26T07:12:50.0506305Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-06-26T07:12:50.0507535Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-06-26T07:12:50.0508756Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-06-26T07:12:50.0519572Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-06-26T07:12:50.0572811Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-06-26T07:12:50.0574108Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:12:50.0575243Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:12:50.0576263Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-06-26T07:12:50.0577211Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-06-26T07:12:50.0578176Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-06-26T07:12:50.0579125Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-06-26T07:12:50.0580021Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-06-26T07:12:50.0581027Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-06-26T07:12:50.0581881Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-06-26T07:12:50.0679376Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-06-26T07:12:50.0720592Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-06-26T07:12:50.0721228Z creating: build/lib/ 2024-06-26T07:12:50.0806296Z inflating: build/lib/libprotobuf-lite.a 2024-06-26T07:12:50.1244110Z inflating: build/lib/libprotobuf.a 2024-06-26T07:12:50.1253770Z inflating: build/lib/libpthreadpool.a 2024-06-26T07:12:50.1261653Z inflating: build/lib/libcpuinfo.a 2024-06-26T07:12:50.1269343Z inflating: build/lib/libcpuinfo_internals.a 2024-06-26T07:12:50.1270361Z inflating: build/lib/libclog.a 2024-06-26T07:12:50.1287960Z inflating: build/lib/libnnpack.a 2024-06-26T07:12:50.1290237Z inflating: build/lib/libnnpack_reference_layers.a 2024-06-26T07:12:50.1353057Z inflating: build/lib/libgtest.a 2024-06-26T07:12:50.1425483Z inflating: build/lib/libbenchmark.a 2024-06-26T07:12:50.1487490Z inflating: build/lib/libasmjit.a 2024-06-26T07:12:50.1495086Z inflating: build/lib/libittnotify.a 2024-06-26T07:12:50.1523510Z inflating: build/lib/libtensorpipe_uv.a 2024-06-26T07:12:50.1670457Z inflating: build/lib/libgloo.a 2024-06-26T07:12:50.1692721Z inflating: build/lib/libfmt.a 2024-06-26T07:12:50.1781949Z inflating: build/lib/libc10.so 2024-06-26T07:12:50.1783344Z inflating: build/lib/libcaffe2_nvrtc.so 2024-06-26T07:12:50.1783948Z inflating: build/lib/libfoxi_loader.a 2024-06-26T07:12:50.1785426Z inflating: build/lib/libtorch_global_deps.so 2024-06-26T07:12:50.1803896Z inflating: build/lib/libpytorch_qnnpack.a 2024-06-26T07:12:50.1821267Z inflating: build/lib/libgmock.a 2024-06-26T07:12:50.1821705Z inflating: build/lib/libgtest_main.a 2024-06-26T07:12:50.1822193Z inflating: build/lib/libbenchmark_main.a 2024-06-26T07:12:50.2306652Z inflating: build/lib/libprotoc.a 2024-06-26T07:12:50.2676333Z inflating: build/lib/libgloo_cuda.a 2024-06-26T07:12:51.2405780Z inflating: build/lib/libdnnl.a 2024-06-26T07:12:51.2953476Z inflating: build/lib/libtensorpipe.a 2024-06-26T07:12:51.3006126Z inflating: build/lib/libc10_cuda.so 2024-06-26T07:12:51.3007352Z inflating: build/lib/libgmock_main.a 2024-06-26T07:12:51.4206036Z inflating: build/lib/libfbgemm.a 2024-06-26T07:12:51.4449748Z inflating: build/lib/libtensorpipe_cuda.a 2024-06-26T07:12:51.4945911Z inflating: build/lib/libkineto.a 2024-06-26T07:12:51.5122878Z inflating: build/lib/libXNNPACK.a 2024-06-26T07:12:51.5164020Z inflating: build/lib/libonnx_proto.a 2024-06-26T07:12:51.5825923Z inflating: build/lib/libonnx.a 2024-06-26T07:12:53.9095066Z inflating: build/lib/libtorch_cpu.so 2024-06-26T07:12:53.9099326Z inflating: build/lib/libunbox_lib.a 2024-06-26T07:12:53.9103125Z inflating: build/lib/libshm.so 2024-06-26T07:12:55.8904171Z inflating: build/lib/libtorch_cuda.so 2024-06-26T07:12:55.8904996Z inflating: build/lib/libtorch.so 2024-06-26T07:12:56.6826210Z inflating: build/lib/libtorch_cuda_linalg.so 2024-06-26T07:12:56.6827467Z inflating: build/lib/libc10d_cuda_test.so 2024-06-26T07:12:56.8718035Z inflating: build/lib/libtorch_python.so 2024-06-26T07:12:56.8784314Z inflating: build/lib/libtorchbind_test.so 2024-06-26T07:12:56.8804199Z inflating: build/lib/libjitbackend_test.so 2024-06-26T07:12:56.8828987Z inflating: build/lib/libbackend_with_compiler.so 2024-06-26T07:12:56.8862504Z inflating: build/lib/libnnapi_backend.so 2024-06-26T07:12:56.8863150Z creating: build/bin/ 2024-06-26T07:12:56.8912114Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-06-26T07:12:56.8962915Z inflating: build/bin/c10_DeviceGuard_test 2024-06-26T07:12:56.9011818Z inflating: build/bin/c10_Device_test 2024-06-26T07:12:56.9069082Z inflating: build/bin/c10_DispatchKeySet_test 2024-06-26T07:12:56.9121364Z inflating: build/bin/c10_Scalar_test 2024-06-26T07:12:56.9168773Z inflating: build/bin/c10_StreamGuard_test 2024-06-26T07:12:56.9221595Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-06-26T07:12:56.9271057Z inflating: build/bin/c10_SymInt_test 2024-06-26T07:12:56.9324969Z inflating: build/bin/c10_InlineStreamGuard_test 2024-06-26T07:12:56.9379683Z inflating: build/bin/c10_SizesAndStrides_test 2024-06-26T07:12:56.9430963Z inflating: build/bin/c10_Bitset_test 2024-06-26T07:12:56.9500304Z inflating: build/bin/c10_cow_test 2024-06-26T07:12:56.9547166Z inflating: build/bin/c10_ConstexprCrc_test 2024-06-26T07:12:56.9595743Z inflating: build/bin/c10_DeadlockDetection_test 2024-06-26T07:12:56.9644704Z inflating: build/bin/c10_Half_test 2024-06-26T07:12:56.9699847Z inflating: build/bin/c10_LeftRight_test 2024-06-26T07:12:56.9752651Z inflating: build/bin/c10_Metaprogramming_test 2024-06-26T07:12:56.9801260Z inflating: build/bin/c10_Synchronized_test 2024-06-26T07:12:56.9854959Z inflating: build/bin/c10_ThreadLocal_test 2024-06-26T07:12:56.9904943Z inflating: build/bin/c10_TypeIndex_test 2024-06-26T07:12:56.9954279Z inflating: build/bin/c10_TypeList_test 2024-06-26T07:12:57.0002475Z inflating: build/bin/c10_TypeTraits_test 2024-06-26T07:12:57.0052287Z inflating: build/bin/c10_accumulate_test 2024-06-26T07:12:57.0105848Z inflating: build/bin/c10_bfloat16_test 2024-06-26T07:12:57.0154656Z inflating: build/bin/c10_bit_cast_test 2024-06-26T07:12:57.0209824Z inflating: build/bin/c10_complex_math_test 2024-06-26T07:12:57.0263089Z inflating: build/bin/c10_complex_test 2024-06-26T07:12:57.0314661Z inflating: build/bin/c10_exception_test 2024-06-26T07:12:57.0363421Z inflating: build/bin/c10_flags_test 2024-06-26T07:12:57.0411513Z inflating: build/bin/c10_generic_math_test 2024-06-26T07:12:57.0569679Z inflating: build/bin/c10_intrusive_ptr_test 2024-06-26T07:12:57.0618711Z inflating: build/bin/c10_irange_test 2024-06-26T07:12:57.0670559Z inflating: build/bin/c10_lazy_test 2024-06-26T07:12:57.0725881Z inflating: build/bin/c10_logging_test 2024-06-26T07:12:57.0798652Z inflating: build/bin/c10_optional_test 2024-06-26T07:12:57.0858469Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-06-26T07:12:57.0910474Z inflating: build/bin/c10_registry_test 2024-06-26T07:12:57.1055850Z inflating: build/bin/c10_small_vector_test 2024-06-26T07:12:57.1105762Z inflating: build/bin/c10_ssize_test 2024-06-26T07:12:57.1156089Z inflating: build/bin/c10_string_util_test 2024-06-26T07:12:57.1213172Z inflating: build/bin/c10_string_view_test 2024-06-26T07:12:57.1261928Z inflating: build/bin/c10_tempfile_test 2024-06-26T07:12:57.1308931Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-06-26T07:12:57.1363166Z inflating: build/bin/c10_typeid_test 2024-06-26T07:12:57.1795601Z inflating: build/bin/protoc-3.13.0.0 2024-06-26T07:12:57.2227243Z inflating: build/bin/protoc 2024-06-26T07:12:57.2278596Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-06-26T07:12:57.2329282Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-06-26T07:12:57.2380296Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-06-26T07:12:57.2430478Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-06-26T07:12:57.2481865Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-06-26T07:12:57.2534036Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-06-26T07:12:57.2580858Z inflating: build/bin/c10_cuda_CUDATest 2024-06-26T07:12:57.2631819Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-06-26T07:12:57.2948912Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-06-26T07:12:57.3280349Z inflating: build/bin/vec_test_all_types_AVX512 2024-06-26T07:12:57.3625470Z inflating: build/bin/vec_test_all_types_AVX2 2024-06-26T07:12:57.3676738Z inflating: build/bin/FileStoreTest 2024-06-26T07:12:57.3730371Z inflating: build/bin/TCPStoreTest 2024-06-26T07:12:57.3782188Z inflating: build/bin/HashStoreTest 2024-06-26T07:12:57.3795111Z inflating: build/bin/ProcessGroupMPITest 2024-06-26T07:12:57.3847243Z inflating: build/bin/test_edge_op_registration 2024-06-26T07:12:57.3851114Z inflating: build/bin/torch_shm_manager 2024-06-26T07:12:57.3854467Z inflating: build/bin/example_allreduce 2024-06-26T07:12:57.3907732Z inflating: build/bin/test_dist_autograd 2024-06-26T07:12:57.3975933Z inflating: build/bin/test_cpp_rpc 2024-06-26T07:12:57.3978160Z inflating: build/bin/parallel_benchmark 2024-06-26T07:12:57.4043146Z inflating: build/bin/test_mobile_nnc 2024-06-26T07:12:57.4051741Z inflating: build/bin/aot_model_compiler_test 2024-06-26T07:12:57.4383926Z inflating: build/bin/test_lazy 2024-06-26T07:12:57.5525288Z inflating: build/bin/test_api 2024-06-26T07:12:57.5597091Z inflating: build/bin/Dict_test 2024-06-26T07:12:57.5647128Z inflating: build/bin/Dimname_test 2024-06-26T07:12:57.5709840Z inflating: build/bin/MaybeOwned_test 2024-06-26T07:12:57.5766076Z inflating: build/bin/NamedTensor_test 2024-06-26T07:12:57.5823038Z inflating: build/bin/apply_utils_test 2024-06-26T07:12:57.5880617Z inflating: build/bin/atest 2024-06-26T07:12:57.5941280Z inflating: build/bin/basic 2024-06-26T07:12:57.5994885Z inflating: build/bin/broadcast_test 2024-06-26T07:12:57.6043802Z inflating: build/bin/cpu_allocator_test 2024-06-26T07:12:57.6099905Z inflating: build/bin/cpu_generator_test 2024-06-26T07:12:57.6151312Z inflating: build/bin/cpu_profiling_allocator_test 2024-06-26T07:12:57.6241731Z inflating: build/bin/cpu_rng_test 2024-06-26T07:12:57.6290456Z inflating: build/bin/dispatch_key_set_test 2024-06-26T07:12:57.6338589Z inflating: build/bin/dlconvertor_test 2024-06-26T07:12:57.6394891Z inflating: build/bin/extension_backend_test 2024-06-26T07:12:57.6448268Z inflating: build/bin/half_test 2024-06-26T07:12:57.6541615Z inflating: build/bin/ivalue_test 2024-06-26T07:12:57.6589666Z inflating: build/bin/lazy_tensor_test 2024-06-26T07:12:57.6642430Z inflating: build/bin/math_kernel_test 2024-06-26T07:12:57.6694928Z inflating: build/bin/memory_format_test 2024-06-26T07:12:57.6746140Z inflating: build/bin/memory_overlapping_test 2024-06-26T07:12:57.6798225Z inflating: build/bin/mobile_memory_cleanup 2024-06-26T07:12:57.6851560Z inflating: build/bin/native_test 2024-06-26T07:12:57.6900744Z inflating: build/bin/operator_name_test 2024-06-26T07:12:57.6950605Z inflating: build/bin/operators_test 2024-06-26T07:12:57.7000651Z inflating: build/bin/packedtensoraccessor_test 2024-06-26T07:12:57.7065789Z inflating: build/bin/pow_test 2024-06-26T07:12:57.7121600Z inflating: build/bin/quantized_test 2024-06-26T07:12:57.7170180Z inflating: build/bin/reduce_ops_test 2024-06-26T07:12:57.7219187Z inflating: build/bin/reportMemoryUsage_test 2024-06-26T07:12:57.7274516Z inflating: build/bin/scalar_tensor_test 2024-06-26T07:12:57.7330340Z inflating: build/bin/scalar_test 2024-06-26T07:12:57.7380597Z inflating: build/bin/StorageUtils_test 2024-06-26T07:12:57.7431312Z inflating: build/bin/stride_properties_test 2024-06-26T07:12:57.7507415Z inflating: build/bin/tensor_iterator_test 2024-06-26T07:12:57.7560278Z inflating: build/bin/test_parallel 2024-06-26T07:12:57.7562658Z inflating: build/bin/thread_init_test 2024-06-26T07:12:57.7616673Z inflating: build/bin/type_ptr_test 2024-06-26T07:12:57.7675349Z inflating: build/bin/type_test 2024-06-26T07:12:57.7725135Z inflating: build/bin/undefined_tensor_test 2024-06-26T07:12:57.7726500Z inflating: build/bin/verify_api_visibility 2024-06-26T07:12:57.7793690Z inflating: build/bin/legacy_vmap_test 2024-06-26T07:12:57.7843572Z inflating: build/bin/weakref_test 2024-06-26T07:12:57.7893552Z inflating: build/bin/wrapdim_test 2024-06-26T07:12:57.7943622Z inflating: build/bin/xla_tensor_test 2024-06-26T07:12:57.8001507Z inflating: build/bin/IListRef_test 2024-06-26T07:12:57.8104037Z inflating: build/bin/List_test 2024-06-26T07:12:57.8168288Z inflating: build/bin/KernelFunction_test 2024-06-26T07:12:57.8286070Z inflating: build/bin/kernel_function_legacy_test 2024-06-26T07:12:57.8379613Z inflating: build/bin/kernel_function_test 2024-06-26T07:12:57.8503503Z inflating: build/bin/kernel_lambda_legacy_test 2024-06-26T07:12:57.8603345Z inflating: build/bin/kernel_lambda_test 2024-06-26T07:12:57.8662400Z inflating: build/bin/kernel_stackbased_test 2024-06-26T07:12:57.8755451Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-06-26T07:12:57.8805596Z inflating: build/bin/CppSignature_test 2024-06-26T07:12:57.8859826Z inflating: build/bin/backend_fallback_test 2024-06-26T07:12:57.8906358Z inflating: build/bin/op_allowlist_test 2024-06-26T07:12:57.9197348Z inflating: build/bin/op_registration_test 2024-06-26T07:12:57.9259630Z inflating: build/bin/inline_container_test 2024-06-26T07:12:57.9310746Z inflating: build/bin/cuda_apply_test 2024-06-26T07:12:57.9359997Z inflating: build/bin/cuda_allocator_test 2024-06-26T07:12:57.9417965Z inflating: build/bin/cuda_atomic_ops_test 2024-06-26T07:12:57.9470142Z inflating: build/bin/cuda_caching_host_allocator_test 2024-06-26T07:12:57.9537834Z inflating: build/bin/cuda_complex_math_test 2024-06-26T07:12:57.9595142Z inflating: build/bin/cuda_complex_test 2024-06-26T07:12:57.9643134Z inflating: build/bin/cuda_device_test 2024-06-26T07:12:57.9699062Z inflating: build/bin/cuda_cub_test 2024-06-26T07:12:57.9748299Z inflating: build/bin/cuda_dlconvertor_test 2024-06-26T07:12:57.9811599Z inflating: build/bin/cuda_distributions_test 2024-06-26T07:12:57.9865809Z inflating: build/bin/cuda_generator_test 2024-06-26T07:12:57.9913794Z inflating: build/bin/cuda_half_test 2024-06-26T07:12:57.9963400Z inflating: build/bin/cuda_integer_divider_test 2024-06-26T07:12:58.0011156Z inflating: build/bin/cuda_optional_test 2024-06-26T07:12:58.0061209Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-06-26T07:12:58.0112128Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-06-26T07:12:58.0160293Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-06-26T07:12:58.0219197Z inflating: build/bin/cuda_stream_test 2024-06-26T07:12:58.0267016Z inflating: build/bin/cuda_cudnn_test 2024-06-26T07:12:58.0317285Z inflating: build/bin/cuda_vectorized_test 2024-06-26T07:12:58.0331769Z inflating: build/bin/tutorial_tensorexpr 2024-06-26T07:12:58.0395228Z inflating: build/bin/ProcessGroupGlooTest 2024-06-26T07:12:58.0451588Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-06-26T07:12:58.0512049Z inflating: build/bin/ProcessGroupNCCLTest 2024-06-26T07:12:58.0572466Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-06-26T07:12:58.1375792Z inflating: build/bin/test_tensorexpr 2024-06-26T07:12:58.1934405Z inflating: build/bin/test_jit 2024-06-26T07:12:58.1934925Z creating: .additional_ci_files/ 2024-06-26T07:12:58.1988491Z inflating: .additional_ci_files/test-times.json 2024-06-26T07:12:58.2199754Z inflating: .additional_ci_files/test-class-times.json 2024-06-26T07:12:58.2227345Z ##[group]Run rm artifacts.zip 2024-06-26T07:12:58.2227744Z rm artifacts.zip 2024-06-26T07:12:58.2236816Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:58.2237312Z env: 2024-06-26T07:12:58.2237593Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:58.2238043Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:58.2238524Z ##[endgroup] 2024-06-26T07:12:58.2945379Z ##[group]Run df -H 2024-06-26T07:12:58.2945825Z df -H 2024-06-26T07:12:58.2953569Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:58.2954066Z env: 2024-06-26T07:12:58.2954343Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:58.2954792Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:58.2955277Z ##[endgroup] 2024-06-26T07:12:58.2981192Z Filesystem Size Used Avail Use% Mounted on 2024-06-26T07:12:58.2981700Z devtmpfs 34G 0 34G 0% /dev 2024-06-26T07:12:58.2982451Z tmpfs 34G 164k 34G 1% /dev/shm 2024-06-26T07:12:58.2982974Z tmpfs 34G 455k 34G 1% /run 2024-06-26T07:12:58.2983459Z tmpfs 34G 0 34G 0% /sys/fs/cgroup 2024-06-26T07:12:58.2983925Z /dev/nvme0n1p1 162G 68G 94G 42% / 2024-06-26T07:12:58.3021388Z Prepare all required actions 2024-06-26T07:12:58.3021821Z Getting action download info 2024-06-26T07:12:58.4143302Z ##[group]Run ./.github/actions/download-td-artifacts 2024-06-26T07:12:58.4143774Z with: 2024-06-26T07:12:58.4144032Z env: 2024-06-26T07:12:58.4144308Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:58.4144757Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:58.4145234Z ##[endgroup] 2024-06-26T07:12:58.4175770Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:12:58.4176240Z with: 2024-06-26T07:12:58.4176507Z name: td_results 2024-06-26T07:12:58.4176832Z s3-bucket: gha-artifacts 2024-06-26T07:12:58.4177194Z region: us-east-1 2024-06-26T07:12:58.4177498Z env: 2024-06-26T07:12:58.4177773Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:58.4178245Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:58.4178760Z ##[endgroup] 2024-06-26T07:12:58.8258063Z (node:12486) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:12:58.8258978Z 2024-06-26T07:12:58.8259252Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:12:58.8259977Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:12:58.8260848Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:12:58.8870325Z Found 0 objects with prefix pytorch/pytorch/9673646806/td_results/ 2024-06-26T07:12:58.8877108Z Artifact download has finished successfully 2024-06-26T07:12:58.9006302Z ##[group]Run mkdir -p .additional_ci_files 2024-06-26T07:12:58.9006790Z mkdir -p .additional_ci_files 2024-06-26T07:12:58.9007347Z mv td_results.json .additional_ci_files/td_results.json 2024-06-26T07:12:58.9015895Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:58.9016392Z env: 2024-06-26T07:12:58.9016664Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:58.9017125Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:58.9017609Z ##[endgroup] 2024-06-26T07:12:58.9067465Z mv: cannot stat 'td_results.json': No such file or directory 2024-06-26T07:12:58.9079453Z ##[error]Process completed with exit code 1. 2024-06-26T07:12:58.9110848Z ##[group]Run .github/scripts/parse_ref.py 2024-06-26T07:12:58.9111331Z .github/scripts/parse_ref.py 2024-06-26T07:12:58.9118326Z shell: /usr/bin/bash -e {0} 2024-06-26T07:12:58.9118667Z env: 2024-06-26T07:12:58.9118944Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:58.9119395Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:58.9119876Z ##[endgroup] 2024-06-26T07:12:58.9350888Z Prepare all required actions 2024-06-26T07:12:58.9387924Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-06-26T07:12:58.9388380Z with: 2024-06-26T07:12:58.9388939Z github-token: *** 2024-06-26T07:12:58.9389248Z env: 2024-06-26T07:12:58.9389520Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:58.9389971Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:58.9390450Z ##[endgroup] 2024-06-26T07:12:58.9405276Z ##[group]Run set -eux 2024-06-26T07:12:58.9405616Z set -eux 2024-06-26T07:12:58.9406200Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-06-26T07:12:58.9414371Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:58.9414868Z env: 2024-06-26T07:12:58.9415143Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:58.9415593Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:58.9416234Z GITHUB_TOKEN: *** 2024-06-26T07:12:58.9416540Z ##[endgroup] 2024-06-26T07:12:58.9435547Z + python3 .github/scripts/get_workflow_job_id.py 9673646806 i-08ac67d1e0a1089db 2024-06-26T07:12:59.4812332Z setting job-id=26688916077 2024-06-26T07:12:59.4814296Z setting job-name=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:12:59.5034834Z Prepare all required actions 2024-06-26T07:12:59.5035279Z Getting action download info 2024-06-26T07:12:59.6102445Z ##[group]Run ./.github/actions/filter-test-configs 2024-06-26T07:12:59.6102904Z with: 2024-06-26T07:12:59.6103373Z github-token: *** 2024-06-26T07:12:59.6109961Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-06-26T07:12:59.6117417Z job-name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:12:59.6118294Z env: 2024-06-26T07:12:59.6118568Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:59.6119027Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:59.6119512Z ##[endgroup] 2024-06-26T07:12:59.6159572Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:12:59.6160131Z with: 2024-06-26T07:12:59.6160403Z shell: bash 2024-06-26T07:12:59.6160706Z timeout_minutes: 10 2024-06-26T07:12:59.6161040Z max_attempts: 5 2024-06-26T07:12:59.6161354Z retry_wait_seconds: 30 2024-06-26T07:12:59.6162484Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:12:59.6163655Z polling_interval_seconds: 1 2024-06-26T07:12:59.6164030Z warning_on_retry: true 2024-06-26T07:12:59.6164378Z continue_on_error: false 2024-06-26T07:12:59.6164724Z env: 2024-06-26T07:12:59.6165004Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:59.6165458Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:59.6166096Z GITHUB_TOKEN: *** 2024-06-26T07:12:59.6166412Z ##[endgroup] 2024-06-26T07:12:59.6660397Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:12:59.8873285Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T07:12:59.9024971Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.7/site-packages (2.27.1) 2024-06-26T07:12:59.9169597Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.7/site-packages (6.0.1) 2024-06-26T07:12:59.9178278Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (1.26.19) 2024-06-26T07:12:59.9375534Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2024.6.2) 2024-06-26T07:12:59.9383641Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2.0.12) 2024-06-26T07:12:59.9404925Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (3.7) 2024-06-26T07:13:00.6658670Z Command completed after 1 attempt(s). 2024-06-26T07:13:00.6698640Z ##[group]Run set -x 2024-06-26T07:13:00.6698977Z set -x 2024-06-26T07:13:00.6699277Z  2024-06-26T07:13:00.6699825Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:13:00.6700504Z # in runner workspace 2024-06-26T07:13:00.6701035Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-06-26T07:13:00.6708759Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:00.6709260Z env: 2024-06-26T07:13:00.6709548Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:00.6709999Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:00.6710482Z ##[endgroup] 2024-06-26T07:13:00.6731196Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-06-26T07:13:00.6925026Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:13:00.6925581Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:13:00.6926047Z echo "Job name: ${JOB_NAME}" 2024-06-26T07:13:00.6926449Z  2024-06-26T07:13:00.6926997Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:13:00.6927686Z # in runner workspace 2024-06-26T07:13:00.6928268Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-06-26T07:13:00.6928916Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-06-26T07:13:00.6929370Z  --job-name "${JOB_NAME}" \ 2024-06-26T07:13:00.6936659Z  --test-matrix "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-06-26T07:13:00.6943868Z  --selected-test-configs "" \ 2024-06-26T07:13:00.6944326Z  --pr-number "${PR_NUMBER}" \ 2024-06-26T07:13:00.6944749Z  --tag "${TAG}" \ 2024-06-26T07:13:00.6945140Z  --event-name "${EVENT_NAME}" \ 2024-06-26T07:13:00.6945576Z  --schedule "${SCHEDULE}" \ 2024-06-26T07:13:00.6946096Z  --branch "${HEAD_BRANCH}" 2024-06-26T07:13:00.6953328Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:00.6953822Z env: 2024-06-26T07:13:00.6954094Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:00.6954549Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:00.6955234Z GITHUB_TOKEN: *** 2024-06-26T07:13:00.6956043Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:13:00.6956909Z PR_NUMBER: 2024-06-26T07:13:00.6957221Z TAG: ciflow/inductor/129470 2024-06-26T07:13:00.6957583Z EVENT_NAME: push 2024-06-26T07:13:00.6974661Z SCHEDULE: 2024-06-26T07:13:00.6974976Z HEAD_BRANCH: 2024-06-26T07:13:00.6975271Z ##[endgroup] 2024-06-26T07:13:00.6994561Z Workflow: inductor-periodic 2024-06-26T07:13:00.6995620Z Job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:13:00.9237329Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-06-26T07:13:01.0689549Z ##[group]Run echo "Filtered matrix:" 2024-06-26T07:13:01.0690001Z echo "Filtered matrix:" 2024-06-26T07:13:01.0697110Z echo "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" 2024-06-26T07:13:01.0704047Z  2024-06-26T07:13:01.0704327Z echo 2024-06-26T07:13:01.0704713Z echo "Is the current job unstable? False" 2024-06-26T07:13:01.0705171Z  2024-06-26T07:13:01.0705436Z echo 2024-06-26T07:13:01.0705790Z echo "Is keep-going label set? False" 2024-06-26T07:13:01.0706226Z  2024-06-26T07:13:01.0706486Z echo 2024-06-26T07:13:01.0706925Z echo "Renabled issues? " 2024-06-26T07:13:01.0714568Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:01.0715059Z env: 2024-06-26T07:13:01.0715328Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:01.0715777Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:01.0716252Z ##[endgroup] 2024-06-26T07:13:01.0736155Z Filtered matrix: 2024-06-26T07:13:01.0744279Z {include: [{config: dynamo_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}]} 2024-06-26T07:13:01.0751026Z 2024-06-26T07:13:01.0751182Z Is the current job unstable? False 2024-06-26T07:13:01.0751475Z 2024-06-26T07:13:01.0751692Z Is keep-going label set? False 2024-06-26T07:13:01.0751955Z 2024-06-26T07:13:01.0752085Z Renabled issues? 2024-06-26T07:13:01.0791745Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:13:01.0792456Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:13:01.0800161Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:01.0800652Z env: 2024-06-26T07:13:01.0800926Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:01.0801376Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:01.0801859Z JOB_TIMEOUT: 240 2024-06-26T07:13:01.0802158Z ##[endgroup] 2024-06-26T07:13:01.0872870Z ##[group]Run set -x 2024-06-26T07:13:01.0873250Z set -x 2024-06-26T07:13:01.0873540Z  2024-06-26T07:13:01.0874054Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-06-26T07:13:01.0874633Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-06-26T07:13:01.0875187Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-06-26T07:13:01.0875705Z  TEST_COMMAND=.ci/onnx/test.sh 2024-06-26T07:13:01.0876115Z else 2024-06-26T07:13:01.0876443Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:13:01.0876858Z fi 2024-06-26T07:13:01.0877126Z  2024-06-26T07:13:01.0877581Z # detached container should get cleaned up by teardown_ec2_linux 2024-06-26T07:13:01.0878320Z # TODO: Stop building test binaries as part of the build phase 2024-06-26T07:13:01.0878988Z # Used for GPU_FLAG since that doesn't play nice 2024-06-26T07:13:01.0879568Z # shellcheck disable=SC2086,SC2090 2024-06-26T07:13:01.0880025Z container_name=$(docker run \ 2024-06-26T07:13:01.0880437Z  ${GPU_FLAG:-} \ 2024-06-26T07:13:01.0880801Z  -e BUILD_ENVIRONMENT \ 2024-06-26T07:13:01.0881193Z  -e PR_NUMBER \ 2024-06-26T07:13:01.0881539Z  -e GITHUB_ACTIONS \ 2024-06-26T07:13:01.0881922Z  -e GITHUB_REPOSITORY \ 2024-06-26T07:13:01.0882446Z  -e GITHUB_WORKFLOW \ 2024-06-26T07:13:01.0882823Z  -e GITHUB_JOB \ 2024-06-26T07:13:01.0883172Z  -e GITHUB_RUN_ID \ 2024-06-26T07:13:01.0883551Z  -e GITHUB_RUN_NUMBER \ 2024-06-26T07:13:01.0883951Z  -e GITHUB_RUN_ATTEMPT \ 2024-06-26T07:13:01.0884332Z  -e JOB_ID \ 2024-06-26T07:13:01.0884667Z  -e JOB_NAME \ 2024-06-26T07:13:01.0885013Z  -e BASE_SHA \ 2024-06-26T07:13:01.0885350Z  -e BRANCH \ 2024-06-26T07:13:01.0885673Z  -e SHA1 \ 2024-06-26T07:13:01.0886017Z  -e AWS_DEFAULT_REGION \ 2024-06-26T07:13:01.0886413Z  -e IN_WHEEL_TEST \ 2024-06-26T07:13:01.0886783Z  -e SHARD_NUMBER \ 2024-06-26T07:13:01.0887138Z  -e TEST_CONFIG \ 2024-06-26T07:13:01.0887500Z  -e NUM_TEST_SHARDS \ 2024-06-26T07:13:01.0887883Z  -e REENABLED_ISSUES \ 2024-06-26T07:13:01.0888289Z  -e CONTINUE_THROUGH_ERROR \ 2024-06-26T07:13:01.0888710Z  -e VERBOSE_TEST_LOGS \ 2024-06-26T07:13:01.0889101Z  -e NO_TEST_TIMEOUT \ 2024-06-26T07:13:01.0889471Z  -e NO_TD \ 2024-06-26T07:13:01.0889805Z  -e TD_DISTRIBUTED \ 2024-06-26T07:13:01.0890173Z  -e PR_LABELS \ 2024-06-26T07:13:01.0890571Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-06-26T07:13:01.0891084Z  -e SCCACHE_BUCKET \ 2024-06-26T07:13:01.0891467Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-06-26T07:13:01.0891870Z  -e XLA_CUDA \ 2024-06-26T07:13:01.0892449Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-06-26T07:13:01.0892955Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-06-26T07:13:01.0893465Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-06-26T07:13:01.0893975Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-06-26T07:13:01.0894434Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-06-26T07:13:01.0894849Z  -e DASHBOARD_TAG \ 2024-06-26T07:13:01.0895321Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-06-26T07:13:01.0895871Z  --security-opt seccomp=unconfined \ 2024-06-26T07:13:01.0896332Z  --cap-add=SYS_PTRACE \ 2024-06-26T07:13:01.0896716Z  --ipc=host \ 2024-06-26T07:13:01.0897064Z  --shm-size="${SHM_SIZE}" \ 2024-06-26T07:13:01.0897454Z  --tty \ 2024-06-26T07:13:01.0897762Z  --detach \ 2024-06-26T07:13:01.0898111Z  --name="${container_name}" \ 2024-06-26T07:13:01.0898516Z  --user jenkins \ 2024-06-26T07:13:01.0899079Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-06-26T07:13:01.0899643Z  -w /var/lib/jenkins/workspace \ 2024-06-26T07:13:01.0900078Z  "${DOCKER_IMAGE}" 2024-06-26T07:13:01.0900420Z ) 2024-06-26T07:13:01.0900818Z # Propagate download.pytorch.org IP to container 2024-06-26T07:13:01.0901740Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-06-26T07:13:01.0902710Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-06-26T07:13:01.0903618Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-06-26T07:13:01.0910901Z shell: /usr/bin/bash -e {0} 2024-06-26T07:13:01.0911249Z env: 2024-06-26T07:13:01.0911524Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:01.0911964Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:01.0912572Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:13:01.0913069Z PR_NUMBER: 2024-06-26T07:13:01.0913388Z GITHUB_REPOSITORY: pytorch/pytorch 2024-06-26T07:13:01.0913809Z GITHUB_WORKFLOW: inductor-periodic 2024-06-26T07:13:01.0914207Z GITHUB_JOB: test 2024-06-26T07:13:01.0914521Z GITHUB_RUN_ID: 9673646806 2024-06-26T07:13:01.0914876Z GITHUB_RUN_NUMBER: 45567 2024-06-26T07:13:01.0915216Z GITHUB_RUN_ATTEMPT: 1 2024-06-26T07:13:01.0915630Z JOB_ID: 26688916077 2024-06-26T07:13:01.0916437Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:13:01.0917303Z BRANCH: 2024-06-26T07:13:01.0917628Z SHA1: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:01.0918134Z BASE_SHA: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:01.0918602Z TEST_CONFIG: dynamo_eager_timm 2024-06-26T07:13:01.0918975Z SHARD_NUMBER: 2 2024-06-26T07:13:01.0919274Z NUM_TEST_SHARDS: 2 2024-06-26T07:13:01.0919594Z REENABLED_ISSUES: 2024-06-26T07:13:01.0919934Z CONTINUE_THROUGH_ERROR: False 2024-06-26T07:13:01.0920313Z VERBOSE_TEST_LOGS: False 2024-06-26T07:13:01.0920665Z NO_TEST_TIMEOUT: False 2024-06-26T07:13:01.0920996Z NO_TD: False 2024-06-26T07:13:01.0921293Z TD_DISTRIBUTED: False 2024-06-26T07:13:01.0921699Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-06-26T07:13:01.0922210Z SCCACHE_S3_KEY_PREFIX: inductor-periodic 2024-06-26T07:13:01.0922640Z SHM_SIZE: 2g 2024-06-26T07:13:01.0923716Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:13:01.0924868Z XLA_CUDA: 2024-06-26T07:13:01.0925347Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:13:01.0925953Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-06-26T07:13:01.0926386Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-06-26T07:13:01.0926785Z DASHBOARD_TAG: 2024-06-26T07:13:01.0927303Z HUGGING_FACE_HUB_TOKEN: *** 2024-06-26T07:13:01.0927659Z ##[endgroup] 2024-06-26T07:13:01.0949494Z + [[ dynamo_eager_timm == \m\u\l\t\i\g\p\u ]] 2024-06-26T07:13:01.0950221Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]] 2024-06-26T07:13:01.0950741Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:13:01.0956421Z +++ nproc --ignore=2 2024-06-26T07:13:01.0969822Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_9673646806 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:13:12.3026229Z + container_name=fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:13:12.3027417Z + grep download.pytorch.org /etc/hosts 2024-06-26T07:13:12.3029395Z + docker exec -i fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 sudo bash -c '/bin/cat >> /etc/hosts' 2024-06-26T07:13:12.3729216Z + echo DOCKER_CONTAINER_ID=fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:13:12.3732499Z ++ echo dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:13:12.3734224Z + docker exec -t fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 sh -c 'pip install dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-06-26T07:13:12.7634317Z Processing ./dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+gitb8c4c54) 2024-06-26T07:13:13.0758147Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.13.1) 2024-06-26T07:13:13.0762137Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (4.12.2) 2024-06-26T07:13:13.0764346Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.12.1) 2024-06-26T07:13:13.0766836Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.8.8) 2024-06-26T07:13:13.0769140Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.1.4) 2024-06-26T07:13:13.0772442Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2024.2.0) 2024-06-26T07:13:13.0797450Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.3.0) 2024-06-26T07:13:13.0854316Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.21.2) 2024-06-26T07:13:13.1283471Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.1.5) 2024-06-26T07:13:13.1440411Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.3.0) 2024-06-26T07:13:14.4594778Z Installing collected packages: torch 2024-06-26T07:13:23.1058178Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-06-26T07:13:23.1059545Z timm 0.9.7 requires torchvision, which is not installed. 2024-06-26T07:13:23.1060214Z Successfully installed torch-2.5.0a0+gitb8c4c54 2024-06-26T07:13:23.1858966Z ++ dirname .ci/pytorch/test.sh 2024-06-26T07:13:23.1864414Z + source .ci/pytorch/common.sh 2024-06-26T07:13:23.1867102Z +++ dirname .ci/pytorch/common.sh 2024-06-26T07:13:23.1872964Z ++ source .ci/pytorch/common_utils.sh 2024-06-26T07:13:23.1874160Z +++ declare -f -t trap_add 2024-06-26T07:13:23.1880188Z ++ set -ex 2024-06-26T07:13:23.1880870Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-06-26T07:13:23.1881419Z ++ BUILD_TEST_LIBTORCH=0 2024-06-26T07:13:23.1881920Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]] 2024-06-26T07:13:23.1884661Z ++ stat -c %u /var/lib/jenkins/workspace 2024-06-26T07:13:23.1893065Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-06-26T07:13:23.1893516Z + trap_add cleanup_workspace EXIT 2024-06-26T07:13:23.1893998Z + trap_add_cmd=cleanup_workspace 2024-06-26T07:13:23.1894364Z + shift 2024-06-26T07:13:23.1894704Z + for trap_add_name in "$@" 2024-06-26T07:13:23.1899554Z +++ trap -p EXIT 2024-06-26T07:13:23.1900155Z ++ eval 'extract_trap_cmd ' 2024-06-26T07:13:23.1900688Z +++ extract_trap_cmd 2024-06-26T07:13:23.1901234Z +++ printf '%s\n' '' 2024-06-26T07:13:23.1901809Z ++ printf '%s\n' cleanup_workspace 2024-06-26T07:13:23.1903505Z + trap -- ' 2024-06-26T07:13:23.1903942Z cleanup_workspace' EXIT 2024-06-26T07:13:23.1904402Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-06-26T07:13:23.7171690Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-06-26T07:13:23.7182586Z + echo 'Environment variables:' 2024-06-26T07:13:23.7183019Z Environment variables: 2024-06-26T07:13:23.7183354Z + env 2024-06-26T07:13:23.7188031Z INSTALLED_DB=yes 2024-06-26T07:13:23.7188517Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-06-26T07:13:23.7189580Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:13:23.7190266Z NV_NVML_DEV_VERSION=12.1.105-1 2024-06-26T07:13:23.7191110Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:13:23.7191827Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:13:23.7192419Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-06-26T07:13:23.7192976Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:13:23.7193653Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:13:23.7194380Z HOSTNAME=fba69756b470 2024-06-26T07:13:23.7195462Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_1548d49f-0fd7-4416-b06b-38128794c60e 2024-06-26T07:13:23.7196425Z GITHUB_ACTION=__self 2024-06-26T07:13:23.7196770Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:13:23.7201158Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-06-26T07:13:23.7205136Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-06-26T07:13:23.7205644Z NV_NVTX_VERSION=12.1.105-1 2024-06-26T07:13:23.7205986Z GITHUB_RUN_NUMBER=45567 2024-06-26T07:13:23.7206331Z TEST_CONFIG=dynamo_eager_timm 2024-06-26T07:13:23.7206716Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:13:23.7207184Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:13:23.7207636Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-06-26T07:13:23.7208069Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-06-26T07:13:23.7208528Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-06-26T07:13:23.7208962Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:13:23.7209445Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:13:23.7209904Z GITHUB_REF_TYPE=tag 2024-06-26T07:13:23.7210227Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:13:23.7210706Z NCCL_VERSION=2.17.1-1 2024-06-26T07:13:23.7211083Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:23.7211690Z XLA_CUDA= 2024-06-26T07:13:23.7212521Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:13:23.7214158Z *** 2024-06-26T07:13:23.7214442Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:13:23.7214844Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:13:23.7215210Z GITHUB_ACTIONS=true 2024-06-26T07:13:23.7215544Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:23.7216037Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-06-26T07:13:23.7216577Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-06-26T07:13:23.7217041Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:23.7217537Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:13:23.7218005Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:23.7218918Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:13:23.7219678Z UCC_HOME=/usr 2024-06-26T07:13:23.7220015Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-06-26T07:13:23.7220414Z VERBOSE_TEST_LOGS=False 2024-06-26T07:13:23.7220754Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:13:23.7221201Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-06-26T07:13:23.7221683Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:13:23.7222143Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-06-26T07:13:23.7222518Z SHARD_NUMBER=2 2024-06-26T07:13:23.7222825Z GITHUB_REF_PROTECTED=false 2024-06-26T07:13:23.7223325Z HOME=/var/lib/jenkins 2024-06-26T07:13:23.7223693Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:13:23.7224132Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:13:23.7224612Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:13:23.7225145Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-06-26T07:13:23.7225554Z CUDA_VERSION=12.1.1 2024-06-26T07:13:23.7225959Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-06-26T07:13:23.7226383Z NUM_TEST_SHARDS=2 2024-06-26T07:13:23.7226682Z UCX_HOME=/usr 2024-06-26T07:13:23.7227179Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-06-26T07:13:23.7228291Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_1548d49f-0fd7-4416-b06b-38128794c60e 2024-06-26T07:13:23.7229678Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:13:23.7231025Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_1548d49f-0fd7-4416-b06b-38128794c60e 2024-06-26T07:13:23.7232128Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:13:23.7232778Z GITHUB_EVENT_NAME=push 2024-06-26T07:13:23.7233101Z DASHBOARD_TAG= 2024-06-26T07:13:23.7233391Z GITHUB_RUN_ID=9673646806 2024-06-26T07:13:23.7233836Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-06-26T07:13:23.7234362Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-06-26T07:13:23.7235334Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_1548d49f-0fd7-4416-b06b-38128794c60e 2024-06-26T07:13:23.7236230Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:13:23.7236631Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-06-26T07:13:23.7236999Z PR_NUMBER= 2024-06-26T07:13:23.7237274Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:13:23.7237604Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:13:23.7238036Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:13:23.7238487Z TERM=xterm 2024-06-26T07:13:23.7238816Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-06-26T07:13:23.7239224Z INSTALLED_VISION=yes 2024-06-26T07:13:23.7239528Z BRANCH= 2024-06-26T07:13:23.7239808Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:13:23.7240199Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:13:23.7240618Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:13:23.7241358Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:13:23.7242097Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:13:23.7242581Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:13:23.7243126Z REENABLED_ISSUES= 2024-06-26T07:13:23.7243424Z SHLVL=1 2024-06-26T07:13:23.7243672Z MAX_JOBS=14 2024-06-26T07:13:23.7243986Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-06-26T07:13:23.7244337Z NVARCH=x86_64 2024-06-26T07:13:23.7244632Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:13:23.7245070Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:23.7245591Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:13:23.7246072Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-06-26T07:13:23.7246709Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:13:23.7247276Z GITHUB_JOB=test 2024-06-26T07:13:23.7247663Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-06-26T07:13:23.7248211Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:13:23.7248708Z NO_TEST_TIMEOUT=False 2024-06-26T07:13:23.7249029Z TD_DISTRIBUTED=False 2024-06-26T07:13:23.7249399Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-06-26T07:13:23.7249835Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:13:23.7250241Z NV_NVPROF_VERSION=12.1.105-1 2024-06-26T07:13:23.7250704Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:13:23.7251051Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:13:23.7251389Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:13:23.7252590Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:13:23.7253748Z GITHUB_BASE_REF= 2024-06-26T07:13:23.7254072Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:13:23.7254430Z CI=true 2024-06-26T07:13:23.7254740Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:13:23.7255147Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:13:23.7255513Z JOB_ID=26688916077 2024-06-26T07:13:23.7255816Z INSTALLED_PROTOBUF=yes 2024-06-26T07:13:23.7256138Z GITHUB_HEAD_REF= 2024-06-26T07:13:23.7256436Z GITHUB_ACTION_REF= 2024-06-26T07:13:23.7256857Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:13:23.7257367Z GITHUB_WORKFLOW=inductor-periodic 2024-06-26T07:13:23.7257768Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:13:23.7258706Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_1548d49f-0fd7-4416-b06b-38128794c60e 2024-06-26T07:13:23.7259508Z NO_TD=False 2024-06-26T07:13:23.7259800Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:13:23.7260160Z _=/usr/bin/env 2024-06-26T07:13:23.7260631Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-06-26T07:13:23.7338137Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-06-26T07:13:23.7339134Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-06-26T07:13:23.7340097Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-06-26T07:13:23.7341049Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-06-26T07:13:23.7341688Z + BUILD_DIR=build 2024-06-26T07:13:23.7342011Z + BUILD_RENAMED_DIR=build_renamed 2024-06-26T07:13:23.7342413Z + BUILD_BIN_DIR=build/bin 2024-06-26T07:13:23.7342755Z + SHARD_NUMBER=2 2024-06-26T07:13:23.7343061Z + NUM_TEST_SHARDS=2 2024-06-26T07:13:23.7343375Z + export VALGRIND=ON 2024-06-26T07:13:23.7343693Z + VALGRIND=ON 2024-06-26T07:13:23.7344154Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]] 2024-06-26T07:13:23.7344642Z + [[ 0 == \1 ]] 2024-06-26T07:13:23.7344939Z + [[ False == \1 ]] 2024-06-26T07:13:23.7345419Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]] 2024-06-26T07:13:23.7345946Z ++ realpath build/custom_test_artifacts 2024-06-26T07:13:23.7349333Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-06-26T07:13:23.7350032Z + [[ -n '' ]] 2024-06-26T07:13:23.7350378Z + echo 'Environment variables' 2024-06-26T07:13:23.7350754Z Environment variables 2024-06-26T07:13:23.7351065Z + env 2024-06-26T07:13:23.7354170Z INSTALLED_DB=yes 2024-06-26T07:13:23.7354542Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-06-26T07:13:23.7355170Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:13:23.7355607Z NV_NVML_DEV_VERSION=12.1.105-1 2024-06-26T07:13:23.7356290Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:13:23.7357071Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:13:23.7357784Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-06-26T07:13:23.7358443Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:13:23.7359090Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:13:23.7359582Z HOSTNAME=fba69756b470 2024-06-26T07:13:23.7360410Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_1548d49f-0fd7-4416-b06b-38128794c60e 2024-06-26T07:13:23.7361200Z GITHUB_ACTION=__self 2024-06-26T07:13:23.7361654Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:13:23.7365745Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-06-26T07:13:23.7369729Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-06-26T07:13:23.7370234Z NV_NVTX_VERSION=12.1.105-1 2024-06-26T07:13:23.7370709Z GITHUB_RUN_NUMBER=45567 2024-06-26T07:13:23.7371056Z TEST_CONFIG=dynamo_eager_timm 2024-06-26T07:13:23.7371440Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:13:23.7371902Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:13:23.7372653Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-06-26T07:13:23.7373098Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-06-26T07:13:23.7373519Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-06-26T07:13:23.7373954Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:13:23.7374436Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:13:23.7374894Z GITHUB_REF_TYPE=tag 2024-06-26T07:13:23.7375218Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:13:23.7375608Z NCCL_VERSION=2.17.1-1 2024-06-26T07:13:23.7375985Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:23.7376420Z XLA_CUDA= 2024-06-26T07:13:23.7376896Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:13:23.7377287Z *** 2024-06-26T07:13:23.7377573Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:13:23.7377962Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:13:23.7378320Z GITHUB_ACTIONS=true 2024-06-26T07:13:23.7378650Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:23.7379124Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-06-26T07:13:23.7379646Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-06-26T07:13:23.7380115Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:23.7380610Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:13:23.7381082Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:23.7381982Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:13:23.7382740Z UCC_HOME=/usr 2024-06-26T07:13:23.7383073Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-06-26T07:13:23.7383460Z VERBOSE_TEST_LOGS=False 2024-06-26T07:13:23.7383807Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:13:23.7384256Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-06-26T07:13:23.7384748Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:13:23.7385202Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-06-26T07:13:23.7385585Z SHARD_NUMBER=2 2024-06-26T07:13:23.7385891Z GITHUB_REF_PROTECTED=false 2024-06-26T07:13:23.7386228Z HOME=/var/lib/jenkins 2024-06-26T07:13:23.7386589Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:13:23.7387128Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:13:23.7387611Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:13:23.7388184Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-06-26T07:13:23.7388606Z CUDA_VERSION=12.1.1 2024-06-26T07:13:23.7389012Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-06-26T07:13:23.7389445Z NUM_TEST_SHARDS=2 2024-06-26T07:13:23.7389732Z UCX_HOME=/usr 2024-06-26T07:13:23.7390241Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-06-26T07:13:23.7391299Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_1548d49f-0fd7-4416-b06b-38128794c60e 2024-06-26T07:13:23.7392678Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:13:23.7394031Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_1548d49f-0fd7-4416-b06b-38128794c60e 2024-06-26T07:13:23.7395135Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:13:23.7395789Z GITHUB_EVENT_NAME=push 2024-06-26T07:13:23.7396113Z DASHBOARD_TAG= 2024-06-26T07:13:23.7396408Z GITHUB_RUN_ID=9673646806 2024-06-26T07:13:23.7396852Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-06-26T07:13:23.7397420Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-06-26T07:13:23.7398401Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_1548d49f-0fd7-4416-b06b-38128794c60e 2024-06-26T07:13:23.7399426Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:13:23.7399821Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-06-26T07:13:23.7400199Z PR_NUMBER= 2024-06-26T07:13:23.7400478Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:13:23.7400782Z VALGRIND=ON 2024-06-26T07:13:23.7401072Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:13:23.7401516Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:13:23.7401975Z TERM=xterm 2024-06-26T07:13:23.7402297Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-06-26T07:13:23.7402717Z INSTALLED_VISION=yes 2024-06-26T07:13:23.7403032Z BRANCH= 2024-06-26T07:13:23.7403318Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:13:23.7403722Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:13:23.7404144Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:13:23.7404880Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:13:23.7405633Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:13:23.7406111Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:13:23.7406573Z REENABLED_ISSUES= 2024-06-26T07:13:23.7406875Z SHLVL=1 2024-06-26T07:13:23.7407132Z MAX_JOBS=14 2024-06-26T07:13:23.7407446Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-06-26T07:13:23.7407796Z NVARCH=x86_64 2024-06-26T07:13:23.7408125Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:13:23.7408581Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:13:23.7409110Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:13:23.7409595Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-06-26T07:13:23.7410245Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:13:23.7427256Z GITHUB_JOB=test 2024-06-26T07:13:23.7427743Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-06-26T07:13:23.7428365Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:13:23.7428874Z NO_TEST_TIMEOUT=False 2024-06-26T07:13:23.7429203Z TD_DISTRIBUTED=False 2024-06-26T07:13:23.7429574Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-06-26T07:13:23.7430012Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:13:23.7430429Z NV_NVPROF_VERSION=12.1.105-1 2024-06-26T07:13:23.7430783Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:13:23.7431127Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:13:23.7431470Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:13:23.7432503Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:13:23.7433572Z GITHUB_BASE_REF= 2024-06-26T07:13:23.7434018Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:13:23.7434400Z CI=true 2024-06-26T07:13:23.7434707Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:13:23.7435116Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:13:23.7435483Z JOB_ID=26688916077 2024-06-26T07:13:23.7435793Z INSTALLED_PROTOBUF=yes 2024-06-26T07:13:23.7436112Z GITHUB_HEAD_REF= 2024-06-26T07:13:23.7436421Z GITHUB_ACTION_REF= 2024-06-26T07:13:23.7436847Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:13:23.7437365Z GITHUB_WORKFLOW=inductor-periodic 2024-06-26T07:13:23.7437769Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:13:23.7438656Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_1548d49f-0fd7-4416-b06b-38128794c60e 2024-06-26T07:13:23.7439458Z NO_TD=False 2024-06-26T07:13:23.7439750Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:13:23.7440109Z _=/usr/bin/env 2024-06-26T07:13:23.7440438Z + echo 'Testing pytorch' 2024-06-26T07:13:23.7440775Z Testing pytorch 2024-06-26T07:13:23.7441090Z + export LANG=C.UTF-8 2024-06-26T07:13:23.7441427Z + LANG=C.UTF-8 2024-06-26T07:13:23.7441718Z + PR_NUMBER= 2024-06-26T07:13:23.7442035Z + [[ dynamo_eager_timm == \d\e\f\a\u\l\t ]] 2024-06-26T07:13:23.7442521Z + [[ dynamo_eager_timm == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-06-26T07:13:23.7442991Z + [[ dynamo_eager_timm == \s\l\o\w ]] 2024-06-26T07:13:23.7443597Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-06-26T07:13:23.7444372Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]] 2024-06-26T07:13:23.7444909Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:13:23.7445377Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:13:23.7445819Z + [[ dynamo_eager_timm == *crossref* ]] 2024-06-26T07:13:23.7446365Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-06-26T07:13:23.7446997Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]] 2024-06-26T07:13:23.7447630Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:13:23.7448187Z + pip_install --user ninja==1.10.2 2024-06-26T07:13:23.7448702Z + pip install --progress-bar off --user ninja==1.10.2 2024-06-26T07:13:24.1023536Z Collecting ninja==1.10.2 2024-06-26T07:13:24.1187477Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-06-26T07:13:24.1315987Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-06-26T07:13:25.4317824Z Installing collected packages: ninja 2024-06-26T07:13:25.4381379Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-06-26T07:13:25.4382654Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-06-26T07:13:25.4423301Z Successfully installed ninja-1.10.2 2024-06-26T07:13:25.5267571Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:13:25.5269733Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:13:25.5271240Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-06-26T07:13:25.5271745Z + install_tlparse 2024-06-26T07:13:25.5272129Z + pip_install --user tlparse==0.3.7 2024-06-26T07:13:25.5272675Z + pip install --progress-bar off --user tlparse==0.3.7 2024-06-26T07:13:25.8859374Z Collecting tlparse==0.3.7 2024-06-26T07:13:25.9031749Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-06-26T07:13:25.9131468Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-06-26T07:13:27.2336777Z Installing collected packages: tlparse 2024-06-26T07:13:27.2692919Z Successfully installed tlparse-0.3.7 2024-06-26T07:13:27.3547421Z ++ python -m site --user-base 2024-06-26T07:13:27.3700881Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:13:27.3702484Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]] 2024-06-26T07:13:27.3703309Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]] 2024-06-26T07:13:27.3704031Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:13:27.3704966Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-06-26T07:13:27.3706097Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass 2024-06-26T07:13:27.3706809Z + cd test 2024-06-26T07:13:27.3707328Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-06-26T07:13:28.8576847Z + [[ dynamo_eager_timm == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-06-26T07:13:28.8577573Z + [[ dynamo_eager_timm == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-06-26T07:13:28.8578341Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:13:28.8578844Z + pushd test 2024-06-26T07:13:28.8579167Z ~/workspace/test ~/workspace 2024-06-26T07:13:28.8579682Z ++ python -c 'import torch; print(torch.version.cuda)' 2024-06-26T07:13:30.3146314Z + CUDA_VERSION=12.1 2024-06-26T07:13:30.3147201Z + '[' 12.1 == 12.4 ']' 2024-06-26T07:13:30.3147550Z + ISCUDA124= 2024-06-26T07:13:30.3147836Z + popd 2024-06-26T07:13:30.3148094Z ~/workspace 2024-06-26T07:13:30.3151537Z + DYNAMO_BENCHMARK_FLAGS=() 2024-06-26T07:13:30.3152151Z + [[ dynamo_eager_timm == *dynamo_eager* ]] 2024-06-26T07:13:30.3152938Z + DYNAMO_BENCHMARK_FLAGS+=(--backend eager) 2024-06-26T07:13:30.3153605Z + [[ dynamo_eager_timm == *dynamic* ]] 2024-06-26T07:13:30.3154251Z + [[ dynamo_eager_timm == *cpu_inductor* ]] 2024-06-26T07:13:30.3154986Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-06-26T07:13:30.3180937Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-06-26T07:13:30.3181774Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-06-26T07:13:30.3182816Z + cd test 2024-06-26T07:13:30.3183377Z + python -c 'import torch; print(torch.__config__.show())' 2024-06-26T07:13:31.6571498Z PyTorch built with: 2024-06-26T07:13:31.6572505Z - GCC 9.4 2024-06-26T07:13:31.6573034Z - C++ Version: 201703 2024-06-26T07:13:31.6573951Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:13:31.6575032Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:13:31.6575692Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:13:31.6576207Z - LAPACK is enabled (usually provided by MKL) 2024-06-26T07:13:31.6576687Z - NNPACK is enabled 2024-06-26T07:13:31.6577064Z - CPU capability usage: AVX2 2024-06-26T07:13:31.6577462Z - CUDA Runtime 12.1 2024-06-26T07:13:31.6577999Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-06-26T07:13:31.6578601Z - CuDNN 90.1 (built against CUDA 12.4) 2024-06-26T07:13:31.6579042Z - Magma 2.6.1 2024-06-26T07:13:31.6586305Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-06-26T07:13:31.6592945Z 2024-06-26T07:13:31.9344174Z + cd test 2024-06-26T07:13:31.9344849Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-06-26T07:13:33.1490343Z ATen/Parallel: 2024-06-26T07:13:33.1491030Z at::get_num_threads() : 8 2024-06-26T07:13:33.1491572Z at::get_num_interop_threads() : 16 2024-06-26T07:13:33.1492004Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:13:33.1492684Z omp_get_max_threads() : 8 2024-06-26T07:13:33.1493675Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:13:33.1494477Z mkl_get_max_threads() : 8 2024-06-26T07:13:33.1495097Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:13:33.1495730Z std::thread::hardware_concurrency() : 16 2024-06-26T07:13:33.1496158Z Environment variables: 2024-06-26T07:13:33.1496507Z OMP_NUM_THREADS : [not set] 2024-06-26T07:13:33.1497211Z MKL_NUM_THREADS : [not set] 2024-06-26T07:13:33.1497582Z ATen parallel backend: OpenMP 2024-06-26T07:13:33.1497841Z 2024-06-26T07:13:33.3918690Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-06-26T07:13:33.3919460Z + [[ dynamo_eager_timm == *backward* ]] 2024-06-26T07:13:33.3920007Z + [[ dynamo_eager_timm == *xla* ]] 2024-06-26T07:13:33.3920422Z + [[ dynamo_eager_timm == *executorch* ]] 2024-06-26T07:13:33.3921024Z + [[ dynamo_eager_timm == \j\i\t\_\l\e\g\a\c\y ]] 2024-06-26T07:13:33.3921696Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-06-26T07:13:33.3922373Z + [[ dynamo_eager_timm == distributed ]] 2024-06-26T07:13:33.3922972Z + [[ dynamo_eager_timm == *inductor_distributed* ]] 2024-06-26T07:13:33.3923530Z + [[ dynamo_eager_timm == *inductor-halide* ]] 2024-06-26T07:13:33.3924092Z + [[ dynamo_eager_timm == *inductor-micro-benchmark* ]] 2024-06-26T07:13:33.3924601Z + [[ dynamo_eager_timm == *huggingface* ]] 2024-06-26T07:13:33.3925039Z + [[ dynamo_eager_timm == *timm* ]] 2024-06-26T07:13:33.3925437Z + install_torchvision 2024-06-26T07:13:33.3925768Z + local orig_preload 2024-06-26T07:13:33.3926086Z + local commit 2024-06-26T07:13:33.3926393Z ++ get_pinned_commit vision 2024-06-26T07:13:33.3926764Z ++ cat .github/ci_commit_pins/vision.txt 2024-06-26T07:13:33.3932568Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:33.3933209Z + orig_preload= 2024-06-26T07:13:33.3933670Z + '[' -n '' ']' 2024-06-26T07:13:33.3934568Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:33.3936000Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:33.7028711Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:33.7034055Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-wpc5fgro 2024-06-26T07:13:33.7051002Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-wpc5fgro 2024-06-26T07:13:35.0320432Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-06-26T07:13:35.0339034Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:36.2310858Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:36.4206868Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:13:38.7174249Z Preparing metadata (setup.py) ... [?25l- \ done 2024-06-26T07:13:38.7225076Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2) 2024-06-26T07:13:38.7228299Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+gitb8c4c54) 2024-06-26T07:13:38.7233799Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-06-26T07:13:38.7438110Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-06-26T07:13:38.7446578Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-06-26T07:13:38.7448895Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.12.1) 2024-06-26T07:13:38.7451069Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-06-26T07:13:38.7454588Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-06-26T07:13:38.7456973Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.2.0) 2024-06-26T07:13:38.7885006Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-06-26T07:13:38.8041127Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-06-26T07:13:38.8117911Z Building wheels for collected packages: torchvision 2024-06-26T07:14:51.7205142Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done 2024-06-26T07:14:51.7232131Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2112797 sha256=182a229016f9046f177d1fb9762290c7c191950caf65d0d1737f6a6e3205643e 2024-06-26T07:14:51.7234476Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-06-26T07:14:51.7269596Z Successfully built torchvision 2024-06-26T07:14:52.9349061Z Installing collected packages: torchvision 2024-06-26T07:14:53.3120971Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-06-26T07:14:53.4243190Z + '[' -n '' ']' 2024-06-26T07:14:53.4243587Z + id=1 2024-06-26T07:14:53.4243938Z + test_dynamo_benchmark timm_models 1 2024-06-26T07:14:53.4244402Z ++ pwd 2024-06-26T07:14:53.4247784Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:14:53.4248500Z + local suite=timm_models 2024-06-26T07:14:53.4248844Z + shift 2024-06-26T07:14:53.4249115Z + local shard_id=1 2024-06-26T07:14:53.4249432Z + shift 2024-06-26T07:14:53.4252492Z + [[ dynamo_eager_timm == *perf_compare* ]] 2024-06-26T07:14:53.4253094Z + [[ dynamo_eager_timm == *perf* ]] 2024-06-26T07:14:53.4253643Z + [[ dynamo_eager_timm == *cpu_inductor* ]] 2024-06-26T07:14:53.4254192Z + [[ dynamo_eager_timm == *aot_inductor* ]] 2024-06-26T07:14:53.4255163Z + test_single_dynamo_benchmark inference timm_models 1 --inference --bfloat16 2024-06-26T07:14:53.4255828Z ++ pwd 2024-06-26T07:14:53.4256308Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:14:53.4256971Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:14:53.4265248Z + local name=inference 2024-06-26T07:14:53.4265680Z + shift 2024-06-26T07:14:53.4266034Z + local suite=timm_models 2024-06-26T07:14:53.4266392Z + shift 2024-06-26T07:14:53.4266663Z + local shard_id=1 2024-06-26T07:14:53.4266962Z + shift 2024-06-26T07:14:53.4267243Z + partition_flags=() 2024-06-26T07:14:53.4267581Z + local partition_flags 2024-06-26T07:14:53.4267977Z + [[ -n 2 ]] 2024-06-26T07:14:53.4268290Z + [[ -n 1 ]] 2024-06-26T07:14:53.4268866Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-06-26T07:14:53.4269535Z + [[ dynamo_eager_timm == *perf_compare* ]] 2024-06-26T07:14:53.4269988Z + [[ dynamo_eager_timm == *perf* ]] 2024-06-26T07:14:53.4270419Z + [[ dynamo_eager_timm == *aot_inductor* ]] 2024-06-26T07:14:53.4272038Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --backend eager --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 1 --output /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv 2024-06-26T07:14:59.2217520Z 2024-06-26T07:14:59.3690140Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:14:59.3690595Z 2024-06-26T07:14:59.4385167Z model.safetensors: 0% 0.00/29.6M [00:00 will be ignored 2024-06-26T07:20:36.2228571Z pass 2024-06-26T07:20:36.2275300Z TIMING: entire_frame_compile:11.6006 backend_compile:0.00066 2024-06-26T07:20:36.2276239Z STATS: call_* op count: 797 | FakeTensor.__torch_dispatch__:3417 | FakeTensorMode.__torch_dispatch__:5806 2024-06-26T07:20:36.2277472Z Dynamo produced 4 graphs covering 797 ops with 7 graph breaks (5 unique) 2024-06-26T07:20:40.0329757Z 2024-06-26T07:20:40.7351510Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:20:40.7351987Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:20:40.7352424Z cuda train mobilenetv2_100 2024-06-26T07:20:45.7045309Z W0626 07:20:45.703000 140155751662208 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:20:53.0109374Z pass 2024-06-26T07:20:53.0184221Z TIMING: entire_frame_compile:11.43331 backend_compile:0.00061 2024-06-26T07:20:53.0185078Z STATS: call_* op count: 796 | FakeTensor.__torch_dispatch__:3425 | FakeTensorMode.__torch_dispatch__:5846 2024-06-26T07:20:53.0185981Z Dynamo produced 3 graphs covering 796 ops with 7 graph breaks (5 unique) 2024-06-26T07:20:56.7892711Z 2024-06-26T07:20:57.5570025Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:20:57.5570816Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:20:57.5571530Z cuda train mobilenetv3_large_100 2024-06-26T07:21:02.2741459Z W0626 07:21:02.273000 139726423679616 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:21:10.6093235Z pass 2024-06-26T07:21:10.6148055Z TIMING: entire_frame_compile:12.22784 backend_compile:0.00067 2024-06-26T07:21:10.6149393Z STATS: call_* op count: 855 | FakeTensor.__torch_dispatch__:3909 | FakeTensorMode.__torch_dispatch__:6188 2024-06-26T07:21:10.6150369Z Dynamo produced 3 graphs covering 855 ops with 7 graph breaks (5 unique) 2024-06-26T07:21:14.3954226Z 2024-06-26T07:21:15.1183452Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:21:15.1183935Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:21:15.1184431Z cuda train mobilevit_s 2024-06-26T07:21:21.2265745Z pass 2024-06-26T07:21:21.2501474Z TIMING: entire_frame_compile:4.58093 backend_compile:0.00024 2024-06-26T07:21:21.2502373Z STATS: call_* op count: 503 | FakeTensor.__torch_dispatch__:1085 | FakeTensorMode.__torch_dispatch__:8426 2024-06-26T07:21:21.2503277Z Dynamo produced 3 graphs covering 503 ops with 6 graph breaks (5 unique) 2024-06-26T07:21:24.8585464Z 2024-06-26T07:21:26.0281509Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:21:26.0282341Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:21:26.0282815Z cuda train nfnet_l0 2024-06-26T07:21:33.9735443Z W0626 07:21:33.972000 140481448538752 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:21:46.9581538Z pass 2024-06-26T07:21:46.9822064Z TIMING: entire_frame_compile:18.79674 backend_compile:0.0008 2024-06-26T07:21:46.9823003Z STATS: call_* op count: 1232 | FakeTensorMode.__torch_dispatch__:7244 | FakeTensor.__torch_dispatch__:4553 2024-06-26T07:21:46.9823920Z Dynamo produced 3 graphs covering 1232 ops with 7 graph breaks (5 unique) 2024-06-26T07:21:50.8781143Z 2024-06-26T07:21:51.7961256Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:21:51.7961815Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:21:51.7962268Z cuda train pit_b_224 2024-06-26T07:21:57.7183984Z pass 2024-06-26T07:21:57.7677499Z TIMING: entire_frame_compile:2.26533 backend_compile:0.00015 2024-06-26T07:21:57.7678462Z STATS: call_* op count: 404 | FakeTensor.__torch_dispatch__:1145 | FakeTensorMode.__torch_dispatch__:6502 2024-06-26T07:21:57.7679673Z Dynamo produced 2 graphs covering 404 ops with 6 graph breaks (5 unique) 2024-06-26T07:22:01.3566357Z 2024-06-26T07:22:02.8755284Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:02.8755819Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:22:02.8756336Z cuda train pnasnet5large 2024-06-26T07:22:23.0696366Z pass 2024-06-26T07:22:23.2068698Z TIMING: entire_frame_compile:10.79054 backend_compile:0.00027 2024-06-26T07:22:23.2069801Z STATS: call_* op count: 963 | FakeTensor.__torch_dispatch__:4574 | FakeTensorMode.__torch_dispatch__:29501 2024-06-26T07:22:23.2071053Z Dynamo produced 2 graphs covering 963 ops with 5 graph breaks (4 unique) 2024-06-26T07:22:26.8621864Z 2024-06-26T07:22:28.0855327Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:28.0855808Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:22:28.0856266Z cuda train poolformer_m36 2024-06-26T07:22:38.9567936Z pass 2024-06-26T07:22:39.0120744Z TIMING: entire_frame_compile:7.14168 backend_compile:0.00025 2024-06-26T07:22:39.0121598Z STATS: call_* op count: 741 | FakeTensor.__torch_dispatch__:964 | FakeTensorMode.__torch_dispatch__:7412 2024-06-26T07:22:39.0122506Z Dynamo produced 2 graphs covering 741 ops with 6 graph breaks (5 unique) 2024-06-26T07:22:42.6708013Z 2024-06-26T07:22:43.3538417Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:43.3538894Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:43.3541116Z cuda train regnety_002 2024-06-26T07:22:48.5590027Z pass 2024-06-26T07:22:48.5625513Z TIMING: entire_frame_compile:4.42731 backend_compile:0.00018 2024-06-26T07:22:48.5626413Z STATS: call_* op count: 369 | FakeTensor.__torch_dispatch__:654 | FakeTensorMode.__torch_dispatch__:6322 2024-06-26T07:22:48.5627326Z Dynamo produced 2 graphs covering 369 ops with 6 graph breaks (5 unique) 2024-06-26T07:22:52.2218644Z 2024-06-26T07:22:53.1658319Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:53.1658829Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:53.1659622Z cuda train repvgg_a2 2024-06-26T07:23:00.6267738Z W0626 07:23:00.625000 140026022494848 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:23:07.8844138Z pass 2024-06-26T07:23:07.8982326Z TIMING: entire_frame_compile:12.49661 backend_compile:0.00068 2024-06-26T07:23:07.8983192Z STATS: call_* op count: 920 | FakeTensor.__torch_dispatch__:3516 | FakeTensorMode.__torch_dispatch__:6196 2024-06-26T07:23:07.8984117Z Dynamo produced 3 graphs covering 920 ops with 7 graph breaks (5 unique) 2024-06-26T07:23:11.6784778Z 2024-06-26T07:23:12.9850062Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:12.9850840Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:23:12.9851485Z cuda train res2net101_26w_4s 2024-06-26T07:23:22.5988649Z pass 2024-06-26T07:23:22.6355153Z TIMING: entire_frame_compile:5.51583 backend_compile:0.00023 2024-06-26T07:23:22.6356362Z STATS: call_* op count: 807 | FakeTensor.__torch_dispatch__:3148 | FakeTensorMode.__torch_dispatch__:22800 2024-06-26T07:23:22.6357607Z Dynamo produced 2 graphs covering 807 ops with 6 graph breaks (5 unique) 2024-06-26T07:23:26.3279372Z 2024-06-26T07:23:27.4252352Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:27.4252912Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:23:27.4256191Z cuda train res2net50_14w_8s 2024-06-26T07:23:34.8028858Z pass 2024-06-26T07:23:34.8239401Z TIMING: entire_frame_compile:4.66014 backend_compile:0.00023 2024-06-26T07:23:34.8241157Z STATS: call_* op count: 703 | FakeTensor.__torch_dispatch__:2773 | FakeTensorMode.__torch_dispatch__:20084 2024-06-26T07:23:34.8242984Z Dynamo produced 2 graphs covering 703 ops with 6 graph breaks (5 unique) 2024-06-26T07:23:38.4554641Z 2024-06-26T07:23:39.3736546Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:39.3737016Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:39.3737451Z cuda train res2next50 2024-06-26T07:23:44.3924851Z pass 2024-06-26T07:23:44.4170637Z TIMING: entire_frame_compile:2.88225 backend_compile:0.00015 2024-06-26T07:23:44.4171753Z STATS: call_* op count: 399 | FakeTensor.__torch_dispatch__:1603 | FakeTensorMode.__torch_dispatch__:11494 2024-06-26T07:23:44.4172857Z Dynamo produced 2 graphs covering 399 ops with 6 graph breaks (5 unique) 2024-06-26T07:23:48.0423237Z 2024-06-26T07:23:48.6162166Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:48.6162733Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:48.6163305Z cuda train resmlp_12_224 2024-06-26T07:23:51.9107208Z pass 2024-06-26T07:23:51.9222492Z TIMING: entire_frame_compile:2.10553 backend_compile:0.00013 2024-06-26T07:23:51.9223940Z STATS: call_* op count: 216 | FakeTensor.__torch_dispatch__:485 | FakeTensorMode.__torch_dispatch__:3734 2024-06-26T07:23:51.9224858Z Dynamo produced 2 graphs covering 216 ops with 6 graph breaks (5 unique) 2024-06-26T07:23:55.4623670Z 2024-06-26T07:23:56.8767461Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:23:56.8768093Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:23:56.8768637Z cuda train resnest101e 2024-06-26T07:24:08.6901095Z pass 2024-06-26T07:24:08.7467327Z TIMING: entire_frame_compile:6.22854 backend_compile:0.00026 2024-06-26T07:24:08.7468395Z STATS: call_* op count: 1022 | FakeTensor.__torch_dispatch__:3176 | FakeTensorMode.__torch_dispatch__:22010 2024-06-26T07:24:08.7469335Z Dynamo produced 2 graphs covering 1022 ops with 6 graph breaks (5 unique) 2024-06-26T07:24:12.3979077Z 2024-06-26T07:24:13.2650411Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:13.2650982Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:13.2651418Z cuda train rexnet_100 2024-06-26T07:24:20.0388114Z W0626 07:24:20.038000 139770749723264 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:24:30.6457676Z pass 2024-06-26T07:24:30.6574403Z TIMING: entire_frame_compile:16.28511 backend_compile:0.00083 2024-06-26T07:24:30.6575301Z STATS: call_* op count: 1104 | FakeTensor.__torch_dispatch__:5334 | FakeTensorMode.__torch_dispatch__:8819 2024-06-26T07:24:30.6576215Z Dynamo produced 3 graphs covering 1104 ops with 7 graph breaks (5 unique) 2024-06-26T07:24:34.6034101Z 2024-06-26T07:24:35.4851952Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:35.4852727Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:35.4853179Z cuda train sebotnet33ts_256 2024-06-26T07:24:41.6648428Z pass 2024-06-26T07:24:41.6844945Z TIMING: entire_frame_compile:4.40416 backend_compile:0.00025 2024-06-26T07:24:41.6845848Z STATS: call_* op count: 466 | FakeTensor.__torch_dispatch__:491 | FakeTensorMode.__torch_dispatch__:6912 2024-06-26T07:24:41.6846750Z Dynamo produced 3 graphs covering 466 ops with 6 graph breaks (5 unique) 2024-06-26T07:24:45.3054400Z 2024-06-26T07:24:46.2751289Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:46.2752212Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:46.2753061Z cuda train selecsls42b 2024-06-26T07:24:49.1999156Z pass 2024-06-26T07:24:49.2271397Z TIMING: entire_frame_compile:1.5395 backend_compile:0.00011 2024-06-26T07:24:49.2272257Z STATS: call_* op count: 137 | FakeTensor.__torch_dispatch__:834 | FakeTensorMode.__torch_dispatch__:5298 2024-06-26T07:24:49.2273165Z Dynamo produced 2 graphs covering 137 ops with 6 graph breaks (5 unique) 2024-06-26T07:24:52.8206158Z 2024-06-26T07:24:53.5160882Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:53.5161777Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:24:53.5162657Z cuda train spnasnet_100 2024-06-26T07:24:59.3916812Z W0626 07:24:59.390000 140256721621632 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:25:08.3902086Z pass 2024-06-26T07:25:08.3957830Z TIMING: entire_frame_compile:13.92448 backend_compile:0.00076 2024-06-26T07:25:08.3958788Z STATS: call_* op count: 976 | FakeTensor.__torch_dispatch__:4175 | FakeTensorMode.__torch_dispatch__:7080 2024-06-26T07:25:08.3959712Z Dynamo produced 3 graphs covering 976 ops with 7 graph breaks (5 unique) 2024-06-26T07:25:12.2555909Z 2024-06-26T07:25:13.9047755Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:13.9048278Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:25:13.9048717Z cuda train swin_base_patch4_window7_224 2024-06-26T07:25:25.4879028Z W0626 07:25:25.487000 140174091854464 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:25:38.1883938Z pass 2024-06-26T07:25:38.2533776Z TIMING: entire_frame_compile:19.02781 backend_compile:0.00124 2024-06-26T07:25:38.2535516Z STATS: call_* op count: 2473 | FakeTensor.__torch_dispatch__:8135 | FakeTensorMode.__torch_dispatch__:17698 2024-06-26T07:25:38.2537347Z Dynamo produced 3 graphs covering 2473 ops with 7 graph breaks (5 unique) 2024-06-26T07:25:42.1968254Z 2024-06-26T07:25:45.0183808Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:25:45.0184456Z loading model: 0it [00:02, ?it/s] 2024-06-26T07:25:45.0185084Z cuda train swsl_resnext101_32x16d 2024-06-26T07:25:59.3426905Z pass 2024-06-26T07:25:59.4998723Z TIMING: entire_frame_compile:3.50424 backend_compile:0.00017 2024-06-26T07:25:59.4999835Z STATS: call_* op count: 415 | FakeTensor.__torch_dispatch__:1942 | FakeTensorMode.__torch_dispatch__:13040 2024-06-26T07:25:59.5000860Z Dynamo produced 2 graphs covering 415 ops with 6 graph breaks (5 unique) 2024-06-26T07:26:03.0789109Z 2024-06-26T07:26:03.8503040Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:03.8503623Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:03.8504072Z cuda train tf_efficientnet_b0 2024-06-26T07:26:09.5957786Z pass 2024-06-26T07:26:09.6058063Z TIMING: entire_frame_compile:4.7994 backend_compile:0.00019 2024-06-26T07:26:09.6059021Z STATS: call_* op count: 371 | FakeTensorMode.__torch_dispatch__:7280 | FakeTensor.__torch_dispatch__:805 2024-06-26T07:26:09.6060219Z Dynamo produced 2 graphs covering 371 ops with 6 graph breaks (5 unique) 2024-06-26T07:26:13.2394936Z 2024-06-26T07:26:14.1147742Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:14.1148261Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:14.1148703Z cuda train tf_mixnet_l 2024-06-26T07:26:22.7582725Z pass 2024-06-26T07:26:22.7756817Z TIMING: entire_frame_compile:7.26525 backend_compile:0.00026 2024-06-26T07:26:22.7757714Z STATS: call_* op count: 692 | FakeTensorMode.__torch_dispatch__:11113 | FakeTensor.__torch_dispatch__:1195 2024-06-26T07:26:22.7758642Z Dynamo produced 2 graphs covering 692 ops with 6 graph breaks (5 unique) 2024-06-26T07:26:26.4843058Z 2024-06-26T07:26:27.2943170Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:27.2943649Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:27.2944086Z cuda train tinynet_a 2024-06-26T07:26:33.6665739Z pass 2024-06-26T07:26:33.6759049Z TIMING: entire_frame_compile:5.36561 backend_compile:0.00022 2024-06-26T07:26:33.6760074Z STATS: call_* op count: 435 | FakeTensor.__torch_dispatch__:965 | FakeTensorMode.__torch_dispatch__:8580 2024-06-26T07:26:33.6760978Z Dynamo produced 2 graphs covering 435 ops with 6 graph breaks (5 unique) 2024-06-26T07:26:37.3851097Z 2024-06-26T07:26:38.2288894Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:38.2289726Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:38.2290162Z cuda train tnt_s_patch16_224 2024-06-26T07:26:44.8547007Z W0626 07:26:44.853000 140522834457216 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:26:58.1970106Z pass 2024-06-26T07:26:58.2225334Z TIMING: entire_frame_compile:16.91435 backend_compile:0.00113 2024-06-26T07:26:58.2226296Z STATS: call_* op count: 1922 | FakeTensor.__torch_dispatch__:8864 | FakeTensorMode.__torch_dispatch__:15132 2024-06-26T07:26:58.2227412Z Dynamo produced 3 graphs covering 1922 ops with 7 graph breaks (5 unique) 2024-06-26T07:27:02.1412496Z 2024-06-26T07:27:03.3285607Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:27:03.3286111Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:27:03.3286545Z cuda train twins_pcpvt_base 2024-06-26T07:27:11.8334773Z W0626 07:27:11.832000 140112708825728 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:27:32.0930347Z pass 2024-06-26T07:27:32.1338888Z TIMING: entire_frame_compile:24.88651 backend_compile:0.00162 2024-06-26T07:27:32.1341090Z STATS: call_* op count: 2517 | FakeTensor.__torch_dispatch__:13234 | FakeTensorMode.__torch_dispatch__:18414 2024-06-26T07:27:32.1342935Z Dynamo produced 3 graphs covering 2517 ops with 7 graph breaks (5 unique) 2024-06-26T07:27:36.3135159Z 2024-06-26T07:27:37.3284059Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:27:37.3284531Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:27:37.3287412Z cuda train visformer_small 2024-06-26T07:27:41.3382337Z W0626 07:27:41.337000 140396805829248 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:27:47.7564782Z pass 2024-06-26T07:27:47.7781372Z TIMING: entire_frame_compile:8.27061 backend_compile:0.00049 2024-06-26T07:27:47.7782365Z STATS: call_* op count: 699 | FakeTensor.__torch_dispatch__:3056 | FakeTensorMode.__torch_dispatch__:6044 2024-06-26T07:27:47.7783348Z Dynamo produced 3 graphs covering 699 ops with 7 graph breaks (5 unique) 2024-06-26T07:27:51.4543638Z 2024-06-26T07:27:52.8398137Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:27:52.8398684Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:27:52.8399131Z cuda train vit_base_patch16_224 2024-06-26T07:27:59.2005672Z W0626 07:27:59.199000 139902382666368 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:28:04.8752284Z pass 2024-06-26T07:28:04.9321766Z TIMING: entire_frame_compile:7.32958 backend_compile:0.00055 2024-06-26T07:28:04.9323467Z STATS: call_* op count: 829 | FakeTensor.__torch_dispatch__:3890 | FakeTensorMode.__torch_dispatch__:5333 2024-06-26T07:28:04.9324479Z Dynamo produced 3 graphs covering 829 ops with 7 graph breaks (5 unique) 2024-06-26T07:28:08.5977152Z 2024-06-26T07:28:09.5949247Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:28:09.5949858Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:28:09.5950407Z cuda train volo_d1_224 2024-06-26T07:28:15.6206951Z W0626 07:28:15.619000 139690238689920 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:28:25.1774480Z pass 2024-06-26T07:28:25.1963744Z TIMING: entire_frame_compile:12.64879 backend_compile:0.00088 2024-06-26T07:28:25.1964776Z STATS: call_* op count: 1410 | FakeTensor.__torch_dispatch__:6377 | FakeTensorMode.__torch_dispatch__:11263 2024-06-26T07:28:25.1965731Z Dynamo produced 3 graphs covering 1410 ops with 7 graph breaks (5 unique) 2024-06-26T07:28:28.9974586Z 2024-06-26T07:28:32.1731765Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:28:32.1732524Z loading model: 0it [00:03, ?it/s] 2024-06-26T07:28:32.1732965Z cuda train xcit_large_24_p8_224 2024-06-26T07:28:32.2615159Z WARNING:common:fp64 golden ref were not generated for xcit_large_24_p8_224. Setting accuracy check to cosine 2024-06-26T07:28:41.9472013Z W0626 07:28:41.946000 140267065262720 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:29:06.7042781Z pass_due_to_skip 2024-06-26T07:29:06.9461788Z TIMING: entire_frame_compile:30.80976 backend_compile:0.00206 2024-06-26T07:29:06.9462699Z STATS: call_* op count: 3114 | FakeTensor.__torch_dispatch__:15292 | FakeTensorMode.__torch_dispatch__:22733 2024-06-26T07:29:06.9463649Z Dynamo produced 3 graphs covering 3114 ops with 7 graph breaks (5 unique) 2024-06-26T07:29:08.4793559Z accuracy pass_rate=96.77% 2024-06-26T07:29:08.4797712Z calls_captured gmean=744.81x mean=934.323x 2024-06-26T07:29:08.4800922Z unique_graphs gmean=2.49x mean=2.548x 2024-06-26T07:29:08.4804418Z graph_breaks gmean=6.39x mean=6.419x 2024-06-26T07:29:08.4807583Z unique_graph_breaks gmean=4.96x mean=4.968x 2024-06-26T07:29:08.4811162Z autograd_captures gmean=0.00x mean=0.000x 2024-06-26T07:29:08.4814166Z autograd_compiles gmean=0.00x mean=0.000x 2024-06-26T07:29:08.4817927Z cudagraph_skips gmean=0.00x mean=0.000x 2024-06-26T07:29:09.0778624Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy//dynamo_eager_timm_training.csv 2024-06-26T07:29:09.3195527Z mixnet_l PASS 2024-06-26T07:29:09.3200738Z mnasnet_100 PASS 2024-06-26T07:29:09.3205107Z mobilenetv2_100 PASS 2024-06-26T07:29:09.3208516Z mobilenetv3_large_100 PASS 2024-06-26T07:29:09.3213368Z mobilevit_s PASS 2024-06-26T07:29:09.3217935Z nfnet_l0 PASS 2024-06-26T07:29:09.3222349Z pit_b_224 PASS 2024-06-26T07:29:09.3226799Z pnasnet5large PASS 2024-06-26T07:29:09.3231426Z poolformer_m36 PASS 2024-06-26T07:29:09.3235820Z regnety_002 PASS 2024-06-26T07:29:09.3240368Z repvgg_a2 PASS 2024-06-26T07:29:09.3244566Z res2net101_26w_4s PASS 2024-06-26T07:29:09.3248853Z res2net50_14w_8s PASS 2024-06-26T07:29:09.3253679Z res2next50 PASS 2024-06-26T07:29:09.3257937Z resmlp_12_224 PASS 2024-06-26T07:29:09.3262436Z resnest101e PASS 2024-06-26T07:29:09.3266818Z rexnet_100 PASS 2024-06-26T07:29:09.3271160Z sebotnet33ts_256 PASS 2024-06-26T07:29:09.3275550Z selecsls42b PASS 2024-06-26T07:29:09.3279969Z spnasnet_100 PASS 2024-06-26T07:29:09.3284312Z swin_base_patch4_window7_224 PASS 2024-06-26T07:29:09.3288769Z swsl_resnext101_32x16d PASS 2024-06-26T07:29:09.3293619Z tf_efficientnet_b0 PASS 2024-06-26T07:29:09.3298267Z tf_mixnet_l PASS 2024-06-26T07:29:09.3302625Z tinynet_a PASS 2024-06-26T07:29:09.3307038Z tnt_s_patch16_224 PASS 2024-06-26T07:29:09.3311442Z twins_pcpvt_base PASS 2024-06-26T07:29:09.3315716Z visformer_small PASS 2024-06-26T07:29:09.3320211Z vit_base_patch16_224 PASS 2024-06-26T07:29:09.3324573Z volo_d1_224 PASS 2024-06-26T07:29:09.3329008Z xcit_large_24_p8_224 XFAIL 2024-06-26T07:29:09.3704959Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy//dynamo_eager_timm_training.csv 2024-06-26T07:29:09.6116475Z mixnet_l PASS 2024-06-26T07:29:09.6121746Z mnasnet_100 PASS 2024-06-26T07:29:09.6126099Z mobilenetv2_100 PASS 2024-06-26T07:29:09.6129631Z mobilenetv3_large_100 PASS 2024-06-26T07:29:09.6135248Z mobilevit_s PASS 2024-06-26T07:29:09.6140323Z nfnet_l0 PASS 2024-06-26T07:29:09.6145614Z pit_b_224 PASS 2024-06-26T07:29:09.6150902Z pnasnet5large PASS 2024-06-26T07:29:09.6156116Z poolformer_m36 PASS 2024-06-26T07:29:09.6161044Z regnety_002 PASS 2024-06-26T07:29:09.6166122Z repvgg_a2 PASS 2024-06-26T07:29:09.6171224Z res2net101_26w_4s PASS 2024-06-26T07:29:09.6176731Z res2net50_14w_8s PASS 2024-06-26T07:29:09.6181941Z res2next50 PASS 2024-06-26T07:29:09.6187070Z resmlp_12_224 PASS 2024-06-26T07:29:09.6192271Z resnest101e PASS 2024-06-26T07:29:09.6197495Z rexnet_100 PASS 2024-06-26T07:29:09.6202687Z sebotnet33ts_256 PASS 2024-06-26T07:29:09.6207776Z selecsls42b PASS 2024-06-26T07:29:09.6213356Z spnasnet_100 PASS 2024-06-26T07:29:09.6218796Z swin_base_patch4_window7_224 PASS 2024-06-26T07:29:09.6223505Z swsl_resnext101_32x16d PASS 2024-06-26T07:29:09.6228741Z tf_efficientnet_b0 PASS 2024-06-26T07:29:09.6233764Z tf_mixnet_l PASS 2024-06-26T07:29:09.6238862Z tinynet_a PASS 2024-06-26T07:29:09.6243796Z tnt_s_patch16_224 PASS 2024-06-26T07:29:09.6248975Z twins_pcpvt_base PASS 2024-06-26T07:29:09.6254847Z visformer_small PASS 2024-06-26T07:29:09.6259931Z vit_base_patch16_224 PASS 2024-06-26T07:29:09.6265087Z volo_d1_224 PASS 2024-06-26T07:29:09.6270303Z xcit_large_24_p8_224 PASS 2024-06-26T07:29:09.6653139Z + cleanup_workspace 2024-06-26T07:29:09.6654324Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-06-26T07:29:09.6655510Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-06-26T07:29:09.6656531Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-06-26T07:29:09.6657343Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-06-26T07:29:09.6658437Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-06-26T07:29:09.6659361Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-06-26T07:29:09.6660371Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-06-26T07:29:10.1187590Z ##[group]Run cat test/**/*_toprint.log || true 2024-06-26T07:29:10.1188117Z cat test/**/*_toprint.log || true 2024-06-26T07:29:10.1195757Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:29:10.1196252Z env: 2024-06-26T07:29:10.1196526Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:10.1196971Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:10.1197690Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:10.1198325Z ##[endgroup] 2024-06-26T07:29:10.1260114Z cat: test/**/*_toprint.log: No such file or directory 2024-06-26T07:29:10.1288520Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-06-26T07:29:10.1288988Z kill "$MONITOR_SCRIPT_PID" 2024-06-26T07:29:10.1297086Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:29:10.1297597Z env: 2024-06-26T07:29:10.1297871Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:10.1298324Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:10.1299061Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:10.1299710Z MONITOR_SCRIPT_PID: 12435 2024-06-26T07:29:10.1300061Z ##[endgroup] 2024-06-26T07:29:10.1484948Z Prepare all required actions 2024-06-26T07:29:10.1485385Z Getting action download info 2024-06-26T07:29:10.3137619Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-06-26T07:29:10.4735475Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-06-26T07:29:10.4735947Z with: 2024-06-26T07:29:10.4736457Z file-suffix: test-dynamo_eager_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688916077 2024-06-26T07:29:10.4737098Z s3-bucket: gha-artifacts 2024-06-26T07:29:10.4737426Z env: 2024-06-26T07:29:10.4737699Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:10.4738150Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:10.4738896Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:10.4739527Z ##[endgroup] 2024-06-26T07:29:10.4763967Z ##[group]Run # Remove any previous test jsons if they exist 2024-06-26T07:29:10.4764618Z # Remove any previous test jsons if they exist 2024-06-26T07:29:10.4765132Z rm -f test-jsons-*.zip 2024-06-26T07:29:10.4765650Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-06-26T07:29:10.4773742Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:29:10.4774246Z env: 2024-06-26T07:29:10.4774522Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:10.4774965Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:10.4775705Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:10.4776587Z FILE_SUFFIX: test-dynamo_eager_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688916077 2024-06-26T07:29:10.4777220Z ##[endgroup] 2024-06-26T07:29:10.4951579Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-06-26T07:29:10.4980524Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-06-26T07:29:10.4981220Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-06-26T07:29:10.4987120Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-06-26T07:29:10.5032227Z ##[group]Run # Remove any previous test reports if they exist 2024-06-26T07:29:10.5032857Z # Remove any previous test reports if they exist 2024-06-26T07:29:10.5033382Z rm -f test-reports-*.zip 2024-06-26T07:29:10.5033973Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-06-26T07:29:10.5041603Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:29:10.5042093Z env: 2024-06-26T07:29:10.5042370Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:10.5042822Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:10.5043677Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:10.5044545Z FILE_SUFFIX: test-dynamo_eager_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688916077 2024-06-26T07:29:10.5045159Z ##[endgroup] 2024-06-26T07:29:10.5211520Z adding: test/test-reports/inference_timm_models.csv (deflated 68%) 2024-06-26T07:29:10.5212501Z adding: test/test-reports/training_timm_models.csv (deflated 66%) 2024-06-26T07:29:10.5216493Z adding: test/test-reports/training_timm_model_graph_breaks.csv (deflated 97%) 2024-06-26T07:29:10.5217427Z adding: test/test-reports/training_timm_model_graph_break_deduped.csv (deflated 74%) 2024-06-26T07:29:10.5242146Z ##[group]Run # Remove any previous usage logs if they exist 2024-06-26T07:29:10.5242786Z # Remove any previous usage logs if they exist 2024-06-26T07:29:10.5243287Z rm -f logs-*.zip 2024-06-26T07:29:10.5243957Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-06-26T07:29:10.5244767Z # so check to see if the file exists first 2024-06-26T07:29:10.5245267Z if [ -f 'usage_log.txt' ]; then 2024-06-26T07:29:10.5245801Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-06-26T07:29:10.5246292Z fi 2024-06-26T07:29:10.5246775Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-06-26T07:29:10.5247355Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-06-26T07:29:10.5247855Z fi 2024-06-26T07:29:10.5255347Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:29:10.5255847Z env: 2024-06-26T07:29:10.5256121Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:10.5256576Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:10.5257315Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:10.5258188Z FILE_SUFFIX: test-dynamo_eager_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688916077 2024-06-26T07:29:10.5258811Z ##[endgroup] 2024-06-26T07:29:10.5296095Z adding: usage_log.txt (deflated 92%) 2024-06-26T07:29:10.5362119Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-06-26T07:29:10.5362833Z # Remove any previous debugging artifacts if they exist 2024-06-26T07:29:10.5363394Z rm -f debug-*.zip 2024-06-26T07:29:10.5363774Z if [ -d 'test/debug' ]; then 2024-06-26T07:29:10.5364268Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-06-26T07:29:10.5364746Z fi 2024-06-26T07:29:10.5371577Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:29:10.5372286Z env: 2024-06-26T07:29:10.5372612Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:10.5373062Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:10.5373790Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:10.5374662Z FILE_SUFFIX: test-dynamo_eager_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688916077 2024-06-26T07:29:10.5375291Z ##[endgroup] 2024-06-26T07:29:10.5447767Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T07:29:10.5448202Z with: 2024-06-26T07:29:10.5448491Z s3-bucket: gha-artifacts 2024-06-26T07:29:10.5448914Z s3-prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:29:10.5449402Z retention-days: 14 2024-06-26T07:29:10.5449728Z if-no-files-found: warn 2024-06-26T07:29:10.5450087Z path: test-jsons-*.zip 2024-06-26T07:29:10.5450428Z name: artifact 2024-06-26T07:29:10.5450829Z region: us-east-1 2024-06-26T07:29:10.5451125Z env: 2024-06-26T07:29:10.5451406Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:10.5451863Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:10.5452733Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:10.5453372Z ##[endgroup] 2024-06-26T07:29:10.8538362Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T07:29:10.8539149Z With the provided path, there will be 1 file uploaded 2024-06-26T07:29:10.8539759Z Uploading to s3 prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:29:10.8564616Z Starting upload of test-jsons-test-dynamo_eager_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688916077.zip 2024-06-26T07:29:10.9769341Z Finished upload of test-jsons-test-dynamo_eager_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688916077.zip 2024-06-26T07:29:10.9897364Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T07:29:10.9897806Z with: 2024-06-26T07:29:10.9898086Z s3-bucket: gha-artifacts 2024-06-26T07:29:10.9898507Z s3-prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:29:10.9898971Z retention-days: 14 2024-06-26T07:29:10.9899300Z if-no-files-found: error 2024-06-26T07:29:10.9899662Z path: test-reports-*.zip 2024-06-26T07:29:10.9900008Z name: artifact 2024-06-26T07:29:10.9900303Z region: us-east-1 2024-06-26T07:29:10.9900605Z env: 2024-06-26T07:29:10.9900878Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:10.9901341Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:10.9902066Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:10.9902703Z ##[endgroup] 2024-06-26T07:29:11.2962764Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T07:29:11.2964326Z With the provided path, there will be 1 file uploaded 2024-06-26T07:29:11.2965541Z Uploading to s3 prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:29:11.2989387Z Starting upload of test-reports-test-dynamo_eager_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688916077.zip 2024-06-26T07:29:11.3997242Z Finished upload of test-reports-test-dynamo_eager_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688916077.zip 2024-06-26T07:29:11.4127934Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T07:29:11.4128373Z with: 2024-06-26T07:29:11.4128652Z s3-bucket: gha-artifacts 2024-06-26T07:29:11.4129072Z s3-prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:29:11.4129547Z retention-days: 14 2024-06-26T07:29:11.4129873Z if-no-files-found: ignore 2024-06-26T07:29:11.4130226Z path: logs-*.zip 2024-06-26T07:29:11.4130529Z name: artifact 2024-06-26T07:29:11.4130939Z region: us-east-1 2024-06-26T07:29:11.4131241Z env: 2024-06-26T07:29:11.4131511Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:11.4131970Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:11.4132972Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:11.4133608Z ##[endgroup] 2024-06-26T07:29:11.7188235Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T07:29:11.7188865Z With the provided path, there will be 1 file uploaded 2024-06-26T07:29:11.7189487Z Uploading to s3 prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:29:11.7214817Z Starting upload of logs-test-dynamo_eager_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688916077.zip 2024-06-26T07:29:11.8204717Z Finished upload of logs-test-dynamo_eager_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688916077.zip 2024-06-26T07:29:11.8339056Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T07:29:11.8339491Z with: 2024-06-26T07:29:11.8339774Z s3-bucket: gha-artifacts 2024-06-26T07:29:11.8340194Z s3-prefix: pytorch/pytorch/9673646806/1/artifact 2024-06-26T07:29:11.8340656Z retention-days: 14 2024-06-26T07:29:11.8340997Z if-no-files-found: ignore 2024-06-26T07:29:11.8341350Z path: debug-*.zip 2024-06-26T07:29:11.8341659Z name: artifact 2024-06-26T07:29:11.8341953Z region: us-east-1 2024-06-26T07:29:11.8342251Z env: 2024-06-26T07:29:11.8342524Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:11.8342972Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:11.8343689Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:11.8344321Z ##[endgroup] 2024-06-26T07:29:12.1338749Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-06-26T07:29:12.1473033Z ##[group]Run # shellcheck disable=SC2156 2024-06-26T07:29:12.1473509Z # shellcheck disable=SC2156 2024-06-26T07:29:12.1474298Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-06-26T07:29:12.1482144Z shell: /usr/bin/bash -e {0} 2024-06-26T07:29:12.1482498Z env: 2024-06-26T07:29:12.1482772Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:12.1483222Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:12.1483945Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:12.1484584Z ##[endgroup] 2024-06-26T07:29:12.3097905Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-06-26T07:29:12.3098673Z with: 2024-06-26T07:29:12.3098931Z env: 2024-06-26T07:29:12.3099207Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:12.3099649Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:12.3100376Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:12.3101022Z ##[endgroup] 2024-06-26T07:29:12.3119227Z ##[group]Run set -eou pipefail 2024-06-26T07:29:12.3119785Z set -eou pipefail 2024-06-26T07:29:12.3120257Z  2024-06-26T07:29:12.3120770Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-06-26T07:29:12.3121400Z for _ in $(seq 1440); do 2024-06-26T07:29:12.3121876Z  # Break if no ssh session exists anymore 2024-06-26T07:29:12.3122374Z  if [ "$(who)" = "" ]; then 2024-06-26T07:29:12.3122782Z  break 2024-06-26T07:29:12.3123129Z  fi 2024-06-26T07:29:12.3123428Z  echo "." 2024-06-26T07:29:12.3123755Z  sleep 5 2024-06-26T07:29:12.3124074Z done 2024-06-26T07:29:12.3131521Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:29:12.3132015Z env: 2024-06-26T07:29:12.3132422Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:12.3132883Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:12.3133611Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:12.3134241Z ##[endgroup] 2024-06-26T07:29:12.3158579Z Holding runner for 2 hours until all ssh sessions have logged out 2024-06-26T07:29:12.3194760Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:29:12.3195529Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:29:12.3196121Z # shellcheck disable=SC2046 2024-06-26T07:29:12.3196637Z docker stop $(docker ps -q) || true 2024-06-26T07:29:12.3197122Z # Prune all of the docker images 2024-06-26T07:29:12.3197571Z docker system prune -af 2024-06-26T07:29:12.3205152Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:29:12.3205650Z env: 2024-06-26T07:29:12.3205922Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:29:12.3206386Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:29:12.3207118Z DOCKER_CONTAINER_ID: fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:12.3207753Z ##[endgroup] 2024-06-26T07:29:12.6415711Z fba69756b470 2024-06-26T07:29:13.3726465Z Deleted Containers: 2024-06-26T07:29:13.3727084Z fba69756b470edd502e63c9202274723c31e222bbdebdf953a5ce1e901f9d5c2 2024-06-26T07:29:13.3727597Z 2024-06-26T07:29:18.4034997Z Deleted Images: 2024-06-26T07:29:18.4036666Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:29:18.4039685Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb 2024-06-26T07:29:18.4041405Z deleted: sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd 2024-06-26T07:29:18.4042454Z deleted: sha256:754ead7012f0b7691885a9660f02e80be6cee8c5665486fb995c1fb06adb6e92 2024-06-26T07:29:18.4043271Z deleted: sha256:fbe4496d8830e11be86d8f222522e7956f32f85bfc06b05ea53961f34a58af74 2024-06-26T07:29:18.4044091Z deleted: sha256:1c83814aa1a1a7e0cc998e3479bdab8e8766b975560cbf00690acf39ed899a80 2024-06-26T07:29:18.4044961Z deleted: sha256:e9ea8e61942c7f8ae4aab97b8d102b01d6bb4b6cf2a605808cd34130a17d8a0f 2024-06-26T07:29:18.4045796Z deleted: sha256:acc5fc32588cd95e082a6ce35e48d0f303ae8f96e3ab65c4e375b85e1f3c4a92 2024-06-26T07:29:18.4046622Z deleted: sha256:a93e53400904d29e4185d3be942ea34f19572ea438962788ae4e9b3d9bf4c498 2024-06-26T07:29:18.4047533Z deleted: sha256:c97092b88b5e4c77c68bbc4b51afab64e38719af1ff58013ec49a696cdd40a6e 2024-06-26T07:29:18.4048346Z deleted: sha256:b132889c90d67ffa53655846603e3fcc2372c23f04dbdbc54690a39d304eb89b 2024-06-26T07:29:18.4049153Z deleted: sha256:915d54081ed607710624abe3f2490114c745858831f4fc13396bd3b4b9a121bc 2024-06-26T07:29:18.4049965Z deleted: sha256:bfa8d446c498c87f535aef102695564de1b22c755e6c63944ae2f5073a2b31ec 2024-06-26T07:29:18.4050943Z deleted: sha256:e71460fb363c29e70bbe357d680aaa7ae694f094ef2f2713609158ad90611744 2024-06-26T07:29:18.4052038Z deleted: sha256:84079bb1b4ee1d0119ae3d3e701de460a1324912297c675425c9a36d30fcdb0f 2024-06-26T07:29:18.4053305Z deleted: sha256:d86ac3ca148d4758cb03b493fdd49109d8f1a2c7c85c4b755493c3c4529553e5 2024-06-26T07:29:18.4054131Z deleted: sha256:532b3ab97c44a673a0d5f62ee46fa6a83248a84de8877754a612355be56ed0a3 2024-06-26T07:29:18.4054944Z deleted: sha256:617e2a35f25ae24500c88c0c9b8475f0687fbcbed3d200013b4c99c6d8941aff 2024-06-26T07:29:18.4055768Z deleted: sha256:b88120bcd5a26a660c09c5df13176e0ef098fb2c8cdb57d74ab0796a3d1194d6 2024-06-26T07:29:18.4056578Z deleted: sha256:c016b2e7764735243205fb76fa9a273bb3dd4bf05532288e3a613ce022131ac3 2024-06-26T07:29:18.4057446Z deleted: sha256:bcaca462c390deddd2995de2d245a7495e7d2e1e94f0edad30a573c83d0588fd 2024-06-26T07:29:18.4058281Z deleted: sha256:9e260eba003d5c25d7b410e5c4c3093845d9e46611896840901c689d7fbf8ee6 2024-06-26T07:29:18.4059090Z deleted: sha256:3e62b86f8b911a9b59c62d55f063e71683ae6a01c22418db5d27930712adaad6 2024-06-26T07:29:18.4059897Z deleted: sha256:905920ccb1d88449b8dde686c34a4b2a8b6f59e78ac44916b189ea3593b9fbbb 2024-06-26T07:29:18.4060701Z deleted: sha256:961201237e03acd5061406238dd7a3c1343ac888d7583212cd0ab5ae1b4f0540 2024-06-26T07:29:18.4061517Z deleted: sha256:70047a2aac62159fa28fb4dbd4567ec2ffbab68fe046967e09acb0eb3e5cb9cb 2024-06-26T07:29:18.4062348Z deleted: sha256:1ec680ad781436ae65fa987f5fa1ed032ecd5c82a6e35eea6707de501f62c0f4 2024-06-26T07:29:18.4063440Z deleted: sha256:ef859623a50e85abf6e6c1dbd6188aa765a66cc7b2bd618a1cbe4825b7b2bd91 2024-06-26T07:29:18.4064282Z deleted: sha256:404ade5cacff59773b93ab438bfa64cd3a171d07f6f168bdc4a5d4bfa0b43526 2024-06-26T07:29:18.4065111Z deleted: sha256:5b27de6ee3d9f2f18217bf18dec725f7b4954c0e48c85d35752634c22d62209b 2024-06-26T07:29:18.4065931Z deleted: sha256:498e39c575b3fa069acaac7e46b91b4e54ae3a05f2ada5c4347091f346f25d4c 2024-06-26T07:29:18.4066729Z deleted: sha256:1405919d2694d65669b1f4c842282021afe89208ce86570e96464903733da2d0 2024-06-26T07:29:18.4067532Z deleted: sha256:d5b41e5d70896c0472fc3e6da0516a17d89cd6635dc75010ecddc31ce9690fad 2024-06-26T07:29:18.4068350Z deleted: sha256:18e9379b62b9d0bb40293d28a652eafaee66a499f66f0776bf14e37a513101b7 2024-06-26T07:29:18.4069172Z deleted: sha256:a781f80ed00ff2926a699fd01a309bb0b4fb7d5655eb652c205bb9c7f811035f 2024-06-26T07:29:18.4069976Z deleted: sha256:c0430973b7cf759075330ee29c79bfd1312c91e6033d9dd883b4b1c29743a171 2024-06-26T07:29:18.4070800Z deleted: sha256:e733b3d66efb9342e5a283376c0e1ddffa47a42fecc26e61ecaf0a3472885d97 2024-06-26T07:29:18.4071621Z deleted: sha256:8929c0b2047f74b84c82c39eff9077a92ae1a828cde00c9ab78fb3e0ed5e0283 2024-06-26T07:29:18.4072434Z deleted: sha256:f5b56155fb58a0a60fe6391d3581cf91610ed69d4f0a321257c19944da296a52 2024-06-26T07:29:18.4073233Z deleted: sha256:b54d2cb355238d1aa91e58df93064447b80f53706d1ca62823acc834f5ac54fe 2024-06-26T07:29:18.4074040Z deleted: sha256:e7311285af9eeddb51aa5629aa98386e89f1456ff32843b6aee6443247edf66c 2024-06-26T07:29:18.4075006Z deleted: sha256:67657ceec347da8dc8085d5731633efeac115ba677fa4b7c5966c4260f7abab0 2024-06-26T07:29:18.4075829Z deleted: sha256:b496e50409dbf414930fc9b4437aa0aa4ce27bc7840e050ae459aaffcf9aca86 2024-06-26T07:29:18.4076641Z deleted: sha256:8f5276170cc4a60d36ad7b964a502306b29546254cedcbcb101ea9123497d2c0 2024-06-26T07:29:18.4077432Z deleted: sha256:53d524d1564745e96a7e515462171b73184d9f1ff5454b32fcfbbca39d361467 2024-06-26T07:29:18.4078249Z deleted: sha256:2acbadb00d69b6aca575f04baba4ea26ae2b5c2d85f09945004ad93a78a8eb75 2024-06-26T07:29:18.4079177Z deleted: sha256:02665efeff878aeb603c1edb2e21cb700c171c025d64f5aae1b2b20b12f3964a 2024-06-26T07:29:18.4080007Z deleted: sha256:19bd23e9a51dfe883a8b8d2ff6d66ff1b3476b8912856406bb79b81762aa6d4b 2024-06-26T07:29:18.4080824Z deleted: sha256:73aa8637bf9299d6298dabd858480c274a62117388f2ca1d5e9eb15c6d1e4155 2024-06-26T07:29:18.4081636Z deleted: sha256:5f8968406305f348b1caebc0e035187d6e9965ace7b9d5f2f21114d37c9fe2e1 2024-06-26T07:29:18.4082456Z deleted: sha256:d827104914db0c503cf5c81f13e2456bf4efce42d4cf1f8d5d70838207278bee 2024-06-26T07:29:18.4083273Z deleted: sha256:867b219724f2f1aaf56f42e3e9f03a5d64bdca8a6a72407efd10328aeea5d510 2024-06-26T07:29:18.4084094Z deleted: sha256:b9ff2a7cceb9237b26bcb17728edf8a558eec5a76f63626e84f80541dfc0f1d4 2024-06-26T07:29:18.4084924Z deleted: sha256:be73f509106819007ce2b0f5a1301ac56ae583cb6c1fb69bee93efa0840368f1 2024-06-26T07:29:18.4085746Z deleted: sha256:2a11c07c915c3fdbb4e73096dea73ea950c970d50fe1a5764060bd62b7b7cf2c 2024-06-26T07:29:18.4086573Z deleted: sha256:5e9989622e2abc1c72f8184343507b45d75e4ff971edb68b96cbcd75ae484ff5 2024-06-26T07:29:18.4087392Z deleted: sha256:05ce642bd9cf5a9205bff107df28b1d6cef7d5f6ab3ea944d28c032ec61270df 2024-06-26T07:29:18.4088210Z deleted: sha256:017702617f6c07b73f2088678f4710b794cfe897a213a5cf122a0e2519eae0ac 2024-06-26T07:29:18.4089018Z deleted: sha256:44b3b37bb1ca39dc5cf356de41a23a5c12f605a1992813e408087fb2b0ad6d1c 2024-06-26T07:29:18.4089841Z deleted: sha256:8c387982e9f27db357a1d149dc149cabc0bd04668ad6956387a5072682dada67 2024-06-26T07:29:18.4090755Z deleted: sha256:10d415dca4e3fbb88a747b40317f3c6333c041ba8f73855f67fb9fe981fae5ee 2024-06-26T07:29:18.4091570Z deleted: sha256:6c65469bf0f53dcd625faefe363b8f3136c08388a6cf64d9158e8576e4ee89b1 2024-06-26T07:29:18.4092628Z deleted: sha256:027db5d8b76deeefbd8c34b485ab7ff14ed2a3e6800425c1ce0d77c2e9fdd4ab 2024-06-26T07:29:18.4093456Z deleted: sha256:24941234ad28cf5c8174d7cf18326c10b4e17f2c2137672ea0322df461597dd7 2024-06-26T07:29:18.4094280Z deleted: sha256:240ea2eebfa8519d6c2d88f4c99e66ce4e038d000f93ce8d03726c01d81180ef 2024-06-26T07:29:18.4095093Z deleted: sha256:09e823cb391a69f6a195e3ca8a41624b5f8b34eee14c6eb36277de814a91966e 2024-06-26T07:29:18.4095892Z deleted: sha256:730479d695169a6593bf0ac16b377491dce65175a6d158e012422367d04577fd 2024-06-26T07:29:18.4096698Z deleted: sha256:b0a30f9f7bd5459ff662453b9346a7ba8ea8db1191528345c1953eab4aa2f54e 2024-06-26T07:29:18.4097521Z deleted: sha256:75cefa87effaa3ad361c74642839923101f0f9982f814f3c9e2ed4ad52f2a008 2024-06-26T07:29:18.4098328Z deleted: sha256:6f97db87ba9512e615e871c6982000602982e1e5a5be334a56e7292251e84e00 2024-06-26T07:29:18.4099136Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-06-26T07:29:18.4099956Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-06-26T07:29:18.4100755Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-06-26T07:29:18.4101562Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-06-26T07:29:18.4102388Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-06-26T07:29:18.4103203Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-06-26T07:29:18.4104036Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-06-26T07:29:18.4104863Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-06-26T07:29:18.4105800Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-06-26T07:29:18.4106632Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-06-26T07:29:18.4107451Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-06-26T07:29:18.4107939Z 2024-06-26T07:29:18.4108091Z Total reclaimed space: 36.29GB 2024-06-26T07:29:18.4169968Z Post job cleanup. 2024-06-26T07:29:18.4218102Z Post job cleanup. 2024-06-26T07:29:18.5032198Z [command]/usr/bin/git version 2024-06-26T07:29:18.5066767Z git version 2.40.1 2024-06-26T07:29:18.5097486Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/13544601-25c5-4ad3-8017-ffa6a6c80aec/.gitconfig' 2024-06-26T07:29:18.5108181Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/13544601-25c5-4ad3-8017-ffa6a6c80aec' before making global git config changes 2024-06-26T07:29:18.5109444Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T07:29:18.5112883Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:29:18.5138941Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T07:29:18.5161636Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T07:29:18.5380558Z Entering 'android/libs/fbjni' 2024-06-26T07:29:18.5415940Z Entering 'third_party/FP16' 2024-06-26T07:29:18.5453820Z Entering 'third_party/FXdiv' 2024-06-26T07:29:18.5488895Z Entering 'third_party/NNPACK' 2024-06-26T07:29:18.5525281Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:29:18.5560663Z Entering 'third_party/XNNPACK' 2024-06-26T07:29:18.5606991Z Entering 'third_party/benchmark' 2024-06-26T07:29:18.5642380Z Entering 'third_party/cpp-httplib' 2024-06-26T07:29:18.5677901Z Entering 'third_party/cpuinfo' 2024-06-26T07:29:18.5714194Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:29:18.5749598Z Entering 'third_party/cutlass' 2024-06-26T07:29:18.5790357Z Entering 'third_party/eigen' 2024-06-26T07:29:18.5824306Z Entering 'third_party/fbgemm' 2024-06-26T07:29:18.5861469Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:29:18.5894064Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:29:18.5928475Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:29:18.5968007Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:29:18.6002370Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:29:18.6038918Z Entering 'third_party/flatbuffers' 2024-06-26T07:29:18.6076441Z Entering 'third_party/fmt' 2024-06-26T07:29:18.6111103Z Entering 'third_party/foxi' 2024-06-26T07:29:18.6146370Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:29:18.6179348Z Entering 'third_party/gloo' 2024-06-26T07:29:18.6214482Z Entering 'third_party/googletest' 2024-06-26T07:29:18.6248257Z Entering 'third_party/ideep' 2024-06-26T07:29:18.6282769Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:29:18.6322787Z Entering 'third_party/ittapi' 2024-06-26T07:29:18.6356115Z Entering 'third_party/kineto' 2024-06-26T07:29:18.6389931Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:29:18.6422056Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:29:18.6457055Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:29:18.6490376Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:29:18.6524994Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:29:18.6559198Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:29:18.6593856Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:29:18.6629221Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:29:18.6663538Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:29:18.6698681Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:29:18.6734559Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:29:18.6768665Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:29:18.6804405Z Entering 'third_party/mimalloc' 2024-06-26T07:29:18.6841507Z Entering 'third_party/nccl/nccl' 2024-06-26T07:29:18.6877255Z Entering 'third_party/nlohmann' 2024-06-26T07:29:18.6912091Z Entering 'third_party/onnx' 2024-06-26T07:29:18.6957557Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:29:18.6992869Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:29:18.7032234Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:29:18.7069727Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:29:18.7101882Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:29:18.7135208Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:29:18.7167199Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:29:18.7202600Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:29:18.7237489Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:29:18.7269453Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:29:18.7300062Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:29:18.7335475Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:29:18.7369913Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:29:18.7421450Z Entering 'third_party/pocketfft' 2024-06-26T07:29:18.7455209Z Entering 'third_party/protobuf' 2024-06-26T07:29:18.7492864Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:29:18.7526735Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:29:18.7560940Z Entering 'third_party/psimd' 2024-06-26T07:29:18.7597626Z Entering 'third_party/pthreadpool' 2024-06-26T07:29:18.7631360Z Entering 'third_party/pybind11' 2024-06-26T07:29:18.7667690Z Entering 'third_party/python-peachpy' 2024-06-26T07:29:18.7701013Z Entering 'third_party/sleef' 2024-06-26T07:29:18.7736663Z Entering 'third_party/tensorpipe' 2024-06-26T07:29:18.7768985Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:29:18.7803470Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:29:18.7836823Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:29:18.7870712Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:29:18.7902229Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:29:18.7951246Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T07:29:18.7968780Z http.https://github.com/.extraheader 2024-06-26T07:29:18.7978363Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-06-26T07:29:18.8007016Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T07:29:18.8217030Z Entering 'android/libs/fbjni' 2024-06-26T07:29:18.8238386Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8262916Z Entering 'third_party/FP16' 2024-06-26T07:29:18.8299674Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8310847Z Entering 'third_party/FXdiv' 2024-06-26T07:29:18.8332461Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8355535Z Entering 'third_party/NNPACK' 2024-06-26T07:29:18.8377016Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8400709Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:29:18.8422638Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8445931Z Entering 'third_party/XNNPACK' 2024-06-26T07:29:18.8467966Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8503256Z Entering 'third_party/benchmark' 2024-06-26T07:29:18.8524654Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8549049Z Entering 'third_party/cpp-httplib' 2024-06-26T07:29:18.8571619Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8594015Z Entering 'third_party/cpuinfo' 2024-06-26T07:29:18.8615091Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8638944Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:29:18.8661395Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8684568Z Entering 'third_party/cutlass' 2024-06-26T07:29:18.8707995Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8737904Z Entering 'third_party/eigen' 2024-06-26T07:29:18.8759870Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8783175Z Entering 'third_party/fbgemm' 2024-06-26T07:29:18.8804559Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8828588Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:29:18.8849438Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8871729Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:29:18.8894556Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8918830Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:29:18.8938821Z http.https://github.com/.extraheader 2024-06-26T07:29:18.8966135Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:29:18.8988838Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9013548Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:29:18.9033958Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9057803Z Entering 'third_party/flatbuffers' 2024-06-26T07:29:18.9079560Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9105385Z Entering 'third_party/fmt' 2024-06-26T07:29:18.9127560Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9152067Z Entering 'third_party/foxi' 2024-06-26T07:29:18.9174907Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9197224Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:29:18.9219384Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9243072Z Entering 'third_party/gloo' 2024-06-26T07:29:18.9265599Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9288650Z Entering 'third_party/googletest' 2024-06-26T07:29:18.9310564Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9334783Z Entering 'third_party/ideep' 2024-06-26T07:29:18.9354321Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9377433Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:29:18.9399458Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9427045Z Entering 'third_party/ittapi' 2024-06-26T07:29:18.9448776Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9471808Z Entering 'third_party/kineto' 2024-06-26T07:29:18.9493874Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9518235Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:29:18.9539563Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9562306Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:29:18.9584459Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9608638Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:29:18.9630126Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9653979Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:29:18.9674730Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9698733Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:29:18.9719467Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9742667Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:29:18.9763235Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9789199Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:29:18.9809701Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9833421Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:29:18.9855038Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9877178Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:29:18.9898746Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9923011Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:29:18.9944417Z http.https://github.com/.extraheader 2024-06-26T07:29:18.9968314Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:29:18.9990130Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0013336Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:29:19.0034304Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0057314Z Entering 'third_party/mimalloc' 2024-06-26T07:29:19.0078359Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0101995Z Entering 'third_party/nccl/nccl' 2024-06-26T07:29:19.0123931Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0150162Z Entering 'third_party/nlohmann' 2024-06-26T07:29:19.0171986Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0195912Z Entering 'third_party/onnx' 2024-06-26T07:29:19.0217861Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0253935Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:29:19.0276331Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0298920Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:29:19.0320496Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0346389Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:29:19.0368744Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0399001Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:29:19.0420804Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0444041Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:29:19.0465201Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0487907Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:29:19.0510059Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0533623Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:29:19.0554011Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0576948Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:29:19.0596998Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0620915Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:29:19.0644018Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0669052Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:29:19.0689650Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0711675Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:29:19.0733613Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0757239Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:29:19.0778801Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0803063Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:29:19.0824777Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0863896Z Entering 'third_party/pocketfft' 2024-06-26T07:29:19.0886292Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0911112Z Entering 'third_party/protobuf' 2024-06-26T07:29:19.0933279Z http.https://github.com/.extraheader 2024-06-26T07:29:19.0958353Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:29:19.0979871Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1002172Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:29:19.1023951Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1048733Z Entering 'third_party/psimd' 2024-06-26T07:29:19.1071124Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1096124Z Entering 'third_party/pthreadpool' 2024-06-26T07:29:19.1116810Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1141243Z Entering 'third_party/pybind11' 2024-06-26T07:29:19.1162657Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1185710Z Entering 'third_party/python-peachpy' 2024-06-26T07:29:19.1206907Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1230037Z Entering 'third_party/sleef' 2024-06-26T07:29:19.1249378Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1271195Z Entering 'third_party/tensorpipe' 2024-06-26T07:29:19.1293515Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1316649Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:29:19.1337649Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1360419Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:29:19.1381713Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1405232Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:29:19.1425806Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1450488Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:29:19.1473002Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1496600Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:29:19.1517244Z http.https://github.com/.extraheader 2024-06-26T07:29:19.1616736Z A job completed hook has been configured by the self-hosted runner administrator 2024-06-26T07:29:19.1633810Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T07:29:19.1640410Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:29:19.1640934Z ##[endgroup] 2024-06-26T07:29:20.0853313Z Cleaning up orphan processes