2024-06-26T06:59:35.8896579Z Current runner version: '2.317.0' 2024-06-26T06:59:35.8902647Z Runner name: 'i-02d9c9aefd6777ce0' 2024-06-26T06:59:35.8903470Z Runner group name: 'Default' 2024-06-26T06:59:35.8904462Z Machine name: 'ip-10-0-32-58' 2024-06-26T06:59:35.8909274Z ##[group]GITHUB_TOKEN Permissions 2024-06-26T06:59:35.8911261Z Actions: read 2024-06-26T06:59:35.8911813Z Attestations: read 2024-06-26T06:59:35.8912410Z Checks: read 2024-06-26T06:59:35.8912923Z Contents: read 2024-06-26T06:59:35.8913441Z Deployments: read 2024-06-26T06:59:35.8914045Z Discussions: read 2024-06-26T06:59:35.8914591Z Issues: read 2024-06-26T06:59:35.8915084Z Metadata: read 2024-06-26T06:59:35.8915625Z Packages: read 2024-06-26T06:59:35.8916152Z Pages: read 2024-06-26T06:59:35.8916673Z PullRequests: read 2024-06-26T06:59:35.8917288Z RepositoryProjects: read 2024-06-26T06:59:35.8917928Z SecurityEvents: read 2024-06-26T06:59:35.8918501Z Statuses: read 2024-06-26T06:59:35.8919052Z ##[endgroup] 2024-06-26T06:59:35.8922212Z Secret source: Actions 2024-06-26T06:59:35.8923096Z Prepare workflow directory 2024-06-26T06:59:36.2527845Z Prepare all required actions 2024-06-26T06:59:36.2710740Z Getting action download info 2024-06-26T06:59:36.4886232Z Download action repository 'pytorch/test-infra@main' (SHA:43a2ce341cc31288e9a38b65ce600a7f43021bd5) 2024-06-26T06:59:36.7812323Z Download action repository 'pytorch/pytorch@main' (SHA:c422a9549d2b320b47a9391f6cbcf592b2876c88) 2024-06-26T06:59:39.5342433Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-06-26T06:59:39.6755629Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-06-26T06:59:39.9727864Z Getting action download info 2024-06-26T06:59:40.1321467Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-06-26T06:59:40.3174873Z Getting action download info 2024-06-26T06:59:40.4069982Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-06-26T06:59:40.5507668Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/129470 (b8c4c54d347aa776934c60784e35936878ef18dc) 2024-06-26T06:59:40.5509825Z ##[group] Inputs 2024-06-26T06:59:40.5510310Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T06:59:40.5519110Z test-matrix: {"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-06-26T06:59:40.5528740Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:59:40.5529942Z sync-tag: 2024-06-26T06:59:40.5530905Z timeout-minutes: 240 2024-06-26T06:59:40.5531253Z use-gha: 2024-06-26T06:59:40.5531543Z dashboard-tag: 2024-06-26T06:59:40.5531881Z s3-bucket: gha-artifacts 2024-06-26T06:59:40.5532432Z aws-role-to-assume: 2024-06-26T06:59:40.5532776Z ##[endgroup] 2024-06-26T06:59:40.5533763Z Complete job name: cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T06:59:40.6091696Z A job started hook has been configured by the self-hosted runner administrator 2024-06-26T06:59:40.6227830Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T06:59:40.6238136Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:59:40.6238679Z ##[endgroup] 2024-06-26T06:59:41.3260432Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-06-26T06:59:41.3261007Z with: 2024-06-26T06:59:41.3261615Z github-secret: *** 2024-06-26T06:59:41.3262584Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-06-26T06:59:41.3263601Z activate-with-label: false 2024-06-26T06:59:41.3263979Z label: with-ssh 2024-06-26T06:59:41.3264323Z remove-existing-keys: true 2024-06-26T06:59:41.3264733Z fail-silently: true 2024-06-26T06:59:41.3265049Z env: 2024-06-26T06:59:41.3265343Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:59:41.3265703Z ##[endgroup] 2024-06-26T06:59:41.4126894Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-06-26T06:59:41.4127910Z ciflow reference detected, attempting to extract PR number 2024-06-26T06:59:41.6679295Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-06-26T06:59:41.7340951Z No SSH keys found for user pytorch-bot[bot] 2024-06-26T06:59:41.7341697Z Grabbing public ssh keys from https://github.com/leslie-fang-intel.keys 2024-06-26T06:59:41.7956392Z No SSH keys found for user leslie-fang-intel 2024-06-26T06:59:41.8062149Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-06-26T06:59:41.8062778Z with: 2024-06-26T06:59:41.8063109Z submodules: recursive 2024-06-26T06:59:41.8063542Z fetch-depth: 0 2024-06-26T06:59:41.8063862Z env: 2024-06-26T06:59:41.8064180Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:59:41.8064590Z ##[endgroup] 2024-06-26T06:59:41.8257696Z ##[group]Run retry () { 2024-06-26T06:59:41.8258131Z retry () { 2024-06-26T06:59:41.8288922Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-06-26T06:59:41.8289558Z } 2024-06-26T06:59:41.8289882Z echo "${GITHUB_WORKSPACE}" 2024-06-26T06:59:41.8290314Z if [ -z "${NO_SUDO}" ]; then 2024-06-26T06:59:41.8290803Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T06:59:41.8291272Z else 2024-06-26T06:59:41.8291625Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T06:59:41.8292222Z fi 2024-06-26T06:59:41.8292633Z mkdir "${GITHUB_WORKSPACE}" 2024-06-26T06:59:41.8300915Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:59:41.8301433Z env: 2024-06-26T06:59:41.8301723Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:59:41.8302079Z NO_SUDO: 2024-06-26T06:59:41.8302356Z ##[endgroup] 2024-06-26T06:59:41.8326340Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T06:59:44.0978759Z ##[group]Run malfet/checkout@silent-checkout 2024-06-26T06:59:44.0979202Z with: 2024-06-26T06:59:44.0979522Z ref: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T06:59:44.0979965Z fetch-depth: 0 2024-06-26T06:59:44.0980285Z submodules: recursive 2024-06-26T06:59:44.0980634Z quiet-checkout: true 2024-06-26T06:59:44.0980983Z repository: pytorch/pytorch 2024-06-26T06:59:44.0981474Z token: *** 2024-06-26T06:59:44.0981767Z ssh-strict: true 2024-06-26T06:59:44.0982128Z persist-credentials: true 2024-06-26T06:59:44.0982505Z clean: true 2024-06-26T06:59:44.0982841Z sparse-checkout-cone-mode: true 2024-06-26T06:59:44.0983236Z lfs: false 2024-06-26T06:59:44.0983532Z set-safe-directory: true 2024-06-26T06:59:44.0983875Z env: 2024-06-26T06:59:44.0984151Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:59:44.0984499Z ##[endgroup] 2024-06-26T06:59:44.1916151Z Syncing repository: pytorch/pytorch 2024-06-26T06:59:44.1917811Z ##[group]Getting Git version info 2024-06-26T06:59:44.1918577Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T06:59:44.1919435Z [command]/usr/bin/git version 2024-06-26T06:59:44.1919812Z git version 2.40.1 2024-06-26T06:59:44.1921192Z ##[endgroup] 2024-06-26T06:59:44.1932810Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/b9f36e1f-a9c2-4773-b61a-13c72f999773/.gitconfig' 2024-06-26T06:59:44.1940524Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/b9f36e1f-a9c2-4773-b61a-13c72f999773' before making global git config changes 2024-06-26T06:59:44.1941813Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T06:59:44.1945799Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T06:59:44.1981760Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T06:59:44.1986393Z ##[group]Initializing the repository 2024-06-26T06:59:44.1989999Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T06:59:44.2009794Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-06-26T06:59:44.2010957Z hint: is subject to change. To configure the initial branch name to use in all 2024-06-26T06:59:44.2012422Z hint: of your new repositories, which will suppress this warning, call: 2024-06-26T06:59:44.2013639Z hint: 2024-06-26T06:59:44.2014325Z hint: git config --global init.defaultBranch 2024-06-26T06:59:44.2015040Z hint: 2024-06-26T06:59:44.2016052Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-06-26T06:59:44.2017387Z hint: 'development'. The just-created branch can be renamed via this command: 2024-06-26T06:59:44.2018283Z hint: 2024-06-26T06:59:44.2018760Z hint: git branch -m 2024-06-26T06:59:44.2019868Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-06-26T06:59:44.2022182Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-06-26T06:59:44.2041172Z ##[endgroup] 2024-06-26T06:59:44.2042060Z ##[group]Disabling automatic garbage collection 2024-06-26T06:59:44.2044760Z [command]/usr/bin/git config --local gc.auto 0 2024-06-26T06:59:44.2064447Z ##[endgroup] 2024-06-26T06:59:44.2065242Z ##[group]Setting up auth 2024-06-26T06:59:44.2070852Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T06:59:44.2092498Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T06:59:44.2285247Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T06:59:44.2308818Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T06:59:44.2502897Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T06:59:44.2546252Z ##[endgroup] 2024-06-26T06:59:44.2546808Z ##[group]Fetching the repository 2024-06-26T06:59:44.2552762Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-06-26T06:59:46.5661215Z remote: Enumerating objects: 987619 2024-06-26T06:59:46.5662302Z remote: Enumerating objects: 990209, done. 2024-06-26T06:59:46.5663375Z remote: Counting objects: 0% (1/2590) 2024-06-26T06:59:46.5663895Z remote: Counting objects: 1% (26/2590) 2024-06-26T06:59:46.5664520Z remote: Counting objects: 2% (52/2590) 2024-06-26T06:59:46.5666011Z remote: Counting objects: 3% (78/2590) 2024-06-26T06:59:46.5666524Z remote: Counting objects: 4% (104/2590) 2024-06-26T06:59:46.5667221Z remote: Counting objects: 5% (130/2590) 2024-06-26T06:59:46.5667740Z remote: Counting objects: 6% (156/2590) 2024-06-26T06:59:46.5668391Z remote: Counting objects: 7% (182/2590) 2024-06-26T06:59:46.5669584Z remote: Counting objects: 8% (208/2590) 2024-06-26T06:59:46.5671071Z remote: Counting objects: 9% (234/2590) 2024-06-26T06:59:46.5671597Z remote: Counting objects: 10% (259/2590) 2024-06-26T06:59:46.5672444Z remote: Counting objects: 11% (285/2590) 2024-06-26T06:59:46.5673407Z remote: Counting objects: 12% (311/2590) 2024-06-26T06:59:46.5673997Z remote: Counting objects: 13% (337/2590) 2024-06-26T06:59:46.5674517Z remote: Counting objects: 14% (363/2590) 2024-06-26T06:59:46.5675021Z remote: Counting objects: 15% (389/2590) 2024-06-26T06:59:46.5675808Z remote: Counting objects: 16% (415/2590) 2024-06-26T06:59:46.5676392Z remote: Counting objects: 17% (441/2590) 2024-06-26T06:59:46.5676921Z remote: Counting objects: 18% (467/2590) 2024-06-26T06:59:46.5677433Z remote: Counting objects: 19% (493/2590) 2024-06-26T06:59:46.5678248Z remote: Counting objects: 20% (518/2590) 2024-06-26T06:59:46.5679797Z remote: Counting objects: 21% (544/2590) 2024-06-26T06:59:46.5680381Z remote: Counting objects: 22% (570/2590) 2024-06-26T06:59:46.5680906Z remote: Counting objects: 23% (596/2590) 2024-06-26T06:59:46.5681415Z remote: Counting objects: 24% (622/2590) 2024-06-26T06:59:46.5681913Z remote: Counting objects: 25% (648/2590) 2024-06-26T06:59:46.5683513Z remote: Counting objects: 26% (674/2590) 2024-06-26T06:59:46.5684843Z remote: Counting objects: 27% (700/2590) 2024-06-26T06:59:46.5685589Z remote: Counting objects: 28% (726/2590) 2024-06-26T06:59:46.5687038Z remote: Counting objects: 29% (752/2590) 2024-06-26T06:59:46.5688432Z remote: Counting objects: 30% (777/2590) 2024-06-26T06:59:46.5689039Z remote: Counting objects: 31% (803/2590) 2024-06-26T06:59:46.5689565Z remote: Counting objects: 32% (829/2590) 2024-06-26T06:59:46.5690137Z remote: Counting objects: 33% (855/2590) 2024-06-26T06:59:46.5690660Z remote: Counting objects: 34% (881/2590) 2024-06-26T06:59:46.5691227Z remote: Counting objects: 35% (907/2590) 2024-06-26T06:59:46.5691744Z remote: Counting objects: 36% (933/2590) 2024-06-26T06:59:46.5692547Z remote: Counting objects: 37% (959/2590) 2024-06-26T06:59:46.5693049Z remote: Counting objects: 38% (985/2590) 2024-06-26T06:59:46.5693706Z remote: Counting objects: 39% (1011/2590) 2024-06-26T06:59:46.5694291Z remote: Counting objects: 40% (1036/2590) 2024-06-26T06:59:46.5694817Z remote: Counting objects: 41% (1062/2590) 2024-06-26T06:59:46.5695350Z remote: Counting objects: 42% (1088/2590) 2024-06-26T06:59:46.5695867Z remote: Counting objects: 43% (1114/2590) 2024-06-26T06:59:46.5696633Z remote: Counting objects: 44% (1140/2590) 2024-06-26T06:59:46.5697155Z remote: Counting objects: 45% (1166/2590) 2024-06-26T06:59:46.5697675Z remote: Counting objects: 46% (1192/2590) 2024-06-26T06:59:46.5698191Z remote: Counting objects: 47% (1218/2590) 2024-06-26T06:59:46.5698782Z remote: Counting objects: 48% (1244/2590) 2024-06-26T06:59:46.5699304Z remote: Counting objects: 49% (1270/2590) 2024-06-26T06:59:46.5699822Z remote: Counting objects: 50% (1295/2590) 2024-06-26T06:59:46.5700336Z remote: Counting objects: 51% (1321/2590) 2024-06-26T06:59:46.5700861Z remote: Counting objects: 52% (1347/2590) 2024-06-26T06:59:46.5701452Z remote: Counting objects: 53% (1373/2590) 2024-06-26T06:59:46.5701976Z remote: Counting objects: 54% (1399/2590) 2024-06-26T06:59:46.5702488Z remote: Counting objects: 55% (1425/2590) 2024-06-26T06:59:46.5703554Z remote: Counting objects: 56% (1451/2590) 2024-06-26T06:59:46.5704087Z remote: Counting objects: 57% (1477/2590) 2024-06-26T06:59:46.5704632Z remote: Counting objects: 58% (1503/2590) 2024-06-26T06:59:46.5705239Z remote: Counting objects: 59% (1529/2590) 2024-06-26T06:59:46.5705752Z remote: Counting objects: 60% (1554/2590) 2024-06-26T06:59:46.5706419Z remote: Counting objects: 61% (1580/2590) 2024-06-26T06:59:46.5707028Z remote: Counting objects: 62% (1606/2590) 2024-06-26T06:59:46.5707556Z remote: Counting objects: 63% (1632/2590) 2024-06-26T06:59:46.5708079Z remote: Counting objects: 64% (1658/2590) 2024-06-26T06:59:46.5708586Z remote: Counting objects: 65% (1684/2590) 2024-06-26T06:59:46.5709137Z remote: Counting objects: 66% (1710/2590) 2024-06-26T06:59:46.5709697Z remote: Counting objects: 67% (1736/2590) 2024-06-26T06:59:46.5710222Z remote: Counting objects: 68% (1762/2590) 2024-06-26T06:59:46.5710742Z remote: Counting objects: 69% (1788/2590) 2024-06-26T06:59:46.5711248Z remote: Counting objects: 70% (1813/2590) 2024-06-26T06:59:46.5711762Z remote: Counting objects: 71% (1839/2590) 2024-06-26T06:59:46.5712316Z remote: Counting objects: 72% (1865/2590) 2024-06-26T06:59:46.5712900Z remote: Counting objects: 73% (1891/2590) 2024-06-26T06:59:46.5713418Z remote: Counting objects: 74% (1917/2590) 2024-06-26T06:59:46.5713939Z remote: Counting objects: 75% (1943/2590) 2024-06-26T06:59:46.5714452Z remote: Counting objects: 76% (1969/2590) 2024-06-26T06:59:46.5715060Z remote: Counting objects: 77% (1995/2590) 2024-06-26T06:59:46.5715607Z remote: Counting objects: 78% (2021/2590) 2024-06-26T06:59:46.5716170Z remote: Counting objects: 79% (2047/2590) 2024-06-26T06:59:46.5716711Z remote: Counting objects: 80% (2072/2590) 2024-06-26T06:59:46.5717253Z remote: Counting objects: 81% (2098/2590) 2024-06-26T06:59:46.5717795Z remote: Counting objects: 82% (2124/2590) 2024-06-26T06:59:46.5718338Z remote: Counting objects: 83% (2150/2590) 2024-06-26T06:59:46.5718878Z remote: Counting objects: 84% (2176/2590) 2024-06-26T06:59:46.5719418Z remote: Counting objects: 85% (2202/2590) 2024-06-26T06:59:46.5719957Z remote: Counting objects: 86% (2228/2590) 2024-06-26T06:59:46.5720503Z remote: Counting objects: 87% (2254/2590) 2024-06-26T06:59:46.5721045Z remote: Counting objects: 88% (2280/2590) 2024-06-26T06:59:46.5721589Z remote: Counting objects: 89% (2306/2590) 2024-06-26T06:59:46.5722130Z remote: Counting objects: 90% (2331/2590) 2024-06-26T06:59:46.5722670Z remote: Counting objects: 91% (2357/2590) 2024-06-26T06:59:46.5723212Z remote: Counting objects: 92% (2383/2590) 2024-06-26T06:59:46.5723810Z remote: Counting objects: 93% (2409/2590) 2024-06-26T06:59:46.5724344Z remote: Counting objects: 94% (2435/2590) 2024-06-26T06:59:46.5724949Z remote: Counting objects: 95% (2461/2590) 2024-06-26T06:59:46.5725487Z remote: Counting objects: 96% (2487/2590) 2024-06-26T06:59:46.5726027Z remote: Counting objects: 97% (2513/2590) 2024-06-26T06:59:46.5726563Z remote: Counting objects: 98% (2539/2590) 2024-06-26T06:59:46.5727107Z remote: Counting objects: 99% (2565/2590) 2024-06-26T06:59:46.5727648Z remote: Counting objects: 100% (2590/2590) 2024-06-26T06:59:46.5728226Z remote: Counting objects: 100% (2590/2590), done. 2024-06-26T06:59:46.5872382Z remote: Compressing objects: 0% (1/1068) 2024-06-26T06:59:46.6306386Z remote: Compressing objects: 1% (11/1068) 2024-06-26T06:59:46.6744237Z remote: Compressing objects: 2% (22/1068) 2024-06-26T06:59:46.7129809Z remote: Compressing objects: 3% (33/1068) 2024-06-26T06:59:46.7356194Z remote: Compressing objects: 4% (43/1068) 2024-06-26T06:59:46.7633228Z remote: Compressing objects: 5% (54/1068) 2024-06-26T06:59:46.7830348Z remote: Compressing objects: 6% (65/1068) 2024-06-26T06:59:46.8229667Z remote: Compressing objects: 7% (75/1068) 2024-06-26T06:59:46.9088158Z remote: Compressing objects: 8% (86/1068) 2024-06-26T06:59:46.9879775Z remote: Compressing objects: 9% (97/1068) 2024-06-26T06:59:47.0150382Z remote: Compressing objects: 10% (107/1068) 2024-06-26T06:59:47.0336227Z remote: Compressing objects: 11% (118/1068) 2024-06-26T06:59:47.0441196Z remote: Compressing objects: 12% (129/1068) 2024-06-26T06:59:47.0442451Z remote: Compressing objects: 13% (139/1068) 2024-06-26T06:59:47.0569055Z remote: Compressing objects: 14% (150/1068) 2024-06-26T06:59:47.0569697Z remote: Compressing objects: 15% (161/1068) 2024-06-26T06:59:47.0636059Z remote: Compressing objects: 16% (171/1068) 2024-06-26T06:59:47.0636684Z remote: Compressing objects: 17% (182/1068) 2024-06-26T06:59:47.0637232Z remote: Compressing objects: 18% (193/1068) 2024-06-26T06:59:47.0651859Z remote: Compressing objects: 19% (203/1068) 2024-06-26T06:59:47.0672714Z remote: Compressing objects: 20% (214/1068) 2024-06-26T06:59:47.0686140Z remote: Compressing objects: 21% (225/1068) 2024-06-26T06:59:47.0704822Z remote: Compressing objects: 22% (235/1068) 2024-06-26T06:59:47.0729534Z remote: Compressing objects: 23% (246/1068) 2024-06-26T06:59:47.0740140Z remote: Compressing objects: 24% (257/1068) 2024-06-26T06:59:47.0749041Z remote: Compressing objects: 25% (267/1068) 2024-06-26T06:59:47.0758123Z remote: Compressing objects: 26% (278/1068) 2024-06-26T06:59:47.0762752Z remote: Compressing objects: 27% (289/1068) 2024-06-26T06:59:47.0767556Z remote: Compressing objects: 28% (300/1068) 2024-06-26T06:59:47.0776464Z remote: Compressing objects: 29% (310/1068) 2024-06-26T06:59:47.0783143Z remote: Compressing objects: 30% (321/1068) 2024-06-26T06:59:47.0788356Z remote: Compressing objects: 31% (332/1068) 2024-06-26T06:59:47.0794913Z remote: Compressing objects: 32% (342/1068) 2024-06-26T06:59:47.0804399Z remote: Compressing objects: 33% (353/1068) 2024-06-26T06:59:47.0818040Z remote: Compressing objects: 34% (364/1068) 2024-06-26T06:59:47.0833777Z remote: Compressing objects: 35% (374/1068) 2024-06-26T06:59:47.0851510Z remote: Compressing objects: 36% (385/1068) 2024-06-26T06:59:47.0868970Z remote: Compressing objects: 37% (396/1068) 2024-06-26T06:59:47.0884302Z remote: Compressing objects: 38% (406/1068) 2024-06-26T06:59:47.0899976Z remote: Compressing objects: 39% (417/1068) 2024-06-26T06:59:47.0915392Z remote: Compressing objects: 40% (428/1068) 2024-06-26T06:59:47.0938001Z remote: Compressing objects: 41% (438/1068) 2024-06-26T06:59:47.0954947Z remote: Compressing objects: 42% (449/1068) 2024-06-26T06:59:47.0971345Z remote: Compressing objects: 43% (460/1068) 2024-06-26T06:59:47.0977685Z remote: Compressing objects: 44% (470/1068) 2024-06-26T06:59:47.0978229Z remote: Compressing objects: 45% (481/1068) 2024-06-26T06:59:47.0986396Z remote: Compressing objects: 46% (492/1068) 2024-06-26T06:59:47.1001988Z remote: Compressing objects: 47% (502/1068) 2024-06-26T06:59:47.1014059Z remote: Compressing objects: 48% (513/1068) 2024-06-26T06:59:47.1018501Z remote: Compressing objects: 49% (524/1068) 2024-06-26T06:59:47.1025515Z remote: Compressing objects: 50% (534/1068) 2024-06-26T06:59:47.1034847Z remote: Compressing objects: 51% (545/1068) 2024-06-26T06:59:47.1047416Z remote: Compressing objects: 52% (556/1068) 2024-06-26T06:59:47.1067201Z remote: Compressing objects: 53% (567/1068) 2024-06-26T06:59:47.1086117Z remote: Compressing objects: 54% (577/1068) 2024-06-26T06:59:47.1098457Z remote: Compressing objects: 55% (588/1068) 2024-06-26T06:59:47.1110610Z remote: Compressing objects: 56% (599/1068) 2024-06-26T06:59:47.1127950Z remote: Compressing objects: 57% (609/1068) 2024-06-26T06:59:47.1143195Z remote: Compressing objects: 58% (620/1068) 2024-06-26T06:59:47.1149411Z remote: Compressing objects: 59% (631/1068) 2024-06-26T06:59:47.1161838Z remote: Compressing objects: 60% (641/1068) 2024-06-26T06:59:47.1174483Z remote: Compressing objects: 61% (652/1068) 2024-06-26T06:59:47.1182087Z remote: Compressing objects: 62% (663/1068) 2024-06-26T06:59:47.1192646Z remote: Compressing objects: 63% (673/1068) 2024-06-26T06:59:47.1200530Z remote: Compressing objects: 64% (684/1068) 2024-06-26T06:59:47.1208981Z remote: Compressing objects: 65% (695/1068) 2024-06-26T06:59:47.1210966Z remote: Compressing objects: 66% (705/1068) 2024-06-26T06:59:47.1222285Z remote: Compressing objects: 67% (716/1068) 2024-06-26T06:59:47.1225278Z remote: Compressing objects: 68% (727/1068) 2024-06-26T06:59:47.1228309Z remote: Compressing objects: 69% (737/1068) 2024-06-26T06:59:47.1232469Z remote: Compressing objects: 70% (748/1068) 2024-06-26T06:59:47.1236059Z remote: Compressing objects: 71% (759/1068) 2024-06-26T06:59:47.1238791Z remote: Compressing objects: 72% (769/1068) 2024-06-26T06:59:47.1240240Z remote: Compressing objects: 73% (780/1068) 2024-06-26T06:59:47.1242192Z remote: Compressing objects: 74% (791/1068) 2024-06-26T06:59:47.1244163Z remote: Compressing objects: 75% (801/1068) 2024-06-26T06:59:47.1244876Z remote: Compressing objects: 76% (812/1068) 2024-06-26T06:59:47.1245494Z remote: Compressing objects: 77% (823/1068) 2024-06-26T06:59:47.1246031Z remote: Compressing objects: 78% (834/1068) 2024-06-26T06:59:47.1248371Z remote: Compressing objects: 79% (844/1068) 2024-06-26T06:59:47.1249856Z remote: Compressing objects: 80% (855/1068) 2024-06-26T06:59:47.1252714Z remote: Compressing objects: 81% (866/1068) 2024-06-26T06:59:47.1255401Z remote: Compressing objects: 82% (876/1068) 2024-06-26T06:59:47.1256928Z remote: Compressing objects: 83% (887/1068) 2024-06-26T06:59:47.1258943Z remote: Compressing objects: 84% (898/1068) 2024-06-26T06:59:47.1261651Z remote: Compressing objects: 85% (908/1068) 2024-06-26T06:59:47.1264069Z remote: Compressing objects: 86% (919/1068) 2024-06-26T06:59:47.1266643Z remote: Compressing objects: 87% (930/1068) 2024-06-26T06:59:47.1268484Z remote: Compressing objects: 88% (940/1068) 2024-06-26T06:59:47.1270193Z remote: Compressing objects: 89% (951/1068) 2024-06-26T06:59:47.1271618Z remote: Compressing objects: 90% (962/1068) 2024-06-26T06:59:47.1272158Z remote: Compressing objects: 91% (972/1068) 2024-06-26T06:59:47.1273492Z remote: Compressing objects: 92% (983/1068) 2024-06-26T06:59:47.1275409Z remote: Compressing objects: 93% (994/1068) 2024-06-26T06:59:47.1276114Z remote: Compressing objects: 94% (1004/1068) 2024-06-26T06:59:47.1282053Z remote: Compressing objects: 95% (1015/1068) 2024-06-26T06:59:47.1283550Z remote: Compressing objects: 96% (1026/1068) 2024-06-26T06:59:47.1284204Z remote: Compressing objects: 97% (1036/1068) 2024-06-26T06:59:47.1284749Z remote: Compressing objects: 98% (1047/1068) 2024-06-26T06:59:47.1285301Z remote: Compressing objects: 99% (1058/1068) 2024-06-26T06:59:47.1285860Z remote: Compressing objects: 100% (1068/1068) 2024-06-26T06:59:47.1286454Z remote: Compressing objects: 100% (1068/1068), done. 2024-06-26T07:00:07.2926595Z remote: Total 990209 (delta 2055), reused 1861 (delta 1521), pack-reused 987619 2024-06-26T07:00:27.1328507Z [command]/usr/bin/git rev-parse --verify --quiet b8c4c54d347aa776934c60784e35936878ef18dc^{object} 2024-06-26T07:00:27.1347258Z b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:00:27.1350876Z ##[endgroup] 2024-06-26T07:00:27.1351540Z ##[group]Determining the checkout info 2024-06-26T07:00:27.1352520Z ##[endgroup] 2024-06-26T07:00:27.1353446Z ##[group]Checking out the ref 2024-06-26T07:00:27.1355487Z [command]/usr/bin/git checkout --quiet --force b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:00:28.3498772Z ##[endgroup] 2024-06-26T07:00:28.3499500Z ##[group]Setting up auth for fetching submodules 2024-06-26T07:00:28.3502794Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:00:28.3551268Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-06-26T07:00:28.3576856Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-06-26T07:00:28.3602310Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-06-26T07:00:28.3624084Z ##[endgroup] 2024-06-26T07:00:28.3624772Z ##[group]Fetching submodules 2024-06-26T07:00:28.3627564Z [command]/usr/bin/git submodule sync --recursive 2024-06-26T07:00:28.3837132Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-06-26T07:00:28.4041131Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-06-26T07:00:28.4042665Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-06-26T07:00:28.4044517Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-06-26T07:00:28.4046423Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-06-26T07:00:28.4049218Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-06-26T07:00:28.4051236Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-06-26T07:00:28.4054576Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-06-26T07:00:28.4057317Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-06-26T07:00:28.4059835Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-06-26T07:00:28.4062724Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-06-26T07:00:28.4065334Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-06-26T07:00:28.4068174Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-06-26T07:00:28.4071199Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-06-26T07:00:28.4074262Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-06-26T07:00:28.4077141Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-06-26T07:00:28.4080343Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-06-26T07:00:28.4083681Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:00:28.4086805Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-06-26T07:00:28.4090091Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-06-26T07:00:28.4093782Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-06-26T07:00:28.4097253Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-06-26T07:00:28.4100709Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-06-26T07:00:28.4104514Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-06-26T07:00:28.4108128Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-06-26T07:00:28.4111859Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-06-26T07:00:28.4115640Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-06-26T07:00:28.4119817Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-06-26T07:00:28.4123472Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-06-26T07:00:28.4127475Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-06-26T07:00:28.4131511Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-06-26T07:00:28.4136309Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-06-26T07:00:28.4140142Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-06-26T07:00:28.4144410Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-06-26T07:00:28.4148491Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-06-26T07:00:28.4152950Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-06-26T07:00:28.4173364Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-06-26T07:00:28.6631676Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-06-26T07:00:28.8164308Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-06-26T07:00:29.0171713Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-06-26T07:00:29.2329871Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-06-26T07:00:31.2609601Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-06-26T07:00:41.0553461Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-06-26T07:00:41.4226575Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-06-26T07:00:41.9267451Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-06-26T07:00:42.4745300Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-06-26T07:00:43.6213057Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-06-26T07:00:45.3617357Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-06-26T07:00:51.1858922Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-06-26T07:00:52.4649821Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-06-26T07:00:54.1589586Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-06-26T07:00:55.3654210Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-06-26T07:00:55.5214040Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-06-26T07:00:55.8974418Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-06-26T07:00:56.1954400Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-06-26T07:00:57.2159289Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-06-26T07:00:57.5285054Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-06-26T07:00:57.8092240Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-06-26T07:00:59.2464260Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-06-26T07:00:59.9505690Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-06-26T07:01:01.6396467Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-06-26T07:01:07.3619613Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-06-26T07:01:13.4719769Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-06-26T07:01:19.0230021Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-06-26T07:01:19.2553946Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-06-26T07:01:27.4602390Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-06-26T07:01:27.6196047Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-06-26T07:01:27.7968822Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-06-26T07:01:28.7720383Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-06-26T07:01:29.0463713Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-06-26T07:01:30.1031681Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-06-26T07:01:30.4898567Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-06-26T07:01:30.4985260Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-06-26T07:01:30.5046750Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-06-26T07:01:30.5228933Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-06-26T07:01:30.5531030Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-06-26T07:01:31.3335496Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-06-26T07:01:31.3509165Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-06-26T07:01:31.3852163Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-06-26T07:01:31.4702040Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-06-26T07:01:31.4971956Z Submodule path 'third_party/cudnn_frontend': checked out 'aa3abd4bc689d6412979c7f55f9cd132848c9c6a' 2024-06-26T07:01:31.9291777Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-06-26T07:01:32.1416122Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-06-26T07:01:32.2050067Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-06-26T07:01:32.2061088Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:01:32.2064251Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:01:32.2065832Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:01:32.2067445Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:01:32.2069204Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:01:32.2084658Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-06-26T07:01:33.2082706Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-06-26T07:01:33.7670990Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-06-26T07:01:35.5013855Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-06-26T07:01:36.5055492Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-06-26T07:01:36.8139796Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-06-26T07:01:36.8987921Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-06-26T07:01:37.2406220Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-06-26T07:01:37.2947892Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-06-26T07:01:37.3035670Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-06-26T07:01:37.3960153Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-06-26T07:01:37.4243709Z Submodule path 'third_party/fmt': checked out 'e69e5f977d458f2650bb346dadf2ad30c5320281' 2024-06-26T07:01:37.4306913Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-06-26T07:01:37.4613666Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-06-26T07:01:37.4810744Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-06-26T07:01:37.5172868Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:01:37.5267286Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-06-26T07:01:37.5277201Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-06-26T07:01:37.5293527Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-06-26T07:01:49.6118506Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-06-26T07:01:49.6250590Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-06-26T07:01:49.7023454Z Submodule path 'third_party/kineto': checked out '8681ff11e1fa54da39023076c5c43eddd87b7a8a' 2024-06-26T07:01:49.7034688Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:01:49.7036520Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:01:49.7038363Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:01:49.7057141Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-06-26T07:01:50.2308952Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-06-26T07:01:51.4585527Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-06-26T07:01:52.4970798Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-06-26T07:01:52.4980552Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:01:52.4982364Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:01:52.4984098Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:01:52.4986042Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:01:52.4988512Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:01:52.4990926Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:01:52.4993335Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:01:52.4995297Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:01:52.5013283Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-06-26T07:01:53.5498555Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-06-26T07:01:53.9385759Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-06-26T07:01:55.1234323Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-06-26T07:01:55.4047202Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-06-26T07:01:55.8552381Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-06-26T07:01:56.8535403Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-06-26T07:02:02.5475616Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-06-26T07:02:02.9017703Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-06-26T07:02:02.9155974Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-06-26T07:02:02.9447419Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-06-26T07:02:02.9545043Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-06-26T07:02:02.9554021Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:02:02.9576106Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-06-26T07:02:03.2501357Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-06-26T07:02:03.2638751Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-06-26T07:02:03.2977455Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-06-26T07:02:03.3782775Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-06-26T07:02:03.3902333Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-06-26T07:02:03.4187199Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out 'a33701196adfad74917046096bf5a2aa0ab0bb50' 2024-06-26T07:02:03.4679538Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-06-26T07:02:03.4986202Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-06-26T07:02:03.5182682Z Submodule path 'third_party/nccl/nccl': checked out '48bb7fec7953112ff37499a272317f6663f8f600' 2024-06-26T07:02:03.6040713Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-06-26T07:02:03.8755084Z Submodule path 'third_party/onnx': checked out '990217f043af7222348ca8f0301e17fa7b841781' 2024-06-26T07:02:03.8781695Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-06-26T07:02:03.8783308Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-06-26T07:02:03.8799024Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-06-26T07:02:04.2512671Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-06-26T07:02:05.3884759Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-06-26T07:02:05.4158104Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-06-26T07:02:05.4688518Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-06-26T07:02:05.4700509Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:02:05.4707433Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:02:05.4710001Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:02:05.4712000Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:02:05.4714251Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:02:05.4716415Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:02:05.4718391Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:02:05.4720128Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:02:05.4729458Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-06-26T07:02:05.8582217Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-06-26T07:02:06.9999395Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-06-26T07:02:07.3403812Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-06-26T07:02:12.9740889Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-06-26T07:02:13.2280844Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-06-26T07:02:13.4113193Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-06-26T07:02:13.7291733Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-06-26T07:02:21.1216722Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-06-26T07:02:21.1545802Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-06-26T07:02:21.1668469Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-06-26T07:02:21.2517398Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-06-26T07:02:21.2607389Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-06-26T07:02:21.2713445Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-06-26T07:02:21.2825927Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-06-26T07:02:21.2835997Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:02:21.2838030Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:02:21.2853409Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-06-26T07:02:23.0284014Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-06-26T07:02:24.1623885Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-06-26T07:02:24.1989359Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:02:24.5878207Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-06-26T07:02:24.5956663Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-06-26T07:02:24.8146445Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-06-26T07:02:24.8159354Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:02:24.8161367Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-06-26T07:02:24.8177213Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-06-26T07:02:25.2817116Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-06-26T07:02:26.2270321Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-06-26T07:02:26.2893811Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-06-26T07:02:26.2954177Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-06-26T07:02:26.3041154Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-06-26T07:02:26.3321928Z Submodule path 'third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b' 2024-06-26T07:02:26.3546250Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-06-26T07:02:26.3875490Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-06-26T07:02:26.4077692Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-06-26T07:02:26.4088539Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:02:26.4090376Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:02:26.4093037Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:02:26.4095619Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:02:26.4113251Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-06-26T07:02:27.4114946Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-06-26T07:02:27.6149667Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-06-26T07:02:28.6396606Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-06-26T07:02:29.7031356Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-06-26T07:02:29.7143208Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-06-26T07:02:29.7711311Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-06-26T07:02:29.7938765Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-06-26T07:02:29.7949657Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:02:29.7965947Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-06-26T07:02:29.9737541Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-06-26T07:02:29.9761810Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-06-26T07:02:29.9954967Z Entering 'android/libs/fbjni' 2024-06-26T07:02:29.9981933Z Entering 'third_party/FP16' 2024-06-26T07:02:30.0008939Z Entering 'third_party/FXdiv' 2024-06-26T07:02:30.0036122Z Entering 'third_party/NNPACK' 2024-06-26T07:02:30.0064539Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:02:30.0089570Z Entering 'third_party/XNNPACK' 2024-06-26T07:02:30.0126961Z Entering 'third_party/benchmark' 2024-06-26T07:02:30.0153672Z Entering 'third_party/cpp-httplib' 2024-06-26T07:02:30.0178327Z Entering 'third_party/cpuinfo' 2024-06-26T07:02:30.0207026Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:02:30.0232182Z Entering 'third_party/cutlass' 2024-06-26T07:02:30.0263554Z Entering 'third_party/eigen' 2024-06-26T07:02:30.0293996Z Entering 'third_party/fbgemm' 2024-06-26T07:02:30.0318846Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:02:30.0344915Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:02:30.0370732Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:02:30.0404839Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:02:30.0429832Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:02:30.0460607Z Entering 'third_party/flatbuffers' 2024-06-26T07:02:30.0491659Z Entering 'third_party/fmt' 2024-06-26T07:02:30.0521340Z Entering 'third_party/foxi' 2024-06-26T07:02:30.0548775Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:02:30.0576449Z Entering 'third_party/gloo' 2024-06-26T07:02:30.0604798Z Entering 'third_party/googletest' 2024-06-26T07:02:30.0632143Z Entering 'third_party/ideep' 2024-06-26T07:02:30.0658571Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:02:30.0692385Z Entering 'third_party/ittapi' 2024-06-26T07:02:30.0722029Z Entering 'third_party/kineto' 2024-06-26T07:02:30.0748540Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:02:30.0777544Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:02:30.0805217Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:02:30.0831903Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:02:30.0858623Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:02:30.0883750Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:02:30.0912216Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:02:30.0938623Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:02:30.0965518Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:02:30.0992397Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:02:30.1021310Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:02:30.1048290Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:02:30.1074807Z Entering 'third_party/mimalloc' 2024-06-26T07:02:30.1100988Z Entering 'third_party/nccl/nccl' 2024-06-26T07:02:30.1129582Z Entering 'third_party/nlohmann' 2024-06-26T07:02:30.1156294Z Entering 'third_party/onnx' 2024-06-26T07:02:30.1194406Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:02:30.1222920Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:02:30.1251308Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:02:30.1281797Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:02:30.1307978Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:02:30.1336003Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:02:30.1361778Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:02:30.1390951Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:02:30.1418436Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:02:30.1445184Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:02:30.1471806Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:02:30.1499032Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:02:30.1527969Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:02:30.1570890Z Entering 'third_party/pocketfft' 2024-06-26T07:02:30.1599893Z Entering 'third_party/protobuf' 2024-06-26T07:02:30.1630865Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:02:30.1657778Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:02:30.1685332Z Entering 'third_party/psimd' 2024-06-26T07:02:30.1713194Z Entering 'third_party/pthreadpool' 2024-06-26T07:02:30.1738128Z Entering 'third_party/pybind11' 2024-06-26T07:02:30.1768692Z Entering 'third_party/python-peachpy' 2024-06-26T07:02:30.1794620Z Entering 'third_party/sleef' 2024-06-26T07:02:30.1819365Z Entering 'third_party/tensorpipe' 2024-06-26T07:02:30.1844175Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:02:30.1869339Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:02:30.1893922Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:02:30.1921234Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:02:30.1946795Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:02:30.1994652Z ##[endgroup] 2024-06-26T07:02:30.1996792Z ##[group]Persisting credentials for submodules 2024-06-26T07:02:30.1999537Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-06-26T07:02:30.2210431Z Entering 'android/libs/fbjni' 2024-06-26T07:02:30.2246850Z Entering 'third_party/FP16' 2024-06-26T07:02:30.2282537Z Entering 'third_party/FXdiv' 2024-06-26T07:02:30.2319272Z Entering 'third_party/NNPACK' 2024-06-26T07:02:30.2353373Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:02:30.2388988Z Entering 'third_party/XNNPACK' 2024-06-26T07:02:30.2433163Z Entering 'third_party/benchmark' 2024-06-26T07:02:30.2467346Z Entering 'third_party/cpp-httplib' 2024-06-26T07:02:30.2503634Z Entering 'third_party/cpuinfo' 2024-06-26T07:02:30.2539387Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:02:30.2574643Z Entering 'third_party/cutlass' 2024-06-26T07:02:30.2613859Z Entering 'third_party/eigen' 2024-06-26T07:02:30.2646574Z Entering 'third_party/fbgemm' 2024-06-26T07:02:30.2682064Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:02:30.2718147Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:02:30.2753862Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:02:30.2793539Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:02:30.2829416Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:02:30.2863978Z Entering 'third_party/flatbuffers' 2024-06-26T07:02:30.2900041Z Entering 'third_party/fmt' 2024-06-26T07:02:30.2933977Z Entering 'third_party/foxi' 2024-06-26T07:02:30.2967009Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:02:30.3001115Z Entering 'third_party/gloo' 2024-06-26T07:02:30.3035872Z Entering 'third_party/googletest' 2024-06-26T07:02:30.3068921Z Entering 'third_party/ideep' 2024-06-26T07:02:30.3103339Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:02:30.3144264Z Entering 'third_party/ittapi' 2024-06-26T07:02:30.3178564Z Entering 'third_party/kineto' 2024-06-26T07:02:30.3212449Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:02:30.3247018Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:02:30.3282201Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:02:30.3317715Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:02:30.3350437Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:02:30.3387676Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:02:30.3422167Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:02:30.3456164Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:02:30.3489709Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:02:30.3524556Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:02:30.3558504Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:02:30.3591759Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:02:30.3625457Z Entering 'third_party/mimalloc' 2024-06-26T07:02:30.3660973Z Entering 'third_party/nccl/nccl' 2024-06-26T07:02:30.3696110Z Entering 'third_party/nlohmann' 2024-06-26T07:02:30.3727546Z Entering 'third_party/onnx' 2024-06-26T07:02:30.3774229Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:02:30.3806896Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:02:30.3843840Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:02:30.3881330Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:02:30.3916414Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:02:30.3949836Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:02:30.3983399Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:02:30.4017603Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:02:30.4049617Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:02:30.4083469Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:02:30.4115649Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:02:30.4149090Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:02:30.4184394Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:02:30.4230924Z Entering 'third_party/pocketfft' 2024-06-26T07:02:30.4266910Z Entering 'third_party/protobuf' 2024-06-26T07:02:30.4303409Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:02:30.4337707Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:02:30.4372781Z Entering 'third_party/psimd' 2024-06-26T07:02:30.4407750Z Entering 'third_party/pthreadpool' 2024-06-26T07:02:30.4439587Z Entering 'third_party/pybind11' 2024-06-26T07:02:30.4472475Z Entering 'third_party/python-peachpy' 2024-06-26T07:02:30.4507948Z Entering 'third_party/sleef' 2024-06-26T07:02:30.4540994Z Entering 'third_party/tensorpipe' 2024-06-26T07:02:30.4576676Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:02:30.4610222Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:02:30.4645034Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:02:30.4679522Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:02:30.4712586Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:02:30.4761318Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-06-26T07:02:30.4966150Z Entering 'android/libs/fbjni' 2024-06-26T07:02:30.4997689Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-06-26T07:02:30.5010413Z Entering 'third_party/FP16' 2024-06-26T07:02:30.5043475Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-06-26T07:02:30.5057204Z Entering 'third_party/FXdiv' 2024-06-26T07:02:30.5088796Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-06-26T07:02:30.5102492Z Entering 'third_party/NNPACK' 2024-06-26T07:02:30.5134181Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-06-26T07:02:30.5147916Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:02:30.5179406Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-06-26T07:02:30.5192211Z Entering 'third_party/XNNPACK' 2024-06-26T07:02:30.5224360Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-06-26T07:02:30.5248627Z Entering 'third_party/benchmark' 2024-06-26T07:02:30.5281085Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:02:30.5295005Z Entering 'third_party/cpp-httplib' 2024-06-26T07:02:30.5327141Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-06-26T07:02:30.5339553Z Entering 'third_party/cpuinfo' 2024-06-26T07:02:30.5371798Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:02:30.5386302Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:02:30.5417696Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-06-26T07:02:30.5429795Z Entering 'third_party/cutlass' 2024-06-26T07:02:30.5461940Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:02:30.5480277Z Entering 'third_party/eigen' 2024-06-26T07:02:30.5512271Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-06-26T07:02:30.5526553Z Entering 'third_party/fbgemm' 2024-06-26T07:02:30.5559007Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-06-26T07:02:30.5573339Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:02:30.5604779Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-06-26T07:02:30.5617678Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:02:30.5649644Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:02:30.5662955Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:02:30.5694393Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:02:30.5710071Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:02:30.5740680Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:02:30.5753494Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:02:30.5784651Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-06-26T07:02:30.5798690Z Entering 'third_party/flatbuffers' 2024-06-26T07:02:30.5829757Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-06-26T07:02:30.5845801Z Entering 'third_party/fmt' 2024-06-26T07:02:30.5876317Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:02:30.5887109Z Entering 'third_party/foxi' 2024-06-26T07:02:30.5919244Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-06-26T07:02:30.5933099Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:02:30.5964851Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-06-26T07:02:30.5979030Z Entering 'third_party/gloo' 2024-06-26T07:02:30.6010244Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-06-26T07:02:30.6023069Z Entering 'third_party/googletest' 2024-06-26T07:02:30.6052484Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:02:30.6068901Z Entering 'third_party/ideep' 2024-06-26T07:02:30.6100357Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-06-26T07:02:30.6112868Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:02:30.6142979Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-06-26T07:02:30.6162070Z Entering 'third_party/ittapi' 2024-06-26T07:02:30.6193960Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-06-26T07:02:30.6206721Z Entering 'third_party/kineto' 2024-06-26T07:02:30.6239871Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-06-26T07:02:30.6253109Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:02:30.6283645Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-06-26T07:02:30.6295733Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:02:30.6326571Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-06-26T07:02:30.6340360Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:02:30.6372329Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-06-26T07:02:30.6385170Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:02:30.6418504Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:02:30.6430539Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:02:30.6462412Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-06-26T07:02:30.6474429Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:02:30.6505883Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-06-26T07:02:30.6520817Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:02:30.6552004Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-06-26T07:02:30.6564536Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:02:30.6597361Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:02:30.6608845Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:02:30.6641723Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-06-26T07:02:30.6655251Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:02:30.6686179Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-06-26T07:02:30.6700323Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:02:30.6730901Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-06-26T07:02:30.6743635Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:02:30.6776075Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-06-26T07:02:30.6789893Z Entering 'third_party/mimalloc' 2024-06-26T07:02:30.6821435Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-06-26T07:02:30.6834293Z Entering 'third_party/nccl/nccl' 2024-06-26T07:02:30.6865523Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-06-26T07:02:30.6879671Z Entering 'third_party/nlohmann' 2024-06-26T07:02:30.6910225Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-06-26T07:02:30.6924760Z Entering 'third_party/onnx' 2024-06-26T07:02:30.6956007Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-06-26T07:02:30.6981989Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:02:30.7014373Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:02:30.7026990Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:02:30.7058348Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:02:30.7072950Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:02:30.7104813Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-06-26T07:02:30.7119243Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:02:30.7150952Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:02:30.7164265Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:02:30.7196231Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:02:30.7209168Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:02:30.7238991Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-06-26T07:02:30.7250866Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:02:30.7281662Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-06-26T07:02:30.7295235Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:02:30.7327008Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-06-26T07:02:30.7338656Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:02:30.7369192Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-06-26T07:02:30.7381223Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:02:30.7413101Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-06-26T07:02:30.7424300Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:02:30.7453788Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-06-26T07:02:30.7467193Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:02:30.7497216Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-06-26T07:02:30.7510704Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:02:30.7542694Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-06-26T07:02:30.7571058Z Entering 'third_party/pocketfft' 2024-06-26T07:02:30.7602191Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-06-26T07:02:30.7613475Z Entering 'third_party/protobuf' 2024-06-26T07:02:30.7644786Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-06-26T07:02:30.7660751Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:02:30.7690889Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:02:30.7704285Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:02:30.7735461Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:02:30.7748884Z Entering 'third_party/psimd' 2024-06-26T07:02:30.7781541Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-06-26T07:02:30.7794857Z Entering 'third_party/pthreadpool' 2024-06-26T07:02:30.7826080Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-06-26T07:02:30.7838179Z Entering 'third_party/pybind11' 2024-06-26T07:02:30.7869561Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:02:30.7883026Z Entering 'third_party/python-peachpy' 2024-06-26T07:02:30.7915888Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-06-26T07:02:30.7927487Z Entering 'third_party/sleef' 2024-06-26T07:02:30.7959400Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-06-26T07:02:30.7973045Z Entering 'third_party/tensorpipe' 2024-06-26T07:02:30.8004178Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-06-26T07:02:30.8016835Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:02:30.8047727Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:02:30.8062154Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:02:30.8091369Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-06-26T07:02:30.8104170Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:02:30.8135132Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-06-26T07:02:30.8148056Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:02:30.8179600Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:02:30.8191626Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:02:30.8222833Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-06-26T07:02:30.9982050Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-06-26T07:02:31.0181358Z Entering 'android/libs/fbjni' 2024-06-26T07:02:31.0209116Z Entering 'third_party/FP16' 2024-06-26T07:02:31.0234598Z Entering 'third_party/FXdiv' 2024-06-26T07:02:31.0260663Z Entering 'third_party/NNPACK' 2024-06-26T07:02:31.0288860Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:02:31.0315026Z Entering 'third_party/XNNPACK' 2024-06-26T07:02:31.0352946Z Entering 'third_party/benchmark' 2024-06-26T07:02:31.0382479Z Entering 'third_party/cpp-httplib' 2024-06-26T07:02:31.0411903Z Entering 'third_party/cpuinfo' 2024-06-26T07:02:31.0439039Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:02:31.0465649Z Entering 'third_party/cutlass' 2024-06-26T07:02:31.0499967Z Entering 'third_party/eigen' 2024-06-26T07:02:31.0528823Z Entering 'third_party/fbgemm' 2024-06-26T07:02:31.0556853Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:02:31.0583578Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:02:31.0609449Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:02:31.0642947Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:02:31.0669767Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:02:31.0699374Z Entering 'third_party/flatbuffers' 2024-06-26T07:02:31.0731033Z Entering 'third_party/fmt' 2024-06-26T07:02:31.0759923Z Entering 'third_party/foxi' 2024-06-26T07:02:31.0786684Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:02:31.0814596Z Entering 'third_party/gloo' 2024-06-26T07:02:31.0843493Z Entering 'third_party/googletest' 2024-06-26T07:02:31.0873696Z Entering 'third_party/ideep' 2024-06-26T07:02:31.0899005Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:02:31.0931830Z Entering 'third_party/ittapi' 2024-06-26T07:02:31.0961030Z Entering 'third_party/kineto' 2024-06-26T07:02:31.0988501Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:02:31.1017684Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:02:31.1044943Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:02:31.1072344Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:02:31.1099035Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:02:31.1125023Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:02:31.1155074Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:02:31.1184793Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:02:31.1211565Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:02:31.1239072Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:02:31.1269821Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:02:31.1297919Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:02:31.1325843Z Entering 'third_party/mimalloc' 2024-06-26T07:02:31.1355699Z Entering 'third_party/nccl/nccl' 2024-06-26T07:02:31.1383106Z Entering 'third_party/nlohmann' 2024-06-26T07:02:31.1409402Z Entering 'third_party/onnx' 2024-06-26T07:02:31.1450352Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:02:31.1479688Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:02:31.1507234Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:02:31.1535726Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:02:31.1564324Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:02:31.1592786Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:02:31.1618501Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:02:31.1645797Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:02:31.1673442Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:02:31.1699777Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:02:31.1726464Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:02:31.1755723Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:02:31.1783715Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:02:31.1827476Z Entering 'third_party/pocketfft' 2024-06-26T07:02:31.1857715Z Entering 'third_party/protobuf' 2024-06-26T07:02:31.1888146Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:02:31.1917941Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:02:31.1948072Z Entering 'third_party/psimd' 2024-06-26T07:02:31.1977632Z Entering 'third_party/pthreadpool' 2024-06-26T07:02:31.2003725Z Entering 'third_party/pybind11' 2024-06-26T07:02:31.2031501Z Entering 'third_party/python-peachpy' 2024-06-26T07:02:31.2057726Z Entering 'third_party/sleef' 2024-06-26T07:02:31.2086783Z Entering 'third_party/tensorpipe' 2024-06-26T07:02:31.2115256Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:02:31.2142539Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:02:31.2168011Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:02:31.2194744Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:02:31.2220196Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:02:31.2261364Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-06-26T07:02:31.2468600Z Entering 'android/libs/fbjni' 2024-06-26T07:02:31.2496897Z Entering 'third_party/FP16' 2024-06-26T07:02:31.2526367Z Entering 'third_party/FXdiv' 2024-06-26T07:02:31.2556982Z Entering 'third_party/NNPACK' 2024-06-26T07:02:31.2584242Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:02:31.2611820Z Entering 'third_party/XNNPACK' 2024-06-26T07:02:31.2651074Z Entering 'third_party/benchmark' 2024-06-26T07:02:31.2680736Z Entering 'third_party/cpp-httplib' 2024-06-26T07:02:31.2707940Z Entering 'third_party/cpuinfo' 2024-06-26T07:02:31.2735300Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:02:31.2763574Z Entering 'third_party/cutlass' 2024-06-26T07:02:31.2798169Z Entering 'third_party/eigen' 2024-06-26T07:02:31.2828589Z Entering 'third_party/fbgemm' 2024-06-26T07:02:31.2858337Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:02:31.2885207Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:02:31.2911947Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:02:31.2945979Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:02:31.2973432Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:02:31.3001468Z Entering 'third_party/flatbuffers' 2024-06-26T07:02:31.3033309Z Entering 'third_party/fmt' 2024-06-26T07:02:31.3058353Z Entering 'third_party/foxi' 2024-06-26T07:02:31.3086159Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:02:31.3115134Z Entering 'third_party/gloo' 2024-06-26T07:02:31.3141608Z Entering 'third_party/googletest' 2024-06-26T07:02:31.3170674Z Entering 'third_party/ideep' 2024-06-26T07:02:31.3197246Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:02:31.3230746Z Entering 'third_party/ittapi' 2024-06-26T07:02:31.3260584Z Entering 'third_party/kineto' 2024-06-26T07:02:31.3289079Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:02:31.3316055Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:02:31.3342741Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:02:31.3370491Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:02:31.3399624Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:02:31.3424471Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:02:31.3454211Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:02:31.3481073Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:02:31.3509738Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:02:31.3540017Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:02:31.3569402Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:02:31.3598472Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:02:31.3625511Z Entering 'third_party/mimalloc' 2024-06-26T07:02:31.3655676Z Entering 'third_party/nccl/nccl' 2024-06-26T07:02:31.3682145Z Entering 'third_party/nlohmann' 2024-06-26T07:02:31.3710372Z Entering 'third_party/onnx' 2024-06-26T07:02:31.3748816Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:02:31.3777110Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:02:31.3807107Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:02:31.3837614Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:02:31.3864028Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:02:31.3890548Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:02:31.3918367Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:02:31.3946053Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:02:31.3971432Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:02:31.4000683Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:02:31.4027301Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:02:31.4056798Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:02:31.4084460Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:02:31.4127075Z Entering 'third_party/pocketfft' 2024-06-26T07:02:31.4156666Z Entering 'third_party/protobuf' 2024-06-26T07:02:31.4185794Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:02:31.4212306Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:02:31.4241767Z Entering 'third_party/psimd' 2024-06-26T07:02:31.4271108Z Entering 'third_party/pthreadpool' 2024-06-26T07:02:31.4297081Z Entering 'third_party/pybind11' 2024-06-26T07:02:31.4325164Z Entering 'third_party/python-peachpy' 2024-06-26T07:02:31.4350931Z Entering 'third_party/sleef' 2024-06-26T07:02:31.4378741Z Entering 'third_party/tensorpipe' 2024-06-26T07:02:31.4405927Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:02:31.4434671Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:02:31.4461698Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:02:31.4489258Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:02:31.4516871Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:02:31.4554542Z ##[endgroup] 2024-06-26T07:02:31.4584103Z [command]/usr/bin/git log -1 --format='%H' 2024-06-26T07:02:31.4604869Z 'b8c4c54d347aa776934c60784e35936878ef18dc' 2024-06-26T07:02:31.4779207Z Prepare all required actions 2024-06-26T07:02:31.4779666Z Getting action download info 2024-06-26T07:02:31.6393649Z ##[group]Run ./.github/actions/setup-linux 2024-06-26T07:02:31.6394141Z env: 2024-06-26T07:02:31.6394485Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:31.6394900Z ##[endgroup] 2024-06-26T07:02:31.6467485Z ##[group]Run set -euo pipefail 2024-06-26T07:02:31.6467985Z set -euo pipefail 2024-06-26T07:02:31.6468447Z function get_ec2_metadata() { 2024-06-26T07:02:31.6469052Z  # Pulled from instance metadata endpoint for EC2 2024-06-26T07:02:31.6470033Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-06-26T07:02:31.6470880Z  category=$1 2024-06-26T07:02:31.6471478Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-06-26T07:02:31.6472198Z  runner_name_str=i-02d9c9aefd6777ce0 2024-06-26T07:02:31.6472765Z  if [[ -f /.inarc ]]; then 2024-06-26T07:02:31.6473330Z  echo "ARC Runner, no info on ec2 metadata" 2024-06-26T07:02:31.6473969Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-06-26T07:02:31.6474733Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-06-26T07:02:31.6475412Z  else 2024-06-26T07:02:31.6475969Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-06-26T07:02:31.6476622Z  fi 2024-06-26T07:02:31.6476975Z } 2024-06-26T07:02:31.6477401Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-06-26T07:02:31.6478058Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-06-26T07:02:31.6478785Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-06-26T07:02:31.6479423Z echo "system info $(uname -a)" 2024-06-26T07:02:31.6487291Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:02:31.6487855Z env: 2024-06-26T07:02:31.6488214Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:31.6488635Z ##[endgroup] 2024-06-26T07:02:31.6558264Z ami-id: ami-0ce0c36d7a00b20e2 2024-06-26T07:02:31.6597683Z instance-id: i-02d9c9aefd6777ce0 2024-06-26T07:02:31.6641294Z instance-type: g5.4xlarge 2024-06-26T07:02:31.6647042Z system info Linux ip-10-0-32-58.ec2.internal 4.14.336-257.562.amzn2.x86_64 #1 SMP Sat Feb 24 09:50:35 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-06-26T07:02:31.6667536Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:02:31.6668470Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:02:31.6676636Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:02:31.6677144Z env: 2024-06-26T07:02:31.6677438Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:31.6677800Z ##[endgroup] 2024-06-26T07:02:31.6739164Z ##[group]Run if systemctl is-active --quiet docker; then 2024-06-26T07:02:31.6739785Z if systemctl is-active --quiet docker; then 2024-06-26T07:02:31.6740338Z  echo "Docker daemon is running..."; 2024-06-26T07:02:31.6740796Z else 2024-06-26T07:02:31.6741299Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-06-26T07:02:31.6741899Z fi 2024-06-26T07:02:31.6749412Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:02:31.6749917Z env: 2024-06-26T07:02:31.6750202Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:31.6750561Z ##[endgroup] 2024-06-26T07:02:31.6785200Z Docker daemon is running... 2024-06-26T07:02:31.6827829Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:02:31.6828397Z with: 2024-06-26T07:02:31.6828665Z shell: bash 2024-06-26T07:02:31.6828964Z timeout_minutes: 5 2024-06-26T07:02:31.6829290Z max_attempts: 3 2024-06-26T07:02:31.6829607Z retry_wait_seconds: 30 2024-06-26T07:02:31.6831142Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-06-26T07:02:31.6832609Z polling_interval_seconds: 1 2024-06-26T07:02:31.6832990Z warning_on_retry: true 2024-06-26T07:02:31.6833349Z continue_on_error: false 2024-06-26T07:02:31.6833690Z env: 2024-06-26T07:02:31.6833963Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:31.6834377Z AWS_RETRY_MODE: standard 2024-06-26T07:02:31.6834732Z AWS_MAX_ATTEMPTS: 5 2024-06-26T07:02:31.6835080Z AWS_DEFAULT_REGION: us-east-1 2024-06-26T07:02:31.6835473Z ##[endgroup] 2024-06-26T07:02:32.5426071Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:02:32.5427263Z Configure a credential helper to remove this warning. See 2024-06-26T07:02:32.5428146Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:02:32.5428697Z 2024-06-26T07:02:32.5428841Z Login Succeeded 2024-06-26T07:02:32.7330388Z Command completed after 1 attempt(s). 2024-06-26T07:02:32.7371649Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:02:32.7372734Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:02:32.7373403Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:02:32.7381312Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:02:32.7381819Z env: 2024-06-26T07:02:32.7382111Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:32.7382477Z ##[endgroup] 2024-06-26T07:02:32.7442883Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:02:32.7443671Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:02:32.7444270Z # shellcheck disable=SC2046 2024-06-26T07:02:32.7444737Z docker stop $(docker ps -q) || true 2024-06-26T07:02:32.7445234Z # Prune all of the docker images 2024-06-26T07:02:32.7445688Z docker system prune -af 2024-06-26T07:02:32.7453865Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:02:32.7454378Z env: 2024-06-26T07:02:32.7454652Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:32.7455002Z ##[endgroup] 2024-06-26T07:02:32.7698551Z "docker stop" requires at least 1 argument. 2024-06-26T07:02:32.7699288Z See 'docker stop --help'. 2024-06-26T07:02:32.7699532Z 2024-06-26T07:02:32.7699759Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-06-26T07:02:32.7700146Z 2024-06-26T07:02:32.7700296Z Stop one or more running containers 2024-06-26T07:02:32.7900626Z Total reclaimed space: 0B 2024-06-26T07:02:32.7930072Z ##[group]Run set +e 2024-06-26T07:02:32.7930428Z set +e 2024-06-26T07:02:32.7930726Z set -x 2024-06-26T07:02:32.7931017Z  2024-06-26T07:02:32.7931336Z PT_DOMAIN=download.pytorch.org 2024-06-26T07:02:32.7932508Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-06-26T07:02:32.7933629Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-06-26T07:02:32.7934488Z # one is returned at random 2024-06-26T07:02:32.7935035Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-06-26T07:02:32.7935546Z  2024-06-26T07:02:32.7935869Z if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:02:32.7936494Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-06-26T07:02:32.7937262Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-06-26T07:02:32.7937849Z  2024-06-26T07:02:32.7938175Z  if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:02:32.7938733Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-06-26T07:02:32.7939246Z  exit 1 2024-06-26T07:02:32.7939555Z  fi 2024-06-26T07:02:32.7939837Z fi 2024-06-26T07:02:32.7940266Z  2024-06-26T07:02:32.7940638Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-06-26T07:02:32.7941156Z  # Clean up any old records first 2024-06-26T07:02:32.7941668Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-06-26T07:02:32.7942120Z fi 2024-06-26T07:02:32.7942387Z  2024-06-26T07:02:32.7942818Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-06-26T07:02:32.7943365Z cat /etc/hosts 2024-06-26T07:02:32.7950985Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:02:32.7951491Z env: 2024-06-26T07:02:32.7951774Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:32.7952128Z ##[endgroup] 2024-06-26T07:02:32.7972892Z + PT_DOMAIN=download.pytorch.org 2024-06-26T07:02:32.7976223Z ++ dig -4 +short download.pytorch.org 2024-06-26T07:02:32.7976673Z ++ tail -n1 2024-06-26T07:02:32.8226047Z + RESOLVED_IP=18.160.10.36 2024-06-26T07:02:32.8226479Z + '[' -z 18.160.10.36 ']' 2024-06-26T07:02:32.8226940Z + grep -r download.pytorch.org /etc/hosts 2024-06-26T07:02:32.8232286Z 18.67.76.99 download.pytorch.org 2024-06-26T07:02:32.8232995Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-06-26T07:02:32.8327286Z + echo '18.160.10.36 download.pytorch.org' 2024-06-26T07:02:32.8327781Z + sudo tee -a /etc/hosts 2024-06-26T07:02:32.8569354Z 18.160.10.36 download.pytorch.org 2024-06-26T07:02:32.8582955Z + cat /etc/hosts 2024-06-26T07:02:32.8587693Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-06-26T07:02:32.8598529Z ::1 localhost6 localhost6.localdomain6 2024-06-26T07:02:32.8599058Z 18.160.10.36 download.pytorch.org 2024-06-26T07:02:32.8806215Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-06-26T07:02:32.8806823Z with: 2024-06-26T07:02:32.8807938Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:32.8809182Z docker-build-dir: .ci/docker 2024-06-26T07:02:32.8809560Z working-directory: . 2024-06-26T07:02:32.8810029Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:02:32.8810560Z force-push: false 2024-06-26T07:02:32.8810856Z env: 2024-06-26T07:02:32.8811126Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:32.8811468Z ##[endgroup] 2024-06-26T07:02:32.8828731Z ##[group]Run set -ex 2024-06-26T07:02:32.8829098Z set -ex 2024-06-26T07:02:32.8829397Z  2024-06-26T07:02:32.8829964Z # If the docker build directory or the build script doesn't exist, the action will 2024-06-26T07:02:32.8831091Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-06-26T07:02:32.8831908Z # job could then download the pre-built image as usual 2024-06-26T07:02:32.8832678Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-06-26T07:02:32.8833373Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:02:32.8834029Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:02:32.8834652Z  2024-06-26T07:02:32.8835181Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-06-26T07:02:32.8835821Z  exit 0 2024-06-26T07:02:32.8836137Z else 2024-06-26T07:02:32.8836520Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-06-26T07:02:32.8836980Z fi 2024-06-26T07:02:32.8837265Z  2024-06-26T07:02:32.8837762Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-06-26T07:02:32.8838645Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-06-26T07:02:32.8839451Z  # use it as it is, but first let's extract the tag 2024-06-26T07:02:32.8840183Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-06-26T07:02:32.8840936Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:02:32.8841660Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:02:32.8842228Z else 2024-06-26T07:02:32.8842675Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-06-26T07:02:32.8843362Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:02:32.8844319Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:02:32.8845091Z fi 2024-06-26T07:02:32.8853008Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:02:32.8853521Z env: 2024-06-26T07:02:32.8853886Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:32.8854257Z REPO_NAME: pytorch 2024-06-26T07:02:32.8855422Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:32.8856662Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:02:32.8857178Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:02:32.8857713Z ##[endgroup] 2024-06-26T07:02:32.8877457Z + [[ ! -d .ci/docker ]] 2024-06-26T07:02:32.8877879Z + [[ ! -f .ci/docker/build.sh ]] 2024-06-26T07:02:32.8878265Z + echo skip=false 2024-06-26T07:02:32.8880241Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-06-26T07:02:32.8883171Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:32.8884516Z ++ awk -F '[:,]' '{print $2}' 2024-06-26T07:02:32.8889979Z + DOCKER_TAG=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:32.8890633Z + echo docker-tag=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:32.8892572Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:32.8919759Z ##[group]Run set +e 2024-06-26T07:02:32.8920139Z set +e 2024-06-26T07:02:32.8920448Z set -x 2024-06-26T07:02:32.8920739Z  2024-06-26T07:02:32.8921031Z login() { 2024-06-26T07:02:32.8921720Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:02:32.8922591Z } 2024-06-26T07:02:32.8922873Z  2024-06-26T07:02:32.8923160Z retry () { 2024-06-26T07:02:32.8923563Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:02:32.8924022Z } 2024-06-26T07:02:32.8924303Z  2024-06-26T07:02:32.8924635Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:02:32.8925067Z  2024-06-26T07:02:32.8925548Z # Check if image already exists, if it does then skip building it 2024-06-26T07:02:32.8926285Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-06-26T07:02:32.8926802Z  exit 0 2024-06-26T07:02:32.8927118Z fi 2024-06-26T07:02:32.8927403Z  2024-06-26T07:02:32.8927920Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-06-26T07:02:32.8928775Z # be empty. The default action would be to continue rebuild the image 2024-06-26T07:02:32.8929553Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-06-26T07:02:32.8930245Z  # if we're on the base branch then use the parent commit 2024-06-26T07:02:32.8930856Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-06-26T07:02:32.8931301Z else 2024-06-26T07:02:32.8931788Z  # otherwise we're on a PR, so use the most recent base commit 2024-06-26T07:02:32.8932785Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-06-26T07:02:32.8933310Z fi 2024-06-26T07:02:32.8933599Z  2024-06-26T07:02:32.8933997Z if [[ -z "${MERGE_BASE}" ]]; then 2024-06-26T07:02:32.8934519Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:02:32.8934993Z  2024-06-26T07:02:32.8935674Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-06-26T07:02:32.8936458Z  exit 0 2024-06-26T07:02:32.8936771Z fi 2024-06-26T07:02:32.8937059Z  2024-06-26T07:02:32.8937519Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-06-26T07:02:32.8938515Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-06-26T07:02:32.8939334Z  exit 1 2024-06-26T07:02:32.8939641Z fi 2024-06-26T07:02:32.8939931Z  2024-06-26T07:02:32.8940436Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-06-26T07:02:32.8941403Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-06-26T07:02:32.8942295Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-06-26T07:02:32.8943291Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-06-26T07:02:32.8944421Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-06-26T07:02:32.8945212Z fi 2024-06-26T07:02:32.8945492Z  2024-06-26T07:02:32.8945857Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:02:32.8953582Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:02:32.8954094Z env: 2024-06-26T07:02:32.8954376Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:32.8954757Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:02:32.8955228Z BASE_REVISION: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:02:32.8956523Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:32.8957784Z DOCKER_TAG: 91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:32.8958394Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:02:32.8958920Z ##[endgroup] 2024-06-26T07:02:32.8978797Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:02:32.8979594Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:02:32.8980389Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:02:32.8981165Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:02:33.3023456Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:02:33.3024289Z Configure a credential helper to remove this warning. See 2024-06-26T07:02:33.3025114Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:02:33.3025661Z 2024-06-26T07:02:33.3025783Z Login Succeeded 2024-06-26T07:02:33.3035954Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:33.5125182Z { 2024-06-26T07:02:33.5125569Z "schemaVersion": 2, 2024-06-26T07:02:33.5126254Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-06-26T07:02:33.5126868Z "config": { 2024-06-26T07:02:33.5127331Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-06-26T07:02:33.5127896Z "size": 48681, 2024-06-26T07:02:33.5130823Z "digest": "sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd" 2024-06-26T07:02:33.5131613Z }, 2024-06-26T07:02:33.5131983Z "layers": [ 2024-06-26T07:02:33.5132627Z { 2024-06-26T07:02:33.5133099Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5133772Z "size": 28580681, 2024-06-26T07:02:33.5134356Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-06-26T07:02:33.5135165Z }, 2024-06-26T07:02:33.5135458Z { 2024-06-26T07:02:33.5136077Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5136829Z "size": 7943451, 2024-06-26T07:02:33.5137490Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-06-26T07:02:33.5138290Z }, 2024-06-26T07:02:33.5138644Z { 2024-06-26T07:02:33.5139243Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5139892Z "size": 55728572, 2024-06-26T07:02:33.5140512Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-06-26T07:02:33.5141242Z }, 2024-06-26T07:02:33.5141572Z { 2024-06-26T07:02:33.5142020Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5142608Z "size": 186, 2024-06-26T07:02:33.5143317Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-06-26T07:02:33.5143984Z }, 2024-06-26T07:02:33.5144264Z { 2024-06-26T07:02:33.5144858Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5145615Z "size": 6886, 2024-06-26T07:02:33.5146245Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-06-26T07:02:33.5146880Z }, 2024-06-26T07:02:33.5147317Z { 2024-06-26T07:02:33.5147795Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5148493Z "size": 1329001756, 2024-06-26T07:02:33.5149267Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-06-26T07:02:33.5150100Z }, 2024-06-26T07:02:33.5150416Z { 2024-06-26T07:02:33.5150978Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5151696Z "size": 62501, 2024-06-26T07:02:33.5152409Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-06-26T07:02:33.5153208Z }, 2024-06-26T07:02:33.5153527Z { 2024-06-26T07:02:33.5154017Z + exit 0 2024-06-26T07:02:33.5154665Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5155391Z "size": 1684, 2024-06-26T07:02:33.5155926Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-06-26T07:02:33.5156689Z }, 2024-06-26T07:02:33.5156944Z { 2024-06-26T07:02:33.5157382Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5157941Z "size": 1523, 2024-06-26T07:02:33.5158498Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-06-26T07:02:33.5159127Z }, 2024-06-26T07:02:33.5159378Z { 2024-06-26T07:02:33.5159808Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5160374Z "size": 2528295403, 2024-06-26T07:02:33.5161162Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-06-26T07:02:33.5161838Z }, 2024-06-26T07:02:33.5162088Z { 2024-06-26T07:02:33.5162522Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5163080Z "size": 86016, 2024-06-26T07:02:33.5163631Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-06-26T07:02:33.5164273Z }, 2024-06-26T07:02:33.5164548Z { 2024-06-26T07:02:33.5164984Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5165538Z "size": 1823, 2024-06-26T07:02:33.5166064Z "digest": "sha256:309b872da805fcdf74137a887464876fa0615dc8b24199307119eb65fc2bfb49" 2024-06-26T07:02:33.5166667Z }, 2024-06-26T07:02:33.5166912Z { 2024-06-26T07:02:33.5167347Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5167908Z "size": 246737542, 2024-06-26T07:02:33.5168455Z "digest": "sha256:e82932cb002c06a782077b8e5965f8733eb8563406a23cab260d9d8c4a994f7d" 2024-06-26T07:02:33.5169061Z }, 2024-06-26T07:02:33.5169312Z { 2024-06-26T07:02:33.5169738Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5170290Z "size": 545, 2024-06-26T07:02:33.5170837Z "digest": "sha256:ca9dd73e3e9378225ebdcfd12199ef44649e34fe55003cfbc8afe2e8a4527c5b" 2024-06-26T07:02:33.5171457Z }, 2024-06-26T07:02:33.5171695Z { 2024-06-26T07:02:33.5172401Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5172965Z "size": 1283, 2024-06-26T07:02:33.5173493Z "digest": "sha256:16d76a79d5d6616e228537785ec83a869f68a402520bbc6e826201621ca6cb98" 2024-06-26T07:02:33.5174189Z }, 2024-06-26T07:02:33.5174438Z { 2024-06-26T07:02:33.5174867Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5175425Z "size": 485, 2024-06-26T07:02:33.5175963Z "digest": "sha256:4bdf41c9ecbe1651e0986ccbea4663a0556af27b7145a679cfa45c35dc96b891" 2024-06-26T07:02:33.5176586Z }, 2024-06-26T07:02:33.5176837Z { 2024-06-26T07:02:33.5177262Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5177826Z "size": 91710937, 2024-06-26T07:02:33.5178394Z "digest": "sha256:f4863801e30cdbcd1d4e6cb1cbbcde7dbcaf6ae317ca80429295741ce60ac31d" 2024-06-26T07:02:33.5179018Z }, 2024-06-26T07:02:33.5179255Z { 2024-06-26T07:02:33.5179685Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5180355Z "size": 3233, 2024-06-26T07:02:33.5180908Z "digest": "sha256:d37dd9ed8948a0d02e1198bbedadbca36477a0980dd553196385aab0c025581d" 2024-06-26T07:02:33.5181518Z }, 2024-06-26T07:02:33.5181765Z { 2024-06-26T07:02:33.5182194Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5182753Z "size": 1905, 2024-06-26T07:02:33.5183299Z "digest": "sha256:cf2f34f225dbed17c1a9a8b9ea4ad030aba797774a716dc0a82fe212324aa751" 2024-06-26T07:02:33.5183927Z }, 2024-06-26T07:02:33.5184174Z { 2024-06-26T07:02:33.5184601Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5185154Z "size": 702, 2024-06-26T07:02:33.5185691Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:02:33.5186312Z }, 2024-06-26T07:02:33.5186552Z { 2024-06-26T07:02:33.5186987Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5187625Z "size": 2784789002, 2024-06-26T07:02:33.5188191Z "digest": "sha256:3d914950f8addfc4d657a2365c76e1143eb80b710cd3f6e9ae05640217fc6347" 2024-06-26T07:02:33.5188795Z }, 2024-06-26T07:02:33.5189044Z { 2024-06-26T07:02:33.5189478Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5190024Z "size": 381, 2024-06-26T07:02:33.5190565Z "digest": "sha256:96d0e2032184c57e62a652aaffcf0241f0d0c7a23e894ca0ad192c4a64a36583" 2024-06-26T07:02:33.5191180Z }, 2024-06-26T07:02:33.5191428Z { 2024-06-26T07:02:33.5191859Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5192419Z "size": 11879, 2024-06-26T07:02:33.5192955Z "digest": "sha256:06e12ce59790cb8b091e5d619c2a62a28329008ee184a8b9a562e3db12471f30" 2024-06-26T07:02:33.5193553Z }, 2024-06-26T07:02:33.5193795Z { 2024-06-26T07:02:33.5194224Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5194834Z "size": 803, 2024-06-26T07:02:33.5195392Z "digest": "sha256:ed712f63cde7070bdec2a6179c4c35e752c6991ddbc69fe9f2724a9903250ceb" 2024-06-26T07:02:33.5196001Z }, 2024-06-26T07:02:33.5196256Z { 2024-06-26T07:02:33.5196693Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5197244Z "size": 106, 2024-06-26T07:02:33.5197775Z "digest": "sha256:a97a71ef8a4e66234dfc2098834f8395ac5630240e60164d1a251fdb0115ba85" 2024-06-26T07:02:33.5198380Z }, 2024-06-26T07:02:33.5198622Z { 2024-06-26T07:02:33.5199044Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5199594Z "size": 503, 2024-06-26T07:02:33.5200126Z "digest": "sha256:25cd0c482ad612f936b35b4f220c580e16be982226aac8801a6a849a6bfdc221" 2024-06-26T07:02:33.5200737Z }, 2024-06-26T07:02:33.5200974Z { 2024-06-26T07:02:33.5201401Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5201958Z "size": 121477177, 2024-06-26T07:02:33.5202520Z "digest": "sha256:9bf94bc456584365fda21d0ed83b7cc690b08a71727b71c4c1e667e15028a386" 2024-06-26T07:02:33.5203125Z }, 2024-06-26T07:02:33.5203374Z { 2024-06-26T07:02:33.5203803Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5204352Z "size": 109, 2024-06-26T07:02:33.5204887Z "digest": "sha256:de45a14c9b227e4b0c9d45d42223b50a71c03522ddd9e2ca0d9517835206c823" 2024-06-26T07:02:33.5205546Z }, 2024-06-26T07:02:33.5205790Z { 2024-06-26T07:02:33.5206214Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5206762Z "size": 491, 2024-06-26T07:02:33.5207289Z "digest": "sha256:e99241e77209b0b35a0fb59a61c0821fc161190178af7e4af45e6eeaa043375e" 2024-06-26T07:02:33.5207904Z }, 2024-06-26T07:02:33.5208142Z { 2024-06-26T07:02:33.5208576Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5209127Z "size": 291, 2024-06-26T07:02:33.5209665Z "digest": "sha256:35e8f4e2a42f9df0c969dedd2199cf71719cf5ecfa676dfa014bbdeb4fdb295f" 2024-06-26T07:02:33.5210362Z }, 2024-06-26T07:02:33.5210608Z { 2024-06-26T07:02:33.5211038Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5211580Z "size": 101, 2024-06-26T07:02:33.5212288Z "digest": "sha256:a823ffa4c541394b52953391dc0a6867e26c37fe4a71fa022a2cc3f07f0ba70d" 2024-06-26T07:02:33.5212905Z }, 2024-06-26T07:02:33.5213155Z { 2024-06-26T07:02:33.5213585Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5214195Z "size": 1473, 2024-06-26T07:02:33.5214723Z "digest": "sha256:44ba272eae2064603605a6238b3f35cc6578859d660068e702a047a9cd8ab2fd" 2024-06-26T07:02:33.5215368Z }, 2024-06-26T07:02:33.5215619Z { 2024-06-26T07:02:33.5216050Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5216609Z "size": 446710781, 2024-06-26T07:02:33.5217160Z "digest": "sha256:da02d278e5240c285bc855ce4e42cd6121d5b903787f5c14f03e575ad1268650" 2024-06-26T07:02:33.5217888Z }, 2024-06-26T07:02:33.5218144Z { 2024-06-26T07:02:33.5218577Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5219126Z "size": 161, 2024-06-26T07:02:33.5219644Z "digest": "sha256:73a0ce681c295135456e190578733273276b38b4dd5babbb39298f384d43b1c6" 2024-06-26T07:02:33.5220241Z }, 2024-06-26T07:02:33.5220491Z { 2024-06-26T07:02:33.5220914Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5221467Z "size": 564, 2024-06-26T07:02:33.5222008Z "digest": "sha256:49737bd5ae709dbf7956e21fc8b05ba009e9e6c9b130b0126a73bd3d3f5efc78" 2024-06-26T07:02:33.5222612Z }, 2024-06-26T07:02:33.5222856Z { 2024-06-26T07:02:33.5223295Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5223852Z "size": 35874600, 2024-06-26T07:02:33.5224450Z "digest": "sha256:9cd4c745f8e4f9d5c9bb8050923575afad21f5b7852975a4a1981bc9eff282c5" 2024-06-26T07:02:33.5225058Z }, 2024-06-26T07:02:33.5225310Z { 2024-06-26T07:02:33.5225746Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5226301Z "size": 104, 2024-06-26T07:02:33.5226818Z "digest": "sha256:b2950a0365345d4c17a48627c905fd8930b3b3154c727f0526eb9ca547102bb7" 2024-06-26T07:02:33.5227410Z }, 2024-06-26T07:02:33.5227652Z { 2024-06-26T07:02:33.5228079Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5228633Z "size": 425, 2024-06-26T07:02:33.5229157Z "digest": "sha256:25e19d81d9fc081962ff4cbc09e098f4035419822712c19a410f5282300dfb9e" 2024-06-26T07:02:33.5229752Z }, 2024-06-26T07:02:33.5230001Z { 2024-06-26T07:02:33.5230430Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5230989Z "size": 20262084, 2024-06-26T07:02:33.5231549Z "digest": "sha256:60156b40fcbec2c25f0170b0a82fa8cd9303e0daae8bb1811fd0fc6562e88e8e" 2024-06-26T07:02:33.5232160Z }, 2024-06-26T07:02:33.5232411Z { 2024-06-26T07:02:33.5232854Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5233407Z "size": 439, 2024-06-26T07:02:33.5233930Z "digest": "sha256:7a868239227fd19b30a19927bbc8f058d247403fe5b343c55f4b4b8987918911" 2024-06-26T07:02:33.5234526Z }, 2024-06-26T07:02:33.5234765Z { 2024-06-26T07:02:33.5235247Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5235807Z "size": 702, 2024-06-26T07:02:33.5236344Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:02:33.5236948Z }, 2024-06-26T07:02:33.5237194Z { 2024-06-26T07:02:33.5237624Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5238175Z "size": 143, 2024-06-26T07:02:33.5238692Z "digest": "sha256:83911d7c7ecf11b5932bb7449975c52e7f737a5b867691935e1ccb85ceb25b52" 2024-06-26T07:02:33.5239299Z }, 2024-06-26T07:02:33.5239542Z { 2024-06-26T07:02:33.5239963Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5240621Z "size": 135, 2024-06-26T07:02:33.5241175Z "digest": "sha256:b3c5d66c679f58c71febdd502889bd8cd3d36e5001754cdf1e30eabe21135cce" 2024-06-26T07:02:33.5241799Z }, 2024-06-26T07:02:33.5242039Z { 2024-06-26T07:02:33.5242472Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5243027Z "size": 4911495247, 2024-06-26T07:02:33.5243582Z "digest": "sha256:84b2c7a899820f6acdb22148c5e8346d8b26c33269fd01842c9573515251e1e1" 2024-06-26T07:02:33.5244178Z }, 2024-06-26T07:02:33.5244423Z { 2024-06-26T07:02:33.5244870Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5245457Z "size": 190, 2024-06-26T07:02:33.5245995Z "digest": "sha256:c1068d03a811d8dead5da9e82fdc8869205fc4ee9cb92d4f7a4acfbb5ba83950" 2024-06-26T07:02:33.5246617Z }, 2024-06-26T07:02:33.5246863Z { 2024-06-26T07:02:33.5247283Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5247899Z "size": 565, 2024-06-26T07:02:33.5248436Z "digest": "sha256:671b4a301d81074fca17d36851f81801552c4ecb2f90ff3412aa9ab6a1a91371" 2024-06-26T07:02:33.5249040Z }, 2024-06-26T07:02:33.5249277Z { 2024-06-26T07:02:33.5249708Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5250265Z "size": 43163095, 2024-06-26T07:02:33.5250821Z "digest": "sha256:67760cf5ec46ab58d3ca98637fd9b26cb47a67446bfd0db176fe5e023606aa0d" 2024-06-26T07:02:33.5251432Z }, 2024-06-26T07:02:33.5251676Z { 2024-06-26T07:02:33.5252279Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5252840Z "size": 106, 2024-06-26T07:02:33.5253379Z "digest": "sha256:ddc9740bcea2b301e99b50525059aea8d9c7121a1c8eab566561a8968438a8c2" 2024-06-26T07:02:33.5254053Z }, 2024-06-26T07:02:33.5254301Z { 2024-06-26T07:02:33.5254724Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5255276Z "size": 1213, 2024-06-26T07:02:33.5255829Z "digest": "sha256:fb2e977685609fcc01de170ba7f50a573e73e6520e47253717e92a28952c5ee8" 2024-06-26T07:02:33.5256446Z }, 2024-06-26T07:02:33.5256684Z { 2024-06-26T07:02:33.5257120Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5257674Z "size": 702, 2024-06-26T07:02:33.5258208Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:02:33.5258816Z }, 2024-06-26T07:02:33.5259058Z { 2024-06-26T07:02:33.5259486Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5260030Z "size": 138, 2024-06-26T07:02:33.5260570Z "digest": "sha256:b599c5a8c7cb461cbd7b3c98b99960b8eb04506e05e79db1cc0cfadb14a47e91" 2024-06-26T07:02:33.5261192Z }, 2024-06-26T07:02:33.5261436Z { 2024-06-26T07:02:33.5261860Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5262414Z "size": 120, 2024-06-26T07:02:33.5262960Z "digest": "sha256:e08e88c1366827455ee7ddbf92b6ae615ceb3cbd3fb11cd03ddd31bd3fa94a9c" 2024-06-26T07:02:33.5263586Z }, 2024-06-26T07:02:33.5263827Z { 2024-06-26T07:02:33.5264253Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5264808Z "size": 1904157495, 2024-06-26T07:02:33.5265368Z "digest": "sha256:12ba7431ff38de8cf6af3052a2a52e79f4e712b8c99f5bafde3c20e63b1f570f" 2024-06-26T07:02:33.5265984Z }, 2024-06-26T07:02:33.5266235Z { 2024-06-26T07:02:33.5266666Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5267206Z "size": 172, 2024-06-26T07:02:33.5267743Z "digest": "sha256:cf7c2cb896cba6a594028cc0fd246e127a2f4481d48905a3aa12c5f5872332d4" 2024-06-26T07:02:33.5268356Z }, 2024-06-26T07:02:33.5268601Z { 2024-06-26T07:02:33.5269021Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5269574Z "size": 908, 2024-06-26T07:02:33.5270120Z "digest": "sha256:3a778ddfac2af5b3a11f49ddf2f3f1ad1731f74ba691f434ab69966d1e2a189f" 2024-06-26T07:02:33.5270861Z }, 2024-06-26T07:02:33.5271105Z { 2024-06-26T07:02:33.5271537Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5272093Z "size": 702, 2024-06-26T07:02:33.5272621Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:02:33.5273236Z }, 2024-06-26T07:02:33.5273480Z { 2024-06-26T07:02:33.5273908Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5274454Z "size": 134, 2024-06-26T07:02:33.5275033Z "digest": "sha256:236c8f0245f4b40616a4322eaf1e1b3e178d5990db1fcaba39622ea4e79c06a1" 2024-06-26T07:02:33.5275642Z }, 2024-06-26T07:02:33.5275887Z { 2024-06-26T07:02:33.5276309Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5276859Z "size": 32, 2024-06-26T07:02:33.5277402Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:02:33.5278087Z }, 2024-06-26T07:02:33.5278341Z { 2024-06-26T07:02:33.5278773Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5279327Z "size": 156, 2024-06-26T07:02:33.5279865Z "digest": "sha256:dda1ce302b9a9df0c8f1145d2e3eaa03c96909bb45a792451f1fcb555b06738d" 2024-06-26T07:02:33.5280483Z }, 2024-06-26T07:02:33.5280725Z { 2024-06-26T07:02:33.5281155Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5281698Z "size": 1841, 2024-06-26T07:02:33.5282241Z "digest": "sha256:e797fe86caef19319e03f8dca92cf96838ab3bb3644f4447c6dfc04230118afa" 2024-06-26T07:02:33.5282855Z }, 2024-06-26T07:02:33.5283101Z { 2024-06-26T07:02:33.5283547Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5284097Z "size": 7529784, 2024-06-26T07:02:33.5284643Z "digest": "sha256:967190076fc0ab2bb2fc0d74e58edd3d662a537d702ff3f8b6e876230154a16f" 2024-06-26T07:02:33.5285241Z }, 2024-06-26T07:02:33.5285492Z { 2024-06-26T07:02:33.5285925Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5286477Z "size": 165, 2024-06-26T07:02:33.5287002Z "digest": "sha256:4fb0a83f06855534ef82b80dcf7c685303edb2bcbd446196066dc2ad2dd71281" 2024-06-26T07:02:33.5287620Z }, 2024-06-26T07:02:33.5287865Z { 2024-06-26T07:02:33.5288294Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5288845Z "size": 7944, 2024-06-26T07:02:33.5289380Z "digest": "sha256:68af2ec4e30a33f26947ece9f3a31b7d3b04c2bca5865d4b48618457832a1da2" 2024-06-26T07:02:33.5289995Z }, 2024-06-26T07:02:33.5290238Z { 2024-06-26T07:02:33.5290663Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5291218Z "size": 8070, 2024-06-26T07:02:33.5291750Z "digest": "sha256:cc91933e985a044c6df53caa15488d2285896f5e991f38ec068c253ceec53f14" 2024-06-26T07:02:33.5292514Z }, 2024-06-26T07:02:33.5292764Z { 2024-06-26T07:02:33.5293219Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5293820Z "size": 303, 2024-06-26T07:02:33.5294351Z "digest": "sha256:9f3187916e7dcad33cc572f8f0e949d2f507b887f99cfd6acefd14826b44960d" 2024-06-26T07:02:33.5294989Z }, 2024-06-26T07:02:33.5295264Z { 2024-06-26T07:02:33.5295694Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5296247Z "size": 7629071, 2024-06-26T07:02:33.5296769Z "digest": "sha256:52608399485738d10d779254946c9e1d536010dd45593c69bf1eb9f62f85dc24" 2024-06-26T07:02:33.5297359Z }, 2024-06-26T07:02:33.5297599Z { 2024-06-26T07:02:33.5298027Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5298580Z "size": 108, 2024-06-26T07:02:33.5299116Z "digest": "sha256:05a6f6dcc1c00bac6660923a22adc7d7c34416718f8c89daf9b3351d9365f694" 2024-06-26T07:02:33.5299726Z }, 2024-06-26T07:02:33.5299970Z { 2024-06-26T07:02:33.5300395Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5301049Z "size": 54145775, 2024-06-26T07:02:33.5301610Z "digest": "sha256:6bc9dc8315b09b27c976bfba9700767ba412d72fd0528b18bbe8f8e62005ec22" 2024-06-26T07:02:33.5302223Z }, 2024-06-26T07:02:33.5302467Z { 2024-06-26T07:02:33.5302888Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5303445Z "size": 474, 2024-06-26T07:02:33.5303981Z "digest": "sha256:1b0e27c35f07b3fbf0aeb548d1949bf1d60a420136da21789d94d196c891d981" 2024-06-26T07:02:33.5304592Z }, 2024-06-26T07:02:33.5304856Z { 2024-06-26T07:02:33.5305309Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5305863Z "size": 1374858747, 2024-06-26T07:02:33.5306407Z "digest": "sha256:95237e7311a1c5ce6937db7196c866632f44e8788514a5c0eb4a3f04da4017e6" 2024-06-26T07:02:33.5306997Z }, 2024-06-26T07:02:33.5307240Z { 2024-06-26T07:02:33.5307669Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5308294Z "size": 106, 2024-06-26T07:02:33.5308841Z "digest": "sha256:789eff6c51aada70bf75eb1acafed63c0d839b768638001500616ff6e53b854a" 2024-06-26T07:02:33.5309459Z }, 2024-06-26T07:02:33.5309702Z { 2024-06-26T07:02:33.5310123Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5310672Z "size": 558, 2024-06-26T07:02:33.5311211Z "digest": "sha256:de0f8a8c45c6077b3c5ba3b84d7f07de98353fd618dad9a1e2f368ffe00493c7" 2024-06-26T07:02:33.5311830Z }, 2024-06-26T07:02:33.5312067Z { 2024-06-26T07:02:33.5312492Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5313044Z "size": 46248513, 2024-06-26T07:02:33.5313586Z "digest": "sha256:2060028f74888d218bebdd10d723f66c33ee1c708935c4ffd4654d34912ac4dc" 2024-06-26T07:02:33.5314179Z }, 2024-06-26T07:02:33.5314426Z { 2024-06-26T07:02:33.5314854Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5315401Z "size": 111, 2024-06-26T07:02:33.5315947Z "digest": "sha256:e412c7f3b2ded440992955bd92512345cbece8a8894cd3592f714ebc758ad1a0" 2024-06-26T07:02:33.5316554Z }, 2024-06-26T07:02:33.5316797Z { 2024-06-26T07:02:33.5317217Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5317770Z "size": 32, 2024-06-26T07:02:33.5318305Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:02:33.5318916Z }, 2024-06-26T07:02:33.5319158Z { 2024-06-26T07:02:33.5319587Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5320139Z "size": 32, 2024-06-26T07:02:33.5320677Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:02:33.5321280Z }, 2024-06-26T07:02:33.5321523Z { 2024-06-26T07:02:33.5321955Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5322500Z "size": 32, 2024-06-26T07:02:33.5323040Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:02:33.5323659Z }, 2024-06-26T07:02:33.5323905Z { 2024-06-26T07:02:33.5324326Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:02:33.5324885Z "size": 32, 2024-06-26T07:02:33.5325465Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:02:33.5326080Z } 2024-06-26T07:02:33.5326316Z ] 2024-06-26T07:02:33.5326558Z } 2024-06-26T07:02:33.5442592Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:02:33.5443092Z tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:02:33.5443643Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-06-26T07:02:33.5451532Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:02:33.5452241Z env: 2024-06-26T07:02:33.5452556Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:33.5453811Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:33.5455004Z ##[endgroup] 2024-06-26T07:02:33.5477556Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:33.5527126Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-06-26T07:02:33.5527701Z with: 2024-06-26T07:02:33.5528790Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:33.5530121Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:02:33.5530644Z env: 2024-06-26T07:02:33.5530927Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:33.5531281Z ##[endgroup] 2024-06-26T07:02:33.5547023Z ##[group]Run set -x 2024-06-26T07:02:33.5547369Z set -x 2024-06-26T07:02:33.5547674Z set +e 2024-06-26T07:02:33.5548080Z  2024-06-26T07:02:33.5548351Z login() { 2024-06-26T07:02:33.5549020Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:02:33.5549750Z } 2024-06-26T07:02:33.5550028Z  2024-06-26T07:02:33.5550328Z retry () { 2024-06-26T07:02:33.5550726Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:02:33.5551171Z } 2024-06-26T07:02:33.5551451Z  2024-06-26T07:02:33.5551771Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:02:33.5552187Z  2024-06-26T07:02:33.5552459Z set -e 2024-06-26T07:02:33.5552938Z # ignore output since only exit code is used for conditional 2024-06-26T07:02:33.5553654Z # only pull docker image if it's not available locally 2024-06-26T07:02:33.5554427Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-06-26T07:02:33.5555136Z  retry docker pull "${DOCKER_IMAGE}" 2024-06-26T07:02:33.5555590Z fi 2024-06-26T07:02:33.5563021Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:02:33.5563517Z env: 2024-06-26T07:02:33.5563797Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:02:33.5565004Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:33.5566320Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:02:33.5566844Z ##[endgroup] 2024-06-26T07:02:33.5584933Z + set +e 2024-06-26T07:02:33.5585432Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:02:33.5586062Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:02:33.5587682Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:02:33.5588461Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:02:33.9593769Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:02:33.9594885Z Configure a credential helper to remove this warning. See 2024-06-26T07:02:33.9595885Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:02:33.9596443Z 2024-06-26T07:02:33.9596565Z Login Succeeded 2024-06-26T07:02:33.9603723Z + set -e 2024-06-26T07:02:33.9605247Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:33.9732646Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:33.9734898Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:02:34.2218947Z 91382da70d5719cd7007b6b80b71d2f48398f6b7: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-06-26T07:02:34.2223463Z 7a2c55901189: Pulling fs layer 2024-06-26T07:02:34.2225380Z 224fe954d725: Pulling fs layer 2024-06-26T07:02:34.2226046Z 75722010b82e: Pulling fs layer 2024-06-26T07:02:34.2226949Z d527cbbb87e3: Pulling fs layer 2024-06-26T07:02:34.2227402Z b57676e46aee: Pulling fs layer 2024-06-26T07:02:34.2227793Z a8c1e85b5e14: Pulling fs layer 2024-06-26T07:02:34.2228231Z a41a8d1c11c8: Pulling fs layer 2024-06-26T07:02:34.2228654Z 0c1227890755: Pulling fs layer 2024-06-26T07:02:34.2229094Z d8d1234baab3: Pulling fs layer 2024-06-26T07:02:34.2229476Z 7ed32bc8e469: Pulling fs layer 2024-06-26T07:02:34.2229858Z ec1e7978c1fe: Pulling fs layer 2024-06-26T07:02:34.2230254Z 309b872da805: Pulling fs layer 2024-06-26T07:02:34.2230730Z e82932cb002c: Pulling fs layer 2024-06-26T07:02:34.2231239Z ca9dd73e3e93: Pulling fs layer 2024-06-26T07:02:34.2231861Z b57676e46aee: Waiting 2024-06-26T07:02:34.2232205Z 16d76a79d5d6: Pulling fs layer 2024-06-26T07:02:34.2232623Z a8c1e85b5e14: Waiting 2024-06-26T07:02:34.2232970Z 4bdf41c9ecbe: Pulling fs layer 2024-06-26T07:02:34.2233344Z f4863801e30c: Pulling fs layer 2024-06-26T07:02:34.2233729Z d37dd9ed8948: Pulling fs layer 2024-06-26T07:02:34.2234117Z cf2f34f225db: Pulling fs layer 2024-06-26T07:02:34.2234494Z df546d7b3e34: Pulling fs layer 2024-06-26T07:02:34.2234873Z 3d914950f8ad: Pulling fs layer 2024-06-26T07:02:34.2235252Z 96d0e2032184: Pulling fs layer 2024-06-26T07:02:34.2235625Z 06e12ce59790: Pulling fs layer 2024-06-26T07:02:34.2236049Z ed712f63cde7: Pulling fs layer 2024-06-26T07:02:34.2236432Z a97a71ef8a4e: Pulling fs layer 2024-06-26T07:02:34.2236865Z 25cd0c482ad6: Pulling fs layer 2024-06-26T07:02:34.2237249Z 9bf94bc45658: Pulling fs layer 2024-06-26T07:02:34.2237624Z de45a14c9b22: Pulling fs layer 2024-06-26T07:02:34.2237990Z a41a8d1c11c8: Waiting 2024-06-26T07:02:34.2238326Z e99241e77209: Pulling fs layer 2024-06-26T07:02:34.2238708Z 35e8f4e2a42f: Pulling fs layer 2024-06-26T07:02:34.2239076Z 4bdf41c9ecbe: Waiting 2024-06-26T07:02:34.2239413Z 0c1227890755: Waiting 2024-06-26T07:02:34.2239840Z 3d914950f8ad: Waiting 2024-06-26T07:02:34.2240263Z f4863801e30c: Waiting 2024-06-26T07:02:34.2240714Z 25cd0c482ad6: Waiting 2024-06-26T07:02:34.2241185Z 96d0e2032184: Waiting 2024-06-26T07:02:34.2241624Z d37dd9ed8948: Waiting 2024-06-26T07:02:34.2241995Z d8d1234baab3: Waiting 2024-06-26T07:02:34.2242314Z 9bf94bc45658: Waiting 2024-06-26T07:02:34.2242630Z cf2f34f225db: Waiting 2024-06-26T07:02:34.2242939Z 06e12ce59790: Waiting 2024-06-26T07:02:34.2243255Z df546d7b3e34: Waiting 2024-06-26T07:02:34.2243574Z de45a14c9b22: Waiting 2024-06-26T07:02:34.2243894Z ed712f63cde7: Waiting 2024-06-26T07:02:34.2244244Z a97a71ef8a4e: Waiting 2024-06-26T07:02:34.2244561Z 7ed32bc8e469: Waiting 2024-06-26T07:02:34.2244872Z e99241e77209: Waiting 2024-06-26T07:02:34.2245178Z 35e8f4e2a42f: Waiting 2024-06-26T07:02:34.2245498Z ec1e7978c1fe: Waiting 2024-06-26T07:02:34.2245824Z ca9dd73e3e93: Waiting 2024-06-26T07:02:34.2246195Z e82932cb002c: Waiting 2024-06-26T07:02:34.2246506Z 309b872da805: Waiting 2024-06-26T07:02:34.2246819Z 16d76a79d5d6: Waiting 2024-06-26T07:02:34.2247157Z a823ffa4c541: Pulling fs layer 2024-06-26T07:02:34.2247577Z 44ba272eae20: Pulling fs layer 2024-06-26T07:02:34.2247992Z da02d278e524: Pulling fs layer 2024-06-26T07:02:34.2248376Z 73a0ce681c29: Pulling fs layer 2024-06-26T07:02:34.2248754Z 49737bd5ae70: Pulling fs layer 2024-06-26T07:02:34.2249170Z 9cd4c745f8e4: Pulling fs layer 2024-06-26T07:02:34.2249541Z b2950a036534: Pulling fs layer 2024-06-26T07:02:34.2249921Z 25e19d81d9fc: Pulling fs layer 2024-06-26T07:02:34.2250301Z 60156b40fcbe: Pulling fs layer 2024-06-26T07:02:34.2250658Z 73a0ce681c29: Waiting 2024-06-26T07:02:34.2250982Z 7a868239227f: Pulling fs layer 2024-06-26T07:02:34.2251338Z da02d278e524: Waiting 2024-06-26T07:02:34.2251659Z a823ffa4c541: Waiting 2024-06-26T07:02:34.2251978Z 49737bd5ae70: Waiting 2024-06-26T07:02:34.2252553Z 83911d7c7ecf: Pulling fs layer 2024-06-26T07:02:34.2252918Z 9cd4c745f8e4: Waiting 2024-06-26T07:02:34.2253251Z b3c5d66c679f: Pulling fs layer 2024-06-26T07:02:34.2253607Z b2950a036534: Waiting 2024-06-26T07:02:34.2254008Z 60156b40fcbe: Waiting 2024-06-26T07:02:34.2254485Z 84b2c7a89982: Pulling fs layer 2024-06-26T07:02:34.2254880Z c1068d03a811: Pulling fs layer 2024-06-26T07:02:34.2255250Z 671b4a301d81: Pulling fs layer 2024-06-26T07:02:34.2255630Z 67760cf5ec46: Pulling fs layer 2024-06-26T07:02:34.2256010Z 25e19d81d9fc: Waiting 2024-06-26T07:02:34.2256354Z ddc9740bcea2: Pulling fs layer 2024-06-26T07:02:34.2256713Z c1068d03a811: Waiting 2024-06-26T07:02:34.2257031Z 84b2c7a89982: Waiting 2024-06-26T07:02:34.2257350Z 671b4a301d81: Waiting 2024-06-26T07:02:34.2257684Z fb2e97768560: Pulling fs layer 2024-06-26T07:02:34.2258053Z 67760cf5ec46: Waiting 2024-06-26T07:02:34.2258376Z ddc9740bcea2: Waiting 2024-06-26T07:02:34.2258716Z b599c5a8c7cb: Pulling fs layer 2024-06-26T07:02:34.2259075Z fb2e97768560: Waiting 2024-06-26T07:02:34.2259485Z e08e88c13668: Pulling fs layer 2024-06-26T07:02:34.2259865Z 12ba7431ff38: Pulling fs layer 2024-06-26T07:02:34.2260241Z cf7c2cb896cb: Pulling fs layer 2024-06-26T07:02:34.2260628Z 3a778ddfac2a: Pulling fs layer 2024-06-26T07:02:34.2261015Z 236c8f0245f4: Pulling fs layer 2024-06-26T07:02:34.2261397Z 4f4fb700ef54: Pulling fs layer 2024-06-26T07:02:34.2261756Z 236c8f0245f4: Waiting 2024-06-26T07:02:34.2262076Z 3a778ddfac2a: Waiting 2024-06-26T07:02:34.2262398Z b599c5a8c7cb: Waiting 2024-06-26T07:02:34.2262723Z cf7c2cb896cb: Waiting 2024-06-26T07:02:34.2263036Z e08e88c13668: Waiting 2024-06-26T07:02:34.2263373Z dda1ce302b9a: Pulling fs layer 2024-06-26T07:02:34.2263743Z 4f4fb700ef54: Waiting 2024-06-26T07:02:34.2264077Z e797fe86caef: Pulling fs layer 2024-06-26T07:02:34.2264447Z 12ba7431ff38: Waiting 2024-06-26T07:02:34.2264809Z 967190076fc0: Pulling fs layer 2024-06-26T07:02:34.2265288Z e797fe86caef: Waiting 2024-06-26T07:02:34.2265746Z 4fb0a83f0685: Pulling fs layer 2024-06-26T07:02:34.2266236Z 68af2ec4e30a: Pulling fs layer 2024-06-26T07:02:34.2266622Z cc91933e985a: Pulling fs layer 2024-06-26T07:02:34.2267004Z 9f3187916e7d: Pulling fs layer 2024-06-26T07:02:34.2267372Z 526083994857: Pulling fs layer 2024-06-26T07:02:34.2267735Z 68af2ec4e30a: Waiting 2024-06-26T07:02:34.2268080Z 05a6f6dcc1c0: Pulling fs layer 2024-06-26T07:02:34.2268456Z 6bc9dc8315b0: Pulling fs layer 2024-06-26T07:02:34.2268841Z 1b0e27c35f07: Pulling fs layer 2024-06-26T07:02:34.2269217Z 95237e7311a1: Pulling fs layer 2024-06-26T07:02:34.2269576Z cc91933e985a: Waiting 2024-06-26T07:02:34.2269905Z 789eff6c51aa: Pulling fs layer 2024-06-26T07:02:34.2270287Z de0f8a8c45c6: Pulling fs layer 2024-06-26T07:02:34.2270738Z 2060028f7488: Pulling fs layer 2024-06-26T07:02:34.2271119Z e412c7f3b2de: Pulling fs layer 2024-06-26T07:02:34.2271478Z 6bc9dc8315b0: Waiting 2024-06-26T07:02:34.2271824Z 526083994857: Waiting 2024-06-26T07:02:34.2272141Z 05a6f6dcc1c0: Waiting 2024-06-26T07:02:34.2272458Z 1b0e27c35f07: Waiting 2024-06-26T07:02:34.2272776Z 2060028f7488: Waiting 2024-06-26T07:02:34.2273108Z 95237e7311a1: Waiting 2024-06-26T07:02:34.2273448Z de0f8a8c45c6: Waiting 2024-06-26T07:02:34.2273759Z 789eff6c51aa: Waiting 2024-06-26T07:02:34.2274115Z e412c7f3b2de: Waiting 2024-06-26T07:02:34.3517721Z 224fe954d725: Verifying Checksum 2024-06-26T07:02:34.3518301Z 224fe954d725: Download complete 2024-06-26T07:02:34.4258231Z d527cbbb87e3: Download complete 2024-06-26T07:02:34.4865028Z b57676e46aee: Download complete 2024-06-26T07:02:34.6463706Z 7a2c55901189: Download complete 2024-06-26T07:02:34.7333981Z a41a8d1c11c8: Download complete 2024-06-26T07:02:34.8091743Z 0c1227890755: Download complete 2024-06-26T07:02:34.9164891Z d8d1234baab3: Verifying Checksum 2024-06-26T07:02:34.9165471Z d8d1234baab3: Download complete 2024-06-26T07:02:35.0197997Z 75722010b82e: Verifying Checksum 2024-06-26T07:02:35.0198555Z 75722010b82e: Download complete 2024-06-26T07:02:35.1048810Z ec1e7978c1fe: Verifying Checksum 2024-06-26T07:02:35.1050061Z ec1e7978c1fe: Download complete 2024-06-26T07:02:35.1887345Z 309b872da805: Verifying Checksum 2024-06-26T07:02:35.1888048Z 309b872da805: Download complete 2024-06-26T07:02:35.6735192Z 7a2c55901189: Pull complete 2024-06-26T07:02:35.9823290Z 224fe954d725: Pull complete 2024-06-26T07:02:36.9526657Z 75722010b82e: Pull complete 2024-06-26T07:02:37.0040880Z d527cbbb87e3: Pull complete 2024-06-26T07:02:37.0593682Z b57676e46aee: Pull complete 2024-06-26T07:02:41.2016294Z e82932cb002c: Verifying Checksum 2024-06-26T07:02:41.2016764Z e82932cb002c: Download complete 2024-06-26T07:02:41.3043700Z ca9dd73e3e93: Verifying Checksum 2024-06-26T07:02:41.3044222Z ca9dd73e3e93: Download complete 2024-06-26T07:02:41.4131983Z 16d76a79d5d6: Download complete 2024-06-26T07:02:41.4885300Z 4bdf41c9ecbe: Verifying Checksum 2024-06-26T07:02:41.4885869Z 4bdf41c9ecbe: Download complete 2024-06-26T07:02:43.9339611Z f4863801e30c: Verifying Checksum 2024-06-26T07:02:43.9340274Z f4863801e30c: Download complete 2024-06-26T07:02:44.0172978Z d37dd9ed8948: Verifying Checksum 2024-06-26T07:02:44.0173652Z d37dd9ed8948: Download complete 2024-06-26T07:02:44.1057758Z cf2f34f225db: Download complete 2024-06-26T07:02:44.1784428Z df546d7b3e34: Verifying Checksum 2024-06-26T07:02:44.1785069Z df546d7b3e34: Download complete 2024-06-26T07:02:49.5639143Z a8c1e85b5e14: Verifying Checksum 2024-06-26T07:02:49.6636035Z a8c1e85b5e14: Download complete 2024-06-26T07:02:49.6636456Z 96d0e2032184: Download complete 2024-06-26T07:02:49.7617115Z 06e12ce59790: Verifying Checksum 2024-06-26T07:02:49.7617538Z 06e12ce59790: Download complete 2024-06-26T07:02:49.8685065Z ed712f63cde7: Verifying Checksum 2024-06-26T07:02:49.8685565Z ed712f63cde7: Download complete 2024-06-26T07:02:50.0052338Z a97a71ef8a4e: Verifying Checksum 2024-06-26T07:02:50.0052818Z a97a71ef8a4e: Download complete 2024-06-26T07:02:50.0990360Z 25cd0c482ad6: Verifying Checksum 2024-06-26T07:02:50.0990807Z 25cd0c482ad6: Download complete 2024-06-26T07:02:53.4330886Z 9bf94bc45658: Verifying Checksum 2024-06-26T07:02:53.4331400Z 9bf94bc45658: Download complete 2024-06-26T07:02:53.5056148Z de45a14c9b22: Verifying Checksum 2024-06-26T07:02:53.5056617Z de45a14c9b22: Download complete 2024-06-26T07:02:53.5886203Z e99241e77209: Verifying Checksum 2024-06-26T07:02:53.5886673Z e99241e77209: Download complete 2024-06-26T07:02:53.6524550Z 35e8f4e2a42f: Verifying Checksum 2024-06-26T07:02:53.6525164Z 35e8f4e2a42f: Download complete 2024-06-26T07:02:53.7524857Z a823ffa4c541: Verifying Checksum 2024-06-26T07:02:53.7525406Z a823ffa4c541: Download complete 2024-06-26T07:02:53.8555152Z 44ba272eae20: Download complete 2024-06-26T07:03:00.3919990Z 7ed32bc8e469: Verifying Checksum 2024-06-26T07:03:00.3920538Z 7ed32bc8e469: Download complete 2024-06-26T07:03:00.4613321Z 73a0ce681c29: Verifying Checksum 2024-06-26T07:03:00.4613908Z 73a0ce681c29: Download complete 2024-06-26T07:03:00.5312337Z 49737bd5ae70: Verifying Checksum 2024-06-26T07:03:00.5312934Z 49737bd5ae70: Download complete 2024-06-26T07:03:01.4809651Z 9cd4c745f8e4: Verifying Checksum 2024-06-26T07:03:01.4810260Z 9cd4c745f8e4: Download complete 2024-06-26T07:03:01.5638397Z b2950a036534: Verifying Checksum 2024-06-26T07:03:01.5638959Z b2950a036534: Download complete 2024-06-26T07:03:01.6730708Z 25e19d81d9fc: Verifying Checksum 2024-06-26T07:03:01.6731225Z 25e19d81d9fc: Download complete 2024-06-26T07:03:02.1701521Z 60156b40fcbe: Verifying Checksum 2024-06-26T07:03:02.1701995Z 60156b40fcbe: Download complete 2024-06-26T07:03:02.2192622Z a8c1e85b5e14: Pull complete 2024-06-26T07:03:02.2510863Z 7a868239227f: Verifying Checksum 2024-06-26T07:03:02.2511427Z 7a868239227f: Download complete 2024-06-26T07:03:02.2551388Z a41a8d1c11c8: Pull complete 2024-06-26T07:03:02.3067350Z 0c1227890755: Pull complete 2024-06-26T07:03:02.3418649Z d8d1234baab3: Pull complete 2024-06-26T07:03:02.3518159Z 83911d7c7ecf: Download complete 2024-06-26T07:03:02.4393637Z b3c5d66c679f: Verifying Checksum 2024-06-26T07:03:02.4394243Z b3c5d66c679f: Download complete 2024-06-26T07:03:04.6294018Z da02d278e524: Verifying Checksum 2024-06-26T07:03:04.6295071Z da02d278e524: Download complete 2024-06-26T07:03:04.7094644Z c1068d03a811: Verifying Checksum 2024-06-26T07:03:04.7095253Z c1068d03a811: Download complete 2024-06-26T07:03:04.7924299Z 671b4a301d81: Verifying Checksum 2024-06-26T07:03:04.7925282Z 671b4a301d81: Download complete 2024-06-26T07:03:06.5041020Z 67760cf5ec46: Verifying Checksum 2024-06-26T07:03:06.5041507Z 67760cf5ec46: Download complete 2024-06-26T07:03:06.6019008Z ddc9740bcea2: Verifying Checksum 2024-06-26T07:03:06.6019500Z ddc9740bcea2: Download complete 2024-06-26T07:03:06.6805656Z fb2e97768560: Download complete 2024-06-26T07:03:06.7522431Z b599c5a8c7cb: Verifying Checksum 2024-06-26T07:03:06.7523016Z b599c5a8c7cb: Download complete 2024-06-26T07:03:06.9000551Z e08e88c13668: Verifying Checksum 2024-06-26T07:03:06.9001168Z e08e88c13668: Download complete 2024-06-26T07:03:32.1623647Z 7ed32bc8e469: Pull complete 2024-06-26T07:03:32.3658457Z ec1e7978c1fe: Pull complete 2024-06-26T07:03:32.5669382Z 309b872da805: Pull complete 2024-06-26T07:03:39.9814733Z e82932cb002c: Pull complete 2024-06-26T07:03:40.1763419Z ca9dd73e3e93: Pull complete 2024-06-26T07:03:40.3867571Z 16d76a79d5d6: Pull complete 2024-06-26T07:03:40.5887000Z 4bdf41c9ecbe: Pull complete 2024-06-26T07:03:42.9457447Z f4863801e30c: Pull complete 2024-06-26T07:03:43.1735518Z d37dd9ed8948: Pull complete 2024-06-26T07:03:43.3983684Z cf2f34f225db: Pull complete 2024-06-26T07:03:43.6118927Z df546d7b3e34: Pull complete 2024-06-26T07:03:45.0176251Z 3d914950f8ad: Verifying Checksum 2024-06-26T07:03:45.0176812Z 3d914950f8ad: Download complete 2024-06-26T07:03:45.0869146Z cf7c2cb896cb: Verifying Checksum 2024-06-26T07:03:45.0869718Z cf7c2cb896cb: Download complete 2024-06-26T07:03:45.1585869Z 3a778ddfac2a: Verifying Checksum 2024-06-26T07:03:45.1586418Z 3a778ddfac2a: Download complete 2024-06-26T07:03:45.2253656Z 236c8f0245f4: Verifying Checksum 2024-06-26T07:03:45.2254427Z 236c8f0245f4: Download complete 2024-06-26T07:03:45.2337321Z 4f4fb700ef54: Verifying Checksum 2024-06-26T07:03:45.2338251Z 4f4fb700ef54: Download complete 2024-06-26T07:03:45.2981069Z dda1ce302b9a: Verifying Checksum 2024-06-26T07:03:45.2981725Z dda1ce302b9a: Download complete 2024-06-26T07:03:45.3799922Z e797fe86caef: Download complete 2024-06-26T07:03:45.7668092Z 967190076fc0: Verifying Checksum 2024-06-26T07:03:45.7668680Z 967190076fc0: Download complete 2024-06-26T07:03:45.8411111Z 4fb0a83f0685: Verifying Checksum 2024-06-26T07:03:45.8411705Z 4fb0a83f0685: Download complete 2024-06-26T07:03:45.9207721Z 68af2ec4e30a: Download complete 2024-06-26T07:03:45.9981403Z cc91933e985a: Download complete 2024-06-26T07:03:46.0982349Z 9f3187916e7d: Verifying Checksum 2024-06-26T07:03:46.0982884Z 9f3187916e7d: Download complete 2024-06-26T07:03:46.4242552Z 526083994857: Verifying Checksum 2024-06-26T07:03:46.4243191Z 526083994857: Download complete 2024-06-26T07:03:46.4957765Z 05a6f6dcc1c0: Verifying Checksum 2024-06-26T07:03:46.4958201Z 05a6f6dcc1c0: Download complete 2024-06-26T07:03:48.5376934Z 6bc9dc8315b0: Verifying Checksum 2024-06-26T07:03:48.5377511Z 6bc9dc8315b0: Download complete 2024-06-26T07:03:48.6353512Z 1b0e27c35f07: Download complete 2024-06-26T07:03:52.0348817Z 12ba7431ff38: Verifying Checksum 2024-06-26T07:03:52.0349353Z 12ba7431ff38: Download complete 2024-06-26T07:03:52.1441562Z 789eff6c51aa: Verifying Checksum 2024-06-26T07:03:52.1442226Z 789eff6c51aa: Download complete 2024-06-26T07:03:52.2251743Z de0f8a8c45c6: Verifying Checksum 2024-06-26T07:03:52.2252584Z de0f8a8c45c6: Download complete 2024-06-26T07:03:53.2487626Z 2060028f7488: Verifying Checksum 2024-06-26T07:03:53.2488278Z 2060028f7488: Download complete 2024-06-26T07:03:53.3284473Z e412c7f3b2de: Verifying Checksum 2024-06-26T07:03:53.3284955Z e412c7f3b2de: Download complete 2024-06-26T07:04:15.7660520Z 95237e7311a1: Verifying Checksum 2024-06-26T07:04:15.7661025Z 95237e7311a1: Download complete 2024-06-26T07:04:49.3712661Z 84b2c7a89982: Verifying Checksum 2024-06-26T07:04:49.3713128Z 84b2c7a89982: Download complete 2024-06-26T07:05:06.2551013Z 3d914950f8ad: Pull complete 2024-06-26T07:05:06.4135627Z 96d0e2032184: Pull complete 2024-06-26T07:05:06.6001180Z 06e12ce59790: Pull complete 2024-06-26T07:05:06.7278114Z ed712f63cde7: Pull complete 2024-06-26T07:05:06.8661623Z a97a71ef8a4e: Pull complete 2024-06-26T07:05:07.0307547Z 25cd0c482ad6: Pull complete 2024-06-26T07:05:10.5829291Z 9bf94bc45658: Pull complete 2024-06-26T07:05:10.7131056Z de45a14c9b22: Pull complete 2024-06-26T07:05:10.7773366Z e99241e77209: Pull complete 2024-06-26T07:05:10.9195948Z 35e8f4e2a42f: Pull complete 2024-06-26T07:05:11.0797329Z a823ffa4c541: Pull complete 2024-06-26T07:05:11.2739789Z 44ba272eae20: Pull complete 2024-06-26T07:05:19.6313715Z da02d278e524: Pull complete 2024-06-26T07:05:19.8335096Z 73a0ce681c29: Pull complete 2024-06-26T07:05:20.0541230Z 49737bd5ae70: Pull complete 2024-06-26T07:05:21.0525027Z 9cd4c745f8e4: Pull complete 2024-06-26T07:05:21.2661592Z b2950a036534: Pull complete 2024-06-26T07:05:21.4743088Z 25e19d81d9fc: Pull complete 2024-06-26T07:05:21.9624359Z 60156b40fcbe: Pull complete 2024-06-26T07:05:22.1628373Z 7a868239227f: Pull complete 2024-06-26T07:05:22.6103050Z 83911d7c7ecf: Pull complete 2024-06-26T07:05:22.8043271Z b3c5d66c679f: Pull complete 2024-06-26T07:06:01.9112212Z 84b2c7a89982: Pull complete 2024-06-26T07:06:02.1237579Z c1068d03a811: Pull complete 2024-06-26T07:06:02.3323722Z 671b4a301d81: Pull complete 2024-06-26T07:06:04.2971143Z 67760cf5ec46: Pull complete 2024-06-26T07:06:04.3502879Z ddc9740bcea2: Pull complete 2024-06-26T07:06:04.3901298Z fb2e97768560: Pull complete 2024-06-26T07:06:04.4736821Z b599c5a8c7cb: Pull complete 2024-06-26T07:06:04.5222423Z e08e88c13668: Pull complete 2024-06-26T07:06:38.9035066Z 12ba7431ff38: Pull complete 2024-06-26T07:06:39.1571711Z cf7c2cb896cb: Pull complete 2024-06-26T07:06:39.3385206Z 3a778ddfac2a: Pull complete 2024-06-26T07:06:39.8063303Z 236c8f0245f4: Pull complete 2024-06-26T07:06:39.9842064Z 4f4fb700ef54: Pull complete 2024-06-26T07:06:40.1886580Z dda1ce302b9a: Pull complete 2024-06-26T07:06:40.3562624Z e797fe86caef: Pull complete 2024-06-26T07:06:40.6971899Z 967190076fc0: Pull complete 2024-06-26T07:06:40.9338553Z 4fb0a83f0685: Pull complete 2024-06-26T07:06:41.1512719Z 68af2ec4e30a: Pull complete 2024-06-26T07:06:41.3553179Z cc91933e985a: Pull complete 2024-06-26T07:06:41.5565702Z 9f3187916e7d: Pull complete 2024-06-26T07:06:42.6455247Z 526083994857: Pull complete 2024-06-26T07:06:42.8508082Z 05a6f6dcc1c0: Pull complete 2024-06-26T07:06:44.9436641Z 6bc9dc8315b0: Pull complete 2024-06-26T07:06:45.1452890Z 1b0e27c35f07: Pull complete 2024-06-26T07:06:58.5576618Z 95237e7311a1: Pull complete 2024-06-26T07:06:58.7825197Z 789eff6c51aa: Pull complete 2024-06-26T07:06:58.8878201Z de0f8a8c45c6: Pull complete 2024-06-26T07:06:59.7319754Z 2060028f7488: Pull complete 2024-06-26T07:06:59.9317014Z e412c7f3b2de: Pull complete 2024-06-26T07:07:01.0163156Z Digest: sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb 2024-06-26T07:07:01.0542111Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:01.0764629Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:01.0805284Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:07:01.0806223Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:07:01.0814752Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:01.0815256Z env: 2024-06-26T07:07:01.0815544Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:01.0815899Z ##[endgroup] 2024-06-26T07:07:01.0972759Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-06-26T07:07:01.0973320Z with: 2024-06-26T07:07:01.0973657Z driver-version: 550.54.15 2024-06-26T07:07:01.0974011Z env: 2024-06-26T07:07:01.0974290Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:01.0974647Z ##[endgroup] 2024-06-26T07:07:01.1143601Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:07:01.1144164Z with: 2024-06-26T07:07:01.1144448Z timeout_minutes: 10 2024-06-26T07:07:01.1144775Z max_attempts: 3 2024-06-26T07:07:01.1176886Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" 2024-06-26T07:07:01.1209333Z retry_wait_seconds: 10 2024-06-26T07:07:01.1209705Z polling_interval_seconds: 1 2024-06-26T07:07:01.1210081Z warning_on_retry: true 2024-06-26T07:07:01.1210439Z continue_on_error: false 2024-06-26T07:07:01.1210780Z env: 2024-06-26T07:07:01.1211055Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:01.1211411Z DRIVER_VERSION: 550.54.15 2024-06-26T07:07:01.1211764Z ##[endgroup] 2024-06-26T07:07:01.1743754Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-06-26T07:07:01.1744605Z + pre_install_nvidia_driver_amzn2 2024-06-26T07:07:01.1745123Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-06-26T07:07:01.4794141Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:07:01.5178051Z No Match for argument: nvidia-driver-latest-dkms 2024-06-26T07:07:01.5414405Z No Packages marked for removal 2024-06-26T07:07:01.5524331Z + install_nvidia_driver_common 2024-06-26T07:07:01.5527570Z + echo 'Before installing NVIDIA driver' 2024-06-26T07:07:01.5528265Z Before installing NVIDIA driver 2024-06-26T07:07:01.5529537Z + lspci 2024-06-26T07:07:01.5608221Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-06-26T07:07:01.5609302Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T07:07:01.5610431Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-06-26T07:07:01.5611506Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-06-26T07:07:01.5612454Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 2024-06-26T07:07:01.5613238Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T07:07:01.5614065Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1) 2024-06-26T07:07:01.5614870Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-06-26T07:07:01.5615530Z + lsmod 2024-06-26T07:07:01.5622335Z Module Size Used by 2024-06-26T07:07:01.5622959Z ib_core 266240 0 2024-06-26T07:07:01.5623385Z nvidia_modeset 1339392 0 2024-06-26T07:07:01.5623774Z veth 16384 0 2024-06-26T07:07:01.5624153Z nvidia_uvm 4599808 0 2024-06-26T07:07:01.5624658Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:07:01.5625367Z drm 421888 1 nvidia 2024-06-26T07:07:01.5627427Z i2c_core 77824 2 nvidia,drm 2024-06-26T07:07:01.5628154Z backlight 16384 1 nvidia_modeset 2024-06-26T07:07:01.5628735Z xt_conntrack 16384 1 2024-06-26T07:07:01.5629237Z ipt_MASQUERADE 16384 1 2024-06-26T07:07:01.5629714Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T07:07:01.5630368Z nf_conntrack_netlink 49152 0 2024-06-26T07:07:01.5631208Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T07:07:01.5631825Z xfrm_user 45056 1 2024-06-26T07:07:01.5632380Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:07:01.5632902Z iptable_nat 16384 1 2024-06-26T07:07:01.5633357Z nf_conntrack_ipv4 16384 3 2024-06-26T07:07:01.5633829Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T07:07:01.5634375Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T07:07:01.5634914Z nf_nat 32768 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 2024-06-26T07:07:01.5635870Z nf_conntrack 155648 7 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink 2024-06-26T07:07:01.5636948Z xt_addrtype 16384 2 2024-06-26T07:07:01.5637343Z iptable_filter 16384 1 2024-06-26T07:07:01.5637726Z br_netfilter 24576 0 2024-06-26T07:07:01.5638143Z bridge 172032 1 br_netfilter 2024-06-26T07:07:01.5638592Z stp 16384 1 bridge 2024-06-26T07:07:01.5639012Z llc 16384 2 bridge,stp 2024-06-26T07:07:01.5639439Z overlay 86016 0 2024-06-26T07:07:01.5639941Z sunrpc 393216 1 2024-06-26T07:07:01.5640320Z dm_mirror 28672 0 2024-06-26T07:07:01.5640729Z dm_region_hash 20480 1 dm_mirror 2024-06-26T07:07:01.5641267Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T07:07:01.5641789Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T07:07:01.5642253Z dax 69632 1 dm_mod 2024-06-26T07:07:01.5642656Z crc32_pclmul 16384 0 2024-06-26T07:07:01.5643046Z ghash_clmulni_intel 16384 0 2024-06-26T07:07:01.5643435Z pcbc 16384 0 2024-06-26T07:07:01.5643817Z aesni_intel 188416 0 2024-06-26T07:07:01.5644208Z aes_x86_64 20480 1 aesni_intel 2024-06-26T07:07:01.5644663Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:07:01.5645123Z glue_helper 16384 1 aesni_intel 2024-06-26T07:07:01.5645680Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T07:07:01.5646226Z mousedev 24576 0 2024-06-26T07:07:01.5646602Z evdev 20480 3 2024-06-26T07:07:01.5646978Z psmouse 32768 0 2024-06-26T07:07:01.5647345Z button 16384 0 2024-06-26T07:07:01.5647719Z ena 139264 0 2024-06-26T07:07:01.5648096Z ptp 20480 1 ena 2024-06-26T07:07:01.5648499Z pps_core 20480 1 ptp 2024-06-26T07:07:01.5648895Z crc32c_intel 24576 0 2024-06-26T07:07:01.5649267Z autofs4 49152 2 2024-06-26T07:07:01.5649632Z + modinfo nvidia 2024-06-26T07:07:01.5650297Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T07:07:01.5651006Z alias: char-major-195-* 2024-06-26T07:07:01.5651401Z version: 550.54.15 2024-06-26T07:07:01.5651753Z supported: external 2024-06-26T07:07:01.5652305Z license: NVIDIA 2024-06-26T07:07:01.5652699Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:07:01.5653190Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:07:01.5653719Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T07:07:01.5654353Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:07:01.5654862Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:07:01.5655363Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:07:01.5655962Z depends: i2c-core,drm 2024-06-26T07:07:01.5656329Z retpoline: Y 2024-06-26T07:07:01.5656643Z name: nvidia 2024-06-26T07:07:01.5657208Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T07:07:01.5657890Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:07:01.5658541Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:07:01.5659151Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:07:01.5659609Z parm: NVreg_RmLogonRC:int 2024-06-26T07:07:01.5660044Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:07:01.5660506Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:07:01.5660950Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:07:01.5661395Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:07:01.5661925Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:07:01.5662493Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:07:01.5662988Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:07:01.5663426Z parm: NVreg_EnableMSI:int 2024-06-26T07:07:01.5663842Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:07:01.5664380Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:07:01.5664923Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:07:01.5665515Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:07:01.5666076Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:07:01.5666735Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:07:01.5667354Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:07:01.5667975Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:07:01.5668566Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:07:01.5669064Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:07:01.5669614Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:07:01.5670173Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:07:01.5670670Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:07:01.5671139Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:07:01.5671636Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:07:01.5672108Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:07:01.5672559Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:07:01.5673074Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:07:01.5673606Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:07:01.5674082Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:07:01.5674579Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:07:01.5675086Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:07:01.5675579Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:07:01.5676077Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:07:01.5676574Z parm: NVreg_RmMsg:charp 2024-06-26T07:07:01.5677039Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:07:01.5677515Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:07:01.5677982Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:07:01.5678453Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:07:01.5678940Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:07:01.5679423Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:07:01.5679879Z parm: rm_firmware_active:charp 2024-06-26T07:07:01.5680300Z + HAS_NVIDIA_DRIVER=0 2024-06-26T07:07:01.5680677Z ++ command -v nvidia-smi 2024-06-26T07:07:01.5681072Z + '[' -x /usr/bin/nvidia-smi ']' 2024-06-26T07:07:01.5681454Z + set +e 2024-06-26T07:07:01.5681969Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-06-26T07:07:03.6793501Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-06-26T07:07:03.6794144Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:07:03.6794728Z + '[' 0 -ne 0 ']' 2024-06-26T07:07:03.6795433Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-06-26T07:07:03.6795926Z + HAS_NVIDIA_DRIVER=1 2024-06-26T07:07:03.6796660Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-06-26T07:07:03.6797435Z + set -e 2024-06-26T07:07:03.6797740Z + '[' 1 -eq 0 ']' 2024-06-26T07:07:03.6798322Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-06-26T07:07:03.6799022Z + post_install_nvidia_driver_common 2024-06-26T07:07:03.6799424Z + sudo modprobe nvidia 2024-06-26T07:07:03.6893200Z + echo 'After installing NVIDIA driver' 2024-06-26T07:07:03.6895094Z + lspci 2024-06-26T07:07:03.6895577Z After installing NVIDIA driver 2024-06-26T07:07:03.6974180Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-06-26T07:07:03.6976009Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T07:07:03.6977659Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-06-26T07:07:03.6978482Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-06-26T07:07:03.6979238Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 2024-06-26T07:07:03.6980012Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T07:07:03.6980959Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1) 2024-06-26T07:07:03.6981760Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-06-26T07:07:03.6982348Z + lsmod 2024-06-26T07:07:03.6988518Z Module Size Used by 2024-06-26T07:07:03.6989181Z ib_core 266240 0 2024-06-26T07:07:03.6989657Z nvidia_modeset 1339392 0 2024-06-26T07:07:03.6990120Z veth 16384 0 2024-06-26T07:07:03.6990550Z nvidia_uvm 4599808 0 2024-06-26T07:07:03.6991017Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:07:03.6991502Z drm 421888 1 nvidia 2024-06-26T07:07:03.6991937Z i2c_core 77824 2 nvidia,drm 2024-06-26T07:07:03.6992408Z backlight 16384 1 nvidia_modeset 2024-06-26T07:07:03.6992866Z xt_conntrack 16384 1 2024-06-26T07:07:03.6993257Z ipt_MASQUERADE 16384 1 2024-06-26T07:07:03.6993706Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T07:07:03.6994186Z nf_conntrack_netlink 49152 0 2024-06-26T07:07:03.6994644Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T07:07:03.6995145Z xfrm_user 45056 1 2024-06-26T07:07:03.6995535Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:07:03.6995957Z iptable_nat 16384 1 2024-06-26T07:07:03.6996476Z nf_conntrack_ipv4 16384 3 2024-06-26T07:07:03.6997039Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T07:07:03.6997585Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T07:07:03.6998287Z nf_nat 32768 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 2024-06-26T07:07:03.6999601Z nf_conntrack 155648 7 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink 2024-06-26T07:07:03.7000732Z xt_addrtype 16384 2 2024-06-26T07:07:03.7001115Z iptable_filter 16384 1 2024-06-26T07:07:03.7001509Z br_netfilter 24576 0 2024-06-26T07:07:03.7001924Z bridge 172032 1 br_netfilter 2024-06-26T07:07:03.7002371Z stp 16384 1 bridge 2024-06-26T07:07:03.7002790Z llc 16384 2 bridge,stp 2024-06-26T07:07:03.7003211Z overlay 86016 0 2024-06-26T07:07:03.7003583Z sunrpc 393216 1 2024-06-26T07:07:03.7003953Z dm_mirror 28672 0 2024-06-26T07:07:03.7004341Z dm_region_hash 20480 1 dm_mirror 2024-06-26T07:07:03.7004833Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T07:07:03.7005458Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T07:07:03.7005918Z dax 69632 1 dm_mod 2024-06-26T07:07:03.7006505Z crc32_pclmul 16384 0 2024-06-26T07:07:03.7006897Z ghash_clmulni_intel 16384 0 2024-06-26T07:07:03.7007287Z pcbc 16384 0 2024-06-26T07:07:03.7007701Z aesni_intel 188416 0 2024-06-26T07:07:03.7008105Z aes_x86_64 20480 1 aesni_intel 2024-06-26T07:07:03.7008560Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:07:03.7009015Z glue_helper 16384 1 aesni_intel 2024-06-26T07:07:03.7009575Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T07:07:03.7010119Z mousedev 24576 0 2024-06-26T07:07:03.7010489Z evdev 20480 3 2024-06-26T07:07:03.7010858Z psmouse 32768 0 2024-06-26T07:07:03.7011224Z button 16384 0 2024-06-26T07:07:03.7011595Z ena 139264 0 2024-06-26T07:07:03.7011971Z ptp 20480 1 ena 2024-06-26T07:07:03.7012677Z pps_core 20480 1 ptp 2024-06-26T07:07:03.7013082Z crc32c_intel 24576 0 2024-06-26T07:07:03.7013553Z autofs4 49152 2 2024-06-26T07:07:03.7013920Z + modinfo nvidia 2024-06-26T07:07:03.7014572Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T07:07:03.7015374Z alias: char-major-195-* 2024-06-26T07:07:03.7015764Z version: 550.54.15 2024-06-26T07:07:03.7016116Z supported: external 2024-06-26T07:07:03.7016453Z license: NVIDIA 2024-06-26T07:07:03.7016847Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:07:03.7017343Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:07:03.7017807Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T07:07:03.7018266Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:07:03.7018767Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:07:03.7019263Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:07:03.7019746Z depends: i2c-core,drm 2024-06-26T07:07:03.7020100Z retpoline: Y 2024-06-26T07:07:03.7020416Z name: nvidia 2024-06-26T07:07:03.7020973Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T07:07:03.7021631Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:07:03.7022297Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:07:03.7022927Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:07:03.7023376Z parm: NVreg_RmLogonRC:int 2024-06-26T07:07:03.7023810Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:07:03.7024271Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:07:03.7024711Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:07:03.7025152Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:07:03.7025683Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:07:03.7026246Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:07:03.7026728Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:07:03.7027164Z parm: NVreg_EnableMSI:int 2024-06-26T07:07:03.7027579Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:07:03.7028044Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:07:03.7028578Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:07:03.7029171Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:07:03.7029732Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:07:03.7030345Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:07:03.7030949Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:07:03.7031564Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:07:03.7032154Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:07:03.7032647Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:07:03.7033185Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:07:03.7033737Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:07:03.7034317Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:07:03.7034788Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:07:03.7035290Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:07:03.7035776Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:07:03.7036221Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:07:03.7036730Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:07:03.7037259Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:07:03.7037740Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:07:03.7038223Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:07:03.7038723Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:07:03.7039214Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:07:03.7039719Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:07:03.7040200Z parm: NVreg_RmMsg:charp 2024-06-26T07:07:03.7040618Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:07:03.7041101Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:07:03.7041568Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:07:03.7042043Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:07:03.7042582Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:07:03.7043068Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:07:03.7043519Z parm: rm_firmware_active:charp 2024-06-26T07:07:03.7043918Z + set +e 2024-06-26T07:07:03.7044228Z + nvidia-smi 2024-06-26T07:07:05.2612960Z Wed Jun 26 07:07:05 2024 2024-06-26T07:07:05.2613828Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:07:05.2614654Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-06-26T07:07:05.2615433Z |-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:07:05.2616255Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-06-26T07:07:05.2617150Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-06-26T07:07:05.2617851Z | | | MIG M. | 2024-06-26T07:07:05.2618406Z |=========================================+========================+======================| 2024-06-26T07:07:05.2752795Z | 0 NVIDIA A10G Off | 00000000:00:1E.0 Off | 0 | 2024-06-26T07:07:05.2753527Z | 0% 33C P0 59W / 300W | 0MiB / 23028MiB | 0% Default | 2024-06-26T07:07:05.2754165Z | | | N/A | 2024-06-26T07:07:05.2754832Z +-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:07:05.2755524Z 2024-06-26T07:07:05.2756177Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:07:05.2756807Z | Processes: | 2024-06-26T07:07:05.2757527Z | GPU GI CI PID Type Process name GPU Memory | 2024-06-26T07:07:05.2771731Z | ID ID Usage | 2024-06-26T07:07:05.2772616Z |=========================================================================================| 2024-06-26T07:07:05.2773293Z | No running processes found | 2024-06-26T07:07:05.2774166Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:07:05.9026364Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-06-26T07:07:07.4692934Z NVIDIA A10G 2024-06-26T07:07:07.9057445Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:07:07.9058554Z + '[' 0 -eq 0 ']' 2024-06-26T07:07:07.9059252Z + echo 'INFO: Ignoring allowed status 0' 2024-06-26T07:07:07.9059689Z + set -e 2024-06-26T07:07:07.9059994Z INFO: Ignoring allowed status 0 2024-06-26T07:07:07.9061176Z == Installing nvidia container toolkit for amzn2 == 2024-06-26T07:07:07.9064165Z + sudo yum install -y yum-utils 2024-06-26T07:07:08.2060783Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:07:09.7154956Z Package yum-utils-1.1.31-46.amzn2.0.1.noarch already installed and latest version 2024-06-26T07:07:09.7155651Z Nothing to do 2024-06-26T07:07:09.8658240Z + [[ amzn2 == \a\m\z\n\2\0\2\3 ]] 2024-06-26T07:07:09.8659057Z + YUM_REPO_URL=https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:07:09.8660168Z + sudo yum-config-manager --add-repo https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:07:10.1870175Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:07:10.2118203Z adding repo from: https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:07:10.2119841Z grabbing file https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T07:07:10.2121025Z repo saved to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T07:07:10.2230919Z + sudo yum install -y nvidia-docker2 2024-06-26T07:07:10.5246899Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:07:11.9737509Z Package nvidia-docker2-2.13.0-1.noarch already installed and latest version 2024-06-26T07:07:11.9738357Z Nothing to do 2024-06-26T07:07:12.1204830Z + sudo systemctl restart docker 2024-06-26T07:07:56.2217533Z Command completed after 1 attempt(s). 2024-06-26T07:07:56.2276169Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:07:56.2276959Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:07:56.2277676Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-06-26T07:07:56.2278368Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:56.2286245Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:56.2286767Z env: 2024-06-26T07:07:56.2287067Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:56.2287542Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:07:56.2288038Z ##[endgroup] 2024-06-26T07:07:56.4447344Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T07:07:56.4609013Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.7/site-packages (5.9.1) 2024-06-26T07:07:56.4706532Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.7/site-packages (11.525.84) 2024-06-26T07:07:56.5800177Z Prepare all required actions 2024-06-26T07:07:56.5800914Z Getting action download info 2024-06-26T07:07:56.6824530Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-06-26T07:07:56.8339871Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-06-26T07:07:56.9641540Z ##[group]Run ./.github/actions/download-build-artifacts 2024-06-26T07:07:56.9642041Z with: 2024-06-26T07:07:56.9642373Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:07:56.9642842Z s3-bucket: gha-artifacts 2024-06-26T07:07:56.9643185Z env: 2024-06-26T07:07:56.9643464Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:56.9643911Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:07:56.9644398Z ##[endgroup] 2024-06-26T07:07:56.9676342Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:07:56.9676790Z with: 2024-06-26T07:07:56.9677175Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:07:56.9677651Z s3-bucket: gha-artifacts 2024-06-26T07:07:56.9678005Z region: us-east-1 2024-06-26T07:07:56.9678311Z env: 2024-06-26T07:07:56.9678598Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:56.9679048Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:07:56.9679545Z ##[endgroup] 2024-06-26T07:07:57.3816365Z (node:26288) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:07:57.3817129Z 2024-06-26T07:07:57.3817407Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:07:57.3818144Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:07:57.3819136Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:07:57.4630791Z Found 1 objects with prefix pytorch/pytorch/9673646808/linux-focal-cuda12.1-py3.10-gcc9-sm86/ 2024-06-26T07:07:57.4631894Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:08:22.0824363Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:08:22.0830589Z Artifact download has finished successfully 2024-06-26T07:08:22.0965748Z ##[group]Run unzip -o artifacts.zip 2024-06-26T07:08:22.0966198Z unzip -o artifacts.zip 2024-06-26T07:08:22.0974275Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:08:22.0974917Z env: 2024-06-26T07:08:22.0975203Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:22.0975675Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:22.0976175Z ##[endgroup] 2024-06-26T07:08:22.1002790Z Archive: artifacts.zip 2024-06-26T07:08:22.1004416Z creating: dist/ 2024-06-26T07:08:24.0679085Z inflating: dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:08:24.0679771Z creating: build/custom_test_artifacts/ 2024-06-26T07:08:24.0682387Z creating: build/custom_test_artifacts/custom-op-build/ 2024-06-26T07:08:24.0683152Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-06-26T07:08:24.0684001Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:08:24.0686018Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:08:24.0686977Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-06-26T07:08:24.0687950Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:08:24.0688949Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:08:24.0689937Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:08:24.0691054Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:08:24.0692617Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:08:24.0693740Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:08:24.0694747Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:08:24.0695921Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:08:24.0697112Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:08:24.0698272Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:08:24.0699425Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:08:24.0700644Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:08:24.0702147Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:08:24.0703243Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:08:24.0704267Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:08:24.0740278Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:08:24.0779286Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:08:24.0782147Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:08:24.0822534Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:08:24.0824049Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:08:24.0825518Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:08:24.0827020Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:08:24.0828539Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:08:24.0830131Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:08:24.0831552Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:08:24.0832971Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:08:24.0834356Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:08:24.0835671Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:08:24.0836944Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:08:24.0838193Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:08:24.0839464Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:08:24.0840687Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:08:24.0841924Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:08:24.0891898Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:08:24.0950214Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:08:24.0951443Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:08:24.0952482Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:08:24.0953387Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:08:24.0954299Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-06-26T07:08:24.0955247Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-06-26T07:08:24.0956448Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-06-26T07:08:24.0957632Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-06-26T07:08:24.0958757Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-06-26T07:08:24.0959800Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-06-26T07:08:24.0960890Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-06-26T07:08:24.0961987Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-06-26T07:08:24.0963095Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-06-26T07:08:24.0964183Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-06-26T07:08:24.0965253Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-06-26T07:08:24.0978296Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-06-26T07:08:24.1105826Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-06-26T07:08:24.1106856Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-06-26T07:08:24.1108040Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-06-26T07:08:24.1109261Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-06-26T07:08:24.1110564Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-06-26T07:08:24.1111672Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-06-26T07:08:24.1112864Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-06-26T07:08:24.1114012Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-06-26T07:08:24.1115164Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-06-26T07:08:24.1116318Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-06-26T07:08:24.1117444Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-06-26T07:08:24.1131723Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-06-26T07:08:24.1211048Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-06-26T07:08:24.1212635Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:08:24.1213941Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:08:24.1215061Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-06-26T07:08:24.1216001Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-06-26T07:08:24.1216920Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-06-26T07:08:24.1217909Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-06-26T07:08:24.1218774Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-06-26T07:08:24.1219573Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-06-26T07:08:24.1220384Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-06-26T07:08:24.1326671Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-06-26T07:08:24.1387045Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-06-26T07:08:24.1387810Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-06-26T07:08:24.1388520Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-06-26T07:08:24.1389333Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:08:24.1394542Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:08:24.1395490Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-06-26T07:08:24.1396434Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:08:24.1397505Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:08:24.1398487Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:08:24.1399594Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:08:24.1400707Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:08:24.1401798Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:08:24.1402847Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:08:24.1404024Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:08:24.1405254Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:08:24.1406521Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:08:24.1407769Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:08:24.1408923Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:08:24.1410092Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:08:24.1411134Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:08:24.1412405Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:08:24.1447284Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:08:24.1483928Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:08:24.1485402Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:08:24.1527777Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:08:24.1529214Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:08:24.1530799Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:08:24.1532449Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:08:24.1534043Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:08:24.1535447Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:08:24.1536844Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:08:24.1538235Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:08:24.1539603Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:08:24.1540895Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:08:24.1542144Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:08:24.1543374Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:08:24.1544619Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:08:24.1545815Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:08:24.1547040Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:08:24.1595258Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:08:24.1652521Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:08:24.1653790Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:08:24.1669974Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:08:24.1670924Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:08:24.1671966Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-06-26T07:08:24.1672977Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-06-26T07:08:24.1674070Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-06-26T07:08:24.1675284Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-06-26T07:08:24.1676450Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-06-26T07:08:24.1677537Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-06-26T07:08:24.1678661Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-06-26T07:08:24.1679786Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-06-26T07:08:24.1680922Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-06-26T07:08:24.1682046Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-06-26T07:08:24.1683203Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-06-26T07:08:24.1684395Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-06-26T07:08:24.1738592Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-06-26T07:08:24.1739798Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:08:24.1740890Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:08:24.1741867Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-06-26T07:08:24.1742786Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-06-26T07:08:24.1743683Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-06-26T07:08:24.1744594Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-06-26T07:08:24.1745441Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-06-26T07:08:24.1746225Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-06-26T07:08:24.1747041Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-06-26T07:08:24.1792380Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-06-26T07:08:24.1793184Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-06-26T07:08:24.1793946Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-06-26T07:08:24.1794840Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:08:24.1799762Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:08:24.1800758Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-06-26T07:08:24.1801757Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:08:24.1802876Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:08:24.1803921Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:08:24.1805090Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:08:24.1806264Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:08:24.1807323Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:08:24.1808488Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:08:24.1809713Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:08:24.1810942Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:08:24.1812382Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:08:24.1813728Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:08:24.1814961Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:08:24.1816195Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:08:24.1817308Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:08:24.1818376Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:08:24.1851767Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:08:24.1889274Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:08:24.1890909Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:08:24.1933457Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:08:24.1935038Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:08:24.1936555Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:08:24.1938103Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:08:24.1939591Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:08:24.1941074Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:08:24.1942563Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:08:24.1944069Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:08:24.1945482Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:08:24.1946841Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:08:24.1948151Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:08:24.1949442Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:08:24.1950746Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:08:24.1952011Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:08:24.1953304Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:08:24.2000421Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:08:24.2058385Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:08:24.2060890Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:08:24.2062589Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:08:24.2063526Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:08:24.2064507Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-06-26T07:08:24.2065529Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-06-26T07:08:24.2066672Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-06-26T07:08:24.2067950Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-06-26T07:08:24.2069194Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-06-26T07:08:24.2070347Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-06-26T07:08:24.2071540Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-06-26T07:08:24.2072849Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-06-26T07:08:24.2074091Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-06-26T07:08:24.2075287Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-06-26T07:08:24.2076482Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-06-26T07:08:24.2077750Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-06-26T07:08:24.2186112Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-06-26T07:08:24.2187298Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-06-26T07:08:24.2188500Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-06-26T07:08:24.2189838Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-06-26T07:08:24.2191118Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-06-26T07:08:24.2192330Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-06-26T07:08:24.2193579Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-06-26T07:08:24.2194837Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-06-26T07:08:24.2196099Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-06-26T07:08:24.2197358Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-06-26T07:08:24.2198594Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-06-26T07:08:24.2210693Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-06-26T07:08:24.2263469Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-06-26T07:08:24.2264906Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:08:24.2266064Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:08:24.2267107Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-06-26T07:08:24.2268085Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-06-26T07:08:24.2269059Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-06-26T07:08:24.2270040Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-06-26T07:08:24.2270962Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-06-26T07:08:24.2271808Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-06-26T07:08:24.2272692Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-06-26T07:08:24.2368524Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-06-26T07:08:24.2409386Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-06-26T07:08:24.2410028Z creating: build/lib/ 2024-06-26T07:08:24.2494424Z inflating: build/lib/libprotobuf-lite.a 2024-06-26T07:08:24.2931495Z inflating: build/lib/libprotobuf.a 2024-06-26T07:08:24.2941076Z inflating: build/lib/libpthreadpool.a 2024-06-26T07:08:24.2948752Z inflating: build/lib/libcpuinfo.a 2024-06-26T07:08:24.2956253Z inflating: build/lib/libcpuinfo_internals.a 2024-06-26T07:08:24.2957227Z inflating: build/lib/libclog.a 2024-06-26T07:08:24.2974362Z inflating: build/lib/libnnpack.a 2024-06-26T07:08:24.2976248Z inflating: build/lib/libnnpack_reference_layers.a 2024-06-26T07:08:24.3038739Z inflating: build/lib/libgtest.a 2024-06-26T07:08:24.3110108Z inflating: build/lib/libbenchmark.a 2024-06-26T07:08:24.3171971Z inflating: build/lib/libasmjit.a 2024-06-26T07:08:24.3178675Z inflating: build/lib/libittnotify.a 2024-06-26T07:08:24.3205062Z inflating: build/lib/libtensorpipe_uv.a 2024-06-26T07:08:24.3326784Z inflating: build/lib/libgloo.a 2024-06-26T07:08:24.3347906Z inflating: build/lib/libfmt.a 2024-06-26T07:08:24.3436786Z inflating: build/lib/libc10.so 2024-06-26T07:08:24.3437702Z inflating: build/lib/libcaffe2_nvrtc.so 2024-06-26T07:08:24.3438191Z inflating: build/lib/libfoxi_loader.a 2024-06-26T07:08:24.3439639Z inflating: build/lib/libtorch_global_deps.so 2024-06-26T07:08:24.3458453Z inflating: build/lib/libpytorch_qnnpack.a 2024-06-26T07:08:24.3475540Z inflating: build/lib/libgmock.a 2024-06-26T07:08:24.3476350Z inflating: build/lib/libgtest_main.a 2024-06-26T07:08:24.3476929Z inflating: build/lib/libbenchmark_main.a 2024-06-26T07:08:24.3960287Z inflating: build/lib/libprotoc.a 2024-06-26T07:08:24.4328937Z inflating: build/lib/libgloo_cuda.a 2024-06-26T07:08:25.4036914Z inflating: build/lib/libdnnl.a 2024-06-26T07:08:25.4585547Z inflating: build/lib/libtensorpipe.a 2024-06-26T07:08:25.4638779Z inflating: build/lib/libc10_cuda.so 2024-06-26T07:08:25.4639911Z inflating: build/lib/libgmock_main.a 2024-06-26T07:08:25.5839325Z inflating: build/lib/libfbgemm.a 2024-06-26T07:08:25.6084550Z inflating: build/lib/libtensorpipe_cuda.a 2024-06-26T07:08:25.6581431Z inflating: build/lib/libkineto.a 2024-06-26T07:08:25.6759480Z inflating: build/lib/libXNNPACK.a 2024-06-26T07:08:25.6799250Z inflating: build/lib/libonnx_proto.a 2024-06-26T07:08:25.7463162Z inflating: build/lib/libonnx.a 2024-06-26T07:08:28.0718852Z inflating: build/lib/libtorch_cpu.so 2024-06-26T07:08:28.0722695Z inflating: build/lib/libunbox_lib.a 2024-06-26T07:08:28.0726503Z inflating: build/lib/libshm.so 2024-06-26T07:08:30.0551199Z inflating: build/lib/libtorch_cuda.so 2024-06-26T07:08:30.0552041Z inflating: build/lib/libtorch.so 2024-06-26T07:08:30.0553900Z inflating: build/lib/libc10d_cuda_test.so 2024-06-26T07:08:30.8474824Z inflating: build/lib/libtorch_cuda_linalg.so 2024-06-26T07:08:31.0365172Z inflating: build/lib/libtorch_python.so 2024-06-26T07:08:31.0431220Z inflating: build/lib/libtorchbind_test.so 2024-06-26T07:08:31.0450953Z inflating: build/lib/libjitbackend_test.so 2024-06-26T07:08:31.0475570Z inflating: build/lib/libbackend_with_compiler.so 2024-06-26T07:08:31.0508833Z inflating: build/lib/libnnapi_backend.so 2024-06-26T07:08:31.0509408Z creating: build/bin/ 2024-06-26T07:08:31.0558969Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-06-26T07:08:31.0608379Z inflating: build/bin/c10_DeviceGuard_test 2024-06-26T07:08:31.0656681Z inflating: build/bin/c10_Device_test 2024-06-26T07:08:31.0713701Z inflating: build/bin/c10_DispatchKeySet_test 2024-06-26T07:08:31.0765465Z inflating: build/bin/c10_Scalar_test 2024-06-26T07:08:31.0812954Z inflating: build/bin/c10_StreamGuard_test 2024-06-26T07:08:31.0862144Z inflating: build/bin/c10_SymInt_test 2024-06-26T07:08:31.0914859Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-06-26T07:08:31.0969356Z inflating: build/bin/c10_InlineStreamGuard_test 2024-06-26T07:08:31.1023922Z inflating: build/bin/c10_SizesAndStrides_test 2024-06-26T07:08:31.1092842Z inflating: build/bin/c10_cow_test 2024-06-26T07:08:31.1143423Z inflating: build/bin/c10_Bitset_test 2024-06-26T07:08:31.1190828Z inflating: build/bin/c10_ConstexprCrc_test 2024-06-26T07:08:31.1239030Z inflating: build/bin/c10_DeadlockDetection_test 2024-06-26T07:08:31.1287994Z inflating: build/bin/c10_Half_test 2024-06-26T07:08:31.1342315Z inflating: build/bin/c10_LeftRight_test 2024-06-26T07:08:31.1395647Z inflating: build/bin/c10_Metaprogramming_test 2024-06-26T07:08:31.1444284Z inflating: build/bin/c10_Synchronized_test 2024-06-26T07:08:31.1498409Z inflating: build/bin/c10_ThreadLocal_test 2024-06-26T07:08:31.1548091Z inflating: build/bin/c10_TypeIndex_test 2024-06-26T07:08:31.1597594Z inflating: build/bin/c10_TypeList_test 2024-06-26T07:08:31.1645526Z inflating: build/bin/c10_TypeTraits_test 2024-06-26T07:08:31.1695905Z inflating: build/bin/c10_accumulate_test 2024-06-26T07:08:31.1749570Z inflating: build/bin/c10_bfloat16_test 2024-06-26T07:08:31.1798303Z inflating: build/bin/c10_bit_cast_test 2024-06-26T07:08:31.1853088Z inflating: build/bin/c10_complex_math_test 2024-06-26T07:08:31.1907547Z inflating: build/bin/c10_complex_test 2024-06-26T07:08:31.1958534Z inflating: build/bin/c10_exception_test 2024-06-26T07:08:31.2007302Z inflating: build/bin/c10_flags_test 2024-06-26T07:08:31.2055863Z inflating: build/bin/c10_generic_math_test 2024-06-26T07:08:31.2214683Z inflating: build/bin/c10_intrusive_ptr_test 2024-06-26T07:08:31.2263624Z inflating: build/bin/c10_irange_test 2024-06-26T07:08:31.2315233Z inflating: build/bin/c10_lazy_test 2024-06-26T07:08:31.2370429Z inflating: build/bin/c10_logging_test 2024-06-26T07:08:31.2442854Z inflating: build/bin/c10_optional_test 2024-06-26T07:08:31.2503689Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-06-26T07:08:31.2555701Z inflating: build/bin/c10_registry_test 2024-06-26T07:08:31.2701488Z inflating: build/bin/c10_small_vector_test 2024-06-26T07:08:31.2751725Z inflating: build/bin/c10_ssize_test 2024-06-26T07:08:31.2802158Z inflating: build/bin/c10_string_util_test 2024-06-26T07:08:31.2859237Z inflating: build/bin/c10_string_view_test 2024-06-26T07:08:31.2907904Z inflating: build/bin/c10_tempfile_test 2024-06-26T07:08:31.2954858Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-06-26T07:08:31.3009136Z inflating: build/bin/c10_typeid_test 2024-06-26T07:08:31.3442855Z inflating: build/bin/protoc-3.13.0.0 2024-06-26T07:08:31.3876232Z inflating: build/bin/protoc 2024-06-26T07:08:31.3927440Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-06-26T07:08:31.3978740Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-06-26T07:08:31.4029852Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-06-26T07:08:31.4080196Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-06-26T07:08:31.4131493Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-06-26T07:08:31.4182843Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-06-26T07:08:31.4230252Z inflating: build/bin/c10_cuda_CUDATest 2024-06-26T07:08:31.4281508Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-06-26T07:08:31.4601463Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-06-26T07:08:31.4933809Z inflating: build/bin/vec_test_all_types_AVX512 2024-06-26T07:08:31.5280153Z inflating: build/bin/vec_test_all_types_AVX2 2024-06-26T07:08:31.5331811Z inflating: build/bin/FileStoreTest 2024-06-26T07:08:31.5385473Z inflating: build/bin/TCPStoreTest 2024-06-26T07:08:31.5437134Z inflating: build/bin/HashStoreTest 2024-06-26T07:08:31.5450058Z inflating: build/bin/ProcessGroupMPITest 2024-06-26T07:08:31.5503404Z inflating: build/bin/test_edge_op_registration 2024-06-26T07:08:31.5507510Z inflating: build/bin/torch_shm_manager 2024-06-26T07:08:31.5510182Z inflating: build/bin/example_allreduce 2024-06-26T07:08:31.5564377Z inflating: build/bin/test_dist_autograd 2024-06-26T07:08:31.5631510Z inflating: build/bin/test_cpp_rpc 2024-06-26T07:08:31.5633592Z inflating: build/bin/parallel_benchmark 2024-06-26T07:08:31.5698690Z inflating: build/bin/test_mobile_nnc 2024-06-26T07:08:31.5707256Z inflating: build/bin/aot_model_compiler_test 2024-06-26T07:08:31.6041002Z inflating: build/bin/test_lazy 2024-06-26T07:08:31.7186053Z inflating: build/bin/test_api 2024-06-26T07:08:31.7257458Z inflating: build/bin/Dict_test 2024-06-26T07:08:31.7307727Z inflating: build/bin/Dimname_test 2024-06-26T07:08:31.7370640Z inflating: build/bin/MaybeOwned_test 2024-06-26T07:08:31.7426176Z inflating: build/bin/NamedTensor_test 2024-06-26T07:08:31.7483290Z inflating: build/bin/apply_utils_test 2024-06-26T07:08:31.7540964Z inflating: build/bin/atest 2024-06-26T07:08:31.7601732Z inflating: build/bin/basic 2024-06-26T07:08:31.7654754Z inflating: build/bin/broadcast_test 2024-06-26T07:08:31.7703629Z inflating: build/bin/cpu_allocator_test 2024-06-26T07:08:31.7759770Z inflating: build/bin/cpu_generator_test 2024-06-26T07:08:31.7811016Z inflating: build/bin/cpu_profiling_allocator_test 2024-06-26T07:08:31.7901606Z inflating: build/bin/cpu_rng_test 2024-06-26T07:08:31.7951095Z inflating: build/bin/dispatch_key_set_test 2024-06-26T07:08:31.7999718Z inflating: build/bin/dlconvertor_test 2024-06-26T07:08:31.8056398Z inflating: build/bin/extension_backend_test 2024-06-26T07:08:31.8110151Z inflating: build/bin/half_test 2024-06-26T07:08:31.8202697Z inflating: build/bin/ivalue_test 2024-06-26T07:08:31.8251286Z inflating: build/bin/lazy_tensor_test 2024-06-26T07:08:31.8303629Z inflating: build/bin/math_kernel_test 2024-06-26T07:08:31.8356427Z inflating: build/bin/memory_format_test 2024-06-26T07:08:31.8407294Z inflating: build/bin/memory_overlapping_test 2024-06-26T07:08:31.8458927Z inflating: build/bin/mobile_memory_cleanup 2024-06-26T07:08:31.8512939Z inflating: build/bin/native_test 2024-06-26T07:08:31.8561799Z inflating: build/bin/operator_name_test 2024-06-26T07:08:31.8611983Z inflating: build/bin/operators_test 2024-06-26T07:08:31.8661984Z inflating: build/bin/packedtensoraccessor_test 2024-06-26T07:08:31.8727332Z inflating: build/bin/pow_test 2024-06-26T07:08:31.8782865Z inflating: build/bin/quantized_test 2024-06-26T07:08:31.8831433Z inflating: build/bin/reduce_ops_test 2024-06-26T07:08:31.8881064Z inflating: build/bin/reportMemoryUsage_test 2024-06-26T07:08:31.8935909Z inflating: build/bin/scalar_tensor_test 2024-06-26T07:08:31.8992981Z inflating: build/bin/scalar_test 2024-06-26T07:08:31.9042971Z inflating: build/bin/StorageUtils_test 2024-06-26T07:08:31.9093702Z inflating: build/bin/stride_properties_test 2024-06-26T07:08:31.9170191Z inflating: build/bin/tensor_iterator_test 2024-06-26T07:08:31.9222259Z inflating: build/bin/test_parallel 2024-06-26T07:08:31.9225034Z inflating: build/bin/thread_init_test 2024-06-26T07:08:31.9278823Z inflating: build/bin/type_ptr_test 2024-06-26T07:08:31.9336938Z inflating: build/bin/type_test 2024-06-26T07:08:31.9387626Z inflating: build/bin/undefined_tensor_test 2024-06-26T07:08:31.9388997Z inflating: build/bin/verify_api_visibility 2024-06-26T07:08:31.9456428Z inflating: build/bin/legacy_vmap_test 2024-06-26T07:08:31.9506429Z inflating: build/bin/weakref_test 2024-06-26T07:08:31.9556071Z inflating: build/bin/wrapdim_test 2024-06-26T07:08:31.9606099Z inflating: build/bin/xla_tensor_test 2024-06-26T07:08:31.9664308Z inflating: build/bin/IListRef_test 2024-06-26T07:08:31.9767318Z inflating: build/bin/List_test 2024-06-26T07:08:31.9831384Z inflating: build/bin/KernelFunction_test 2024-06-26T07:08:31.9949266Z inflating: build/bin/kernel_function_legacy_test 2024-06-26T07:08:32.0042950Z inflating: build/bin/kernel_function_test 2024-06-26T07:08:32.0166827Z inflating: build/bin/kernel_lambda_legacy_test 2024-06-26T07:08:32.0266891Z inflating: build/bin/kernel_lambda_test 2024-06-26T07:08:32.0326016Z inflating: build/bin/kernel_stackbased_test 2024-06-26T07:08:32.0418616Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-06-26T07:08:32.0467904Z inflating: build/bin/CppSignature_test 2024-06-26T07:08:32.0522410Z inflating: build/bin/backend_fallback_test 2024-06-26T07:08:32.0569111Z inflating: build/bin/op_allowlist_test 2024-06-26T07:08:32.0631155Z inflating: build/bin/inline_container_test 2024-06-26T07:08:32.0923059Z inflating: build/bin/op_registration_test 2024-06-26T07:08:32.0974664Z inflating: build/bin/cuda_apply_test 2024-06-26T07:08:32.1023971Z inflating: build/bin/cuda_allocator_test 2024-06-26T07:08:32.1081305Z inflating: build/bin/cuda_atomic_ops_test 2024-06-26T07:08:32.1134581Z inflating: build/bin/cuda_caching_host_allocator_test 2024-06-26T07:08:32.1201401Z inflating: build/bin/cuda_complex_math_test 2024-06-26T07:08:32.1258709Z inflating: build/bin/cuda_complex_test 2024-06-26T07:08:32.1306623Z inflating: build/bin/cuda_device_test 2024-06-26T07:08:32.1362004Z inflating: build/bin/cuda_cub_test 2024-06-26T07:08:32.1411313Z inflating: build/bin/cuda_dlconvertor_test 2024-06-26T07:08:32.1474198Z inflating: build/bin/cuda_distributions_test 2024-06-26T07:08:32.1529073Z inflating: build/bin/cuda_generator_test 2024-06-26T07:08:32.1577211Z inflating: build/bin/cuda_half_test 2024-06-26T07:08:32.1626801Z inflating: build/bin/cuda_integer_divider_test 2024-06-26T07:08:32.1674361Z inflating: build/bin/cuda_optional_test 2024-06-26T07:08:32.1724335Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-06-26T07:08:32.1775324Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-06-26T07:08:32.1823470Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-06-26T07:08:32.1882118Z inflating: build/bin/cuda_stream_test 2024-06-26T07:08:32.1929983Z inflating: build/bin/cuda_cudnn_test 2024-06-26T07:08:32.1980358Z inflating: build/bin/cuda_vectorized_test 2024-06-26T07:08:32.1994672Z inflating: build/bin/tutorial_tensorexpr 2024-06-26T07:08:32.2058361Z inflating: build/bin/ProcessGroupGlooTest 2024-06-26T07:08:32.2114033Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-06-26T07:08:32.2175590Z inflating: build/bin/ProcessGroupNCCLTest 2024-06-26T07:08:32.2235874Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-06-26T07:08:32.3040173Z inflating: build/bin/test_tensorexpr 2024-06-26T07:08:32.3600112Z inflating: build/bin/test_jit 2024-06-26T07:08:32.3601058Z creating: .additional_ci_files/ 2024-06-26T07:08:32.3652706Z inflating: .additional_ci_files/test-times.json 2024-06-26T07:08:32.3863223Z inflating: .additional_ci_files/test-class-times.json 2024-06-26T07:08:32.3888751Z ##[group]Run rm artifacts.zip 2024-06-26T07:08:32.3889147Z rm artifacts.zip 2024-06-26T07:08:32.3897127Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:08:32.3897642Z env: 2024-06-26T07:08:32.3897937Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:32.3898402Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:32.3898899Z ##[endgroup] 2024-06-26T07:08:32.7356870Z ##[group]Run df -H 2024-06-26T07:08:32.7357189Z df -H 2024-06-26T07:08:32.7364728Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:08:32.7365244Z env: 2024-06-26T07:08:32.7365521Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:32.7366000Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:32.7366503Z ##[endgroup] 2024-06-26T07:08:32.7392083Z Filesystem Size Used Avail Use% Mounted on 2024-06-26T07:08:32.7392833Z devtmpfs 34G 0 34G 0% /dev 2024-06-26T07:08:32.7393433Z tmpfs 34G 0 34G 0% /dev/shm 2024-06-26T07:08:32.7394001Z tmpfs 34G 455k 34G 1% /run 2024-06-26T07:08:32.7394501Z tmpfs 34G 0 34G 0% /sys/fs/cgroup 2024-06-26T07:08:32.7394989Z /dev/nvme0n1p1 162G 66G 96G 41% / 2024-06-26T07:08:32.7433262Z Prepare all required actions 2024-06-26T07:08:32.7433701Z Getting action download info 2024-06-26T07:08:32.8875583Z ##[group]Run ./.github/actions/download-td-artifacts 2024-06-26T07:08:32.8876055Z with: 2024-06-26T07:08:32.8876308Z env: 2024-06-26T07:08:32.8876583Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:32.8877046Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:32.8877529Z ##[endgroup] 2024-06-26T07:08:32.8909148Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:08:32.8909599Z with: 2024-06-26T07:08:32.8909867Z name: td_results 2024-06-26T07:08:32.8910190Z s3-bucket: gha-artifacts 2024-06-26T07:08:32.8910543Z region: us-east-1 2024-06-26T07:08:32.8910843Z env: 2024-06-26T07:08:32.8911112Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:32.8911567Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:32.8912157Z ##[endgroup] 2024-06-26T07:08:33.3054654Z (node:26315) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:08:33.3055554Z 2024-06-26T07:08:33.3055875Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:08:33.3056730Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:08:33.3057601Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:08:33.3760348Z Found 0 objects with prefix pytorch/pytorch/9673646808/td_results/ 2024-06-26T07:08:33.3767077Z Artifact download has finished successfully 2024-06-26T07:08:33.3902748Z ##[group]Run mkdir -p .additional_ci_files 2024-06-26T07:08:33.3903242Z mkdir -p .additional_ci_files 2024-06-26T07:08:33.3903817Z mv td_results.json .additional_ci_files/td_results.json 2024-06-26T07:08:33.3911640Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:08:33.3912149Z env: 2024-06-26T07:08:33.3912431Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:33.3912896Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:33.3913387Z ##[endgroup] 2024-06-26T07:08:33.3947411Z mv: cannot stat 'td_results.json': No such file or directory 2024-06-26T07:08:33.3957416Z ##[error]Process completed with exit code 1. 2024-06-26T07:08:33.3989522Z ##[group]Run .github/scripts/parse_ref.py 2024-06-26T07:08:33.3990016Z .github/scripts/parse_ref.py 2024-06-26T07:08:33.3996807Z shell: /usr/bin/bash -e {0} 2024-06-26T07:08:33.3997303Z env: 2024-06-26T07:08:33.3997597Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:33.3998072Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:33.3998579Z ##[endgroup] 2024-06-26T07:08:33.4223160Z Prepare all required actions 2024-06-26T07:08:33.4261035Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-06-26T07:08:33.4261504Z with: 2024-06-26T07:08:33.4262053Z github-token: *** 2024-06-26T07:08:33.4262366Z env: 2024-06-26T07:08:33.4262640Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:33.4263099Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:33.4263598Z ##[endgroup] 2024-06-26T07:08:33.4278608Z ##[group]Run set -eux 2024-06-26T07:08:33.4278942Z set -eux 2024-06-26T07:08:33.4279529Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-06-26T07:08:33.4287079Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:08:33.4287581Z env: 2024-06-26T07:08:33.4287866Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:33.4288336Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:33.4288982Z GITHUB_TOKEN: *** 2024-06-26T07:08:33.4289296Z ##[endgroup] 2024-06-26T07:08:33.4308615Z + python3 .github/scripts/get_workflow_job_id.py 9673646808 i-02d9c9aefd6777ce0 2024-06-26T07:08:34.7615849Z setting job-id=26688911835 2024-06-26T07:08:34.7616731Z setting job-name=cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:08:34.7812648Z Prepare all required actions 2024-06-26T07:08:34.7813103Z Getting action download info 2024-06-26T07:08:34.9041580Z ##[group]Run ./.github/actions/filter-test-configs 2024-06-26T07:08:34.9042052Z with: 2024-06-26T07:08:34.9042510Z github-token: *** 2024-06-26T07:08:34.9050956Z test-matrix: {"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-06-26T07:08:34.9060274Z job-name: cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:08:34.9061018Z env: 2024-06-26T07:08:34.9061307Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:34.9061919Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:34.9062413Z ##[endgroup] 2024-06-26T07:08:34.9103772Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:08:34.9104336Z with: 2024-06-26T07:08:34.9104613Z shell: bash 2024-06-26T07:08:34.9104920Z timeout_minutes: 10 2024-06-26T07:08:34.9105255Z max_attempts: 5 2024-06-26T07:08:34.9105569Z retry_wait_seconds: 30 2024-06-26T07:08:34.9106730Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:08:34.9107930Z polling_interval_seconds: 1 2024-06-26T07:08:34.9108311Z warning_on_retry: true 2024-06-26T07:08:34.9108663Z continue_on_error: false 2024-06-26T07:08:34.9109008Z env: 2024-06-26T07:08:34.9109287Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:34.9109744Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:34.9110397Z GITHUB_TOKEN: *** 2024-06-26T07:08:34.9110723Z ##[endgroup] 2024-06-26T07:08:34.9608675Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:08:35.1787908Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T07:08:35.1939975Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.7/site-packages (2.27.1) 2024-06-26T07:08:35.2083730Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.7/site-packages (6.0.1) 2024-06-26T07:08:35.2090872Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2024.6.2) 2024-06-26T07:08:35.2100231Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (3.7) 2024-06-26T07:08:35.2113031Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (1.26.19) 2024-06-26T07:08:35.2311048Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2.0.12) 2024-06-26T07:08:35.9612776Z Command completed after 1 attempt(s). 2024-06-26T07:08:35.9651767Z ##[group]Run set -x 2024-06-26T07:08:35.9652376Z set -x 2024-06-26T07:08:35.9652719Z  2024-06-26T07:08:35.9653295Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:08:35.9654075Z # in runner workspace 2024-06-26T07:08:35.9654612Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-06-26T07:08:35.9662385Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:08:35.9662894Z env: 2024-06-26T07:08:35.9663168Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:35.9663627Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:35.9664129Z ##[endgroup] 2024-06-26T07:08:35.9688146Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-06-26T07:08:35.9903468Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:08:35.9904023Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:08:35.9904500Z echo "Job name: ${JOB_NAME}" 2024-06-26T07:08:35.9904903Z  2024-06-26T07:08:35.9905459Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:08:35.9906156Z # in runner workspace 2024-06-26T07:08:35.9906738Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-06-26T07:08:35.9907387Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-06-26T07:08:35.9907852Z  --job-name "${JOB_NAME}" \ 2024-06-26T07:08:35.9916745Z  --test-matrix "{"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-06-26T07:08:35.9925788Z  --selected-test-configs "" \ 2024-06-26T07:08:35.9926256Z  --pr-number "${PR_NUMBER}" \ 2024-06-26T07:08:35.9926685Z  --tag "${TAG}" \ 2024-06-26T07:08:35.9927084Z  --event-name "${EVENT_NAME}" \ 2024-06-26T07:08:35.9927537Z  --schedule "${SCHEDULE}" \ 2024-06-26T07:08:35.9927970Z  --branch "${HEAD_BRANCH}" 2024-06-26T07:08:35.9935802Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:08:35.9936334Z env: 2024-06-26T07:08:35.9936618Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:35.9937085Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:35.9937761Z GITHUB_TOKEN: *** 2024-06-26T07:08:35.9938431Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:08:35.9939168Z PR_NUMBER: 2024-06-26T07:08:35.9939482Z TAG: ciflow/inductor/129470 2024-06-26T07:08:35.9939848Z EVENT_NAME: push 2024-06-26T07:08:35.9940162Z SCHEDULE: 2024-06-26T07:08:35.9940447Z HEAD_BRANCH: 2024-06-26T07:08:35.9940745Z ##[endgroup] 2024-06-26T07:08:35.9959428Z Workflow: inductor 2024-06-26T07:08:35.9960405Z Job name: cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:08:36.2032669Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-06-26T07:08:36.3476398Z ##[group]Run echo "Filtered matrix:" 2024-06-26T07:08:36.3476871Z echo "Filtered matrix:" 2024-06-26T07:08:36.3485331Z echo "{"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" 2024-06-26T07:08:36.3494672Z  2024-06-26T07:08:36.3494961Z echo 2024-06-26T07:08:36.3495353Z echo "Is the current job unstable? False" 2024-06-26T07:08:36.3495822Z  2024-06-26T07:08:36.3496094Z echo 2024-06-26T07:08:36.3496458Z echo "Is keep-going label set? False" 2024-06-26T07:08:36.3496906Z  2024-06-26T07:08:36.3497181Z echo 2024-06-26T07:08:36.3497493Z echo "Renabled issues? " 2024-06-26T07:08:36.3505146Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:08:36.3505661Z env: 2024-06-26T07:08:36.3505937Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:36.3506405Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:36.3506896Z ##[endgroup] 2024-06-26T07:08:36.3525933Z Filtered matrix: 2024-06-26T07:08:36.3536430Z {include: [{config: inductor, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_distributed, shard: 1, num_shards: 1, runner: linux.g5.12xlarge.nvidia.gpu}, {config: inductor_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_cpp_wrapper_abi_compatible, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}]} 2024-06-26T07:08:36.3545129Z 2024-06-26T07:08:36.3545291Z Is the current job unstable? False 2024-06-26T07:08:36.3545582Z 2024-06-26T07:08:36.3545794Z Is keep-going label set? False 2024-06-26T07:08:36.3546061Z 2024-06-26T07:08:36.3546187Z Renabled issues? 2024-06-26T07:08:36.3586940Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:08:36.3587653Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:08:36.3595233Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:08:36.3595735Z env: 2024-06-26T07:08:36.3596018Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:36.3596479Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:36.3596979Z JOB_TIMEOUT: 240 2024-06-26T07:08:36.3597285Z ##[endgroup] 2024-06-26T07:08:36.3665107Z ##[group]Run set -x 2024-06-26T07:08:36.3665502Z set -x 2024-06-26T07:08:36.3665803Z  2024-06-26T07:08:36.3666175Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-06-26T07:08:36.3666773Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-06-26T07:08:36.3667350Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-06-26T07:08:36.3667874Z  TEST_COMMAND=.ci/onnx/test.sh 2024-06-26T07:08:36.3668291Z else 2024-06-26T07:08:36.3668629Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:08:36.3669055Z fi 2024-06-26T07:08:36.3669334Z  2024-06-26T07:08:36.3669812Z # detached container should get cleaned up by teardown_ec2_linux 2024-06-26T07:08:36.3670580Z # TODO: Stop building test binaries as part of the build phase 2024-06-26T07:08:36.3671276Z # Used for GPU_FLAG since that doesn't play nice 2024-06-26T07:08:36.3671875Z # shellcheck disable=SC2086,SC2090 2024-06-26T07:08:36.3672341Z container_name=$(docker run \ 2024-06-26T07:08:36.3672769Z  ${GPU_FLAG:-} \ 2024-06-26T07:08:36.3673155Z  -e BUILD_ENVIRONMENT \ 2024-06-26T07:08:36.3673563Z  -e PR_NUMBER \ 2024-06-26T07:08:36.3673922Z  -e GITHUB_ACTIONS \ 2024-06-26T07:08:36.3674322Z  -e GITHUB_REPOSITORY \ 2024-06-26T07:08:36.3674735Z  -e GITHUB_WORKFLOW \ 2024-06-26T07:08:36.3675129Z  -e GITHUB_JOB \ 2024-06-26T07:08:36.3675493Z  -e GITHUB_RUN_ID \ 2024-06-26T07:08:36.3675885Z  -e GITHUB_RUN_NUMBER \ 2024-06-26T07:08:36.3676352Z  -e GITHUB_RUN_ATTEMPT \ 2024-06-26T07:08:36.3676747Z  -e JOB_ID \ 2024-06-26T07:08:36.3677091Z  -e JOB_NAME \ 2024-06-26T07:08:36.3677443Z  -e BASE_SHA \ 2024-06-26T07:08:36.3677788Z  -e BRANCH \ 2024-06-26T07:08:36.3678117Z  -e SHA1 \ 2024-06-26T07:08:36.3678469Z  -e AWS_DEFAULT_REGION \ 2024-06-26T07:08:36.3678879Z  -e IN_WHEEL_TEST \ 2024-06-26T07:08:36.3679261Z  -e SHARD_NUMBER \ 2024-06-26T07:08:36.3679635Z  -e TEST_CONFIG \ 2024-06-26T07:08:36.3680009Z  -e NUM_TEST_SHARDS \ 2024-06-26T07:08:36.3680411Z  -e REENABLED_ISSUES \ 2024-06-26T07:08:36.3680829Z  -e CONTINUE_THROUGH_ERROR \ 2024-06-26T07:08:36.3681258Z  -e VERBOSE_TEST_LOGS \ 2024-06-26T07:08:36.3681669Z  -e NO_TEST_TIMEOUT \ 2024-06-26T07:08:36.3682052Z  -e NO_TD \ 2024-06-26T07:08:36.3682395Z  -e TD_DISTRIBUTED \ 2024-06-26T07:08:36.3682778Z  -e PR_LABELS \ 2024-06-26T07:08:36.3683190Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-06-26T07:08:36.3683656Z  -e SCCACHE_BUCKET \ 2024-06-26T07:08:36.3684051Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-06-26T07:08:36.3684465Z  -e XLA_CUDA \ 2024-06-26T07:08:36.3684877Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-06-26T07:08:36.3685394Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-06-26T07:08:36.3685921Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-06-26T07:08:36.3686557Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-06-26T07:08:36.3687040Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-06-26T07:08:36.3687466Z  -e DASHBOARD_TAG \ 2024-06-26T07:08:36.3687942Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-06-26T07:08:36.3688511Z  --security-opt seccomp=unconfined \ 2024-06-26T07:08:36.3688980Z  --cap-add=SYS_PTRACE \ 2024-06-26T07:08:36.3689378Z  --ipc=host \ 2024-06-26T07:08:36.3689730Z  --shm-size="${SHM_SIZE}" \ 2024-06-26T07:08:36.3690132Z  --tty \ 2024-06-26T07:08:36.3690449Z  --detach \ 2024-06-26T07:08:36.3690816Z  --name="${container_name}" \ 2024-06-26T07:08:36.3691235Z  --user jenkins \ 2024-06-26T07:08:36.3691787Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-06-26T07:08:36.3692570Z  -w /var/lib/jenkins/workspace \ 2024-06-26T07:08:36.3693013Z  "${DOCKER_IMAGE}" 2024-06-26T07:08:36.3693369Z ) 2024-06-26T07:08:36.3693837Z # Propagate download.pytorch.org IP to container 2024-06-26T07:08:36.3694774Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-06-26T07:08:36.3695754Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-06-26T07:08:36.3696667Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-06-26T07:08:36.3703684Z shell: /usr/bin/bash -e {0} 2024-06-26T07:08:36.3704039Z env: 2024-06-26T07:08:36.3704318Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:08:36.3704776Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:36.3705391Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:08:36.3705900Z PR_NUMBER: 2024-06-26T07:08:36.3706223Z GITHUB_REPOSITORY: pytorch/pytorch 2024-06-26T07:08:36.3706648Z GITHUB_WORKFLOW: inductor 2024-06-26T07:08:36.3707004Z GITHUB_JOB: test 2024-06-26T07:08:36.3707324Z GITHUB_RUN_ID: 9673646808 2024-06-26T07:08:36.3707685Z GITHUB_RUN_NUMBER: 76977 2024-06-26T07:08:36.3708030Z GITHUB_RUN_ATTEMPT: 1 2024-06-26T07:08:36.3708371Z JOB_ID: 26688911835 2024-06-26T07:08:36.3709034Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:08:36.3709759Z BRANCH: 2024-06-26T07:08:36.3710091Z SHA1: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:08:36.3710606Z BASE_SHA: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:08:36.3711092Z TEST_CONFIG: dynamic_inductor_timm 2024-06-26T07:08:36.3711496Z SHARD_NUMBER: 2 2024-06-26T07:08:36.3711799Z NUM_TEST_SHARDS: 2 2024-06-26T07:08:36.3712124Z REENABLED_ISSUES: 2024-06-26T07:08:36.3712472Z CONTINUE_THROUGH_ERROR: False 2024-06-26T07:08:36.3712866Z VERBOSE_TEST_LOGS: False 2024-06-26T07:08:36.3713215Z NO_TEST_TIMEOUT: False 2024-06-26T07:08:36.3713552Z NO_TD: False 2024-06-26T07:08:36.3713864Z TD_DISTRIBUTED: False 2024-06-26T07:08:36.3714283Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-06-26T07:08:36.3714777Z SCCACHE_S3_KEY_PREFIX: inductor 2024-06-26T07:08:36.3715163Z SHM_SIZE: 2g 2024-06-26T07:08:36.3716264Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:08:36.3717449Z XLA_CUDA: 2024-06-26T07:08:36.3717923Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:08:36.3718536Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-06-26T07:08:36.3718979Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-06-26T07:08:36.3719392Z DASHBOARD_TAG: 2024-06-26T07:08:36.3719896Z HUGGING_FACE_HUB_TOKEN: *** 2024-06-26T07:08:36.3720265Z ##[endgroup] 2024-06-26T07:08:36.3738866Z + [[ dynamic_inductor_timm == \m\u\l\t\i\g\p\u ]] 2024-06-26T07:08:36.3739647Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]] 2024-06-26T07:08:36.3740302Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:08:36.3745613Z +++ nproc --ignore=2 2024-06-26T07:08:36.3762319Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_9673646808 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:08:47.5185573Z + container_name=9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T07:08:47.5186656Z + grep download.pytorch.org /etc/hosts 2024-06-26T07:08:47.5187820Z + docker exec -i 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 sudo bash -c '/bin/cat >> /etc/hosts' 2024-06-26T07:08:47.5773011Z + echo DOCKER_CONTAINER_ID=9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T07:08:47.5775961Z ++ echo dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:08:47.5777792Z + docker exec -t 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 sh -c 'pip install dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-06-26T07:08:47.9656764Z Processing ./dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+gitb8c4c54) 2024-06-26T07:08:48.2752050Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.13.1) 2024-06-26T07:08:48.2756239Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (4.12.2) 2024-06-26T07:08:48.2759201Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.12.1) 2024-06-26T07:08:48.2762693Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.8.8) 2024-06-26T07:08:48.2765923Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.1.4) 2024-06-26T07:08:48.2769302Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2024.2.0) 2024-06-26T07:08:48.2795902Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.3.0) 2024-06-26T07:08:48.2853037Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.21.2) 2024-06-26T07:08:48.3277392Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.1.5) 2024-06-26T07:08:48.3433450Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.3.0) 2024-06-26T07:08:49.6319469Z Installing collected packages: torch 2024-06-26T07:08:58.3716633Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-06-26T07:08:58.3718016Z timm 0.9.7 requires torchvision, which is not installed. 2024-06-26T07:08:58.3718700Z Successfully installed torch-2.5.0a0+gitb8c4c54 2024-06-26T07:08:58.4486190Z ++ dirname .ci/pytorch/test.sh 2024-06-26T07:08:58.4490376Z + source .ci/pytorch/common.sh 2024-06-26T07:08:58.4492494Z +++ dirname .ci/pytorch/common.sh 2024-06-26T07:08:58.4498276Z ++ source .ci/pytorch/common_utils.sh 2024-06-26T07:08:58.4499220Z +++ declare -f -t trap_add 2024-06-26T07:08:58.4504807Z ++ set -ex 2024-06-26T07:08:58.4505308Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-06-26T07:08:58.4505840Z ++ BUILD_TEST_LIBTORCH=0 2024-06-26T07:08:58.4506444Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]] 2024-06-26T07:08:58.4507624Z ++ stat -c %u /var/lib/jenkins/workspace 2024-06-26T07:08:58.4517183Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-06-26T07:08:58.4517642Z + trap_add cleanup_workspace EXIT 2024-06-26T07:08:58.4518074Z + trap_add_cmd=cleanup_workspace 2024-06-26T07:08:58.4518459Z + shift 2024-06-26T07:08:58.4518744Z + for trap_add_name in "$@" 2024-06-26T07:08:58.4523085Z +++ trap -p EXIT 2024-06-26T07:08:58.4525916Z ++ eval 'extract_trap_cmd ' 2024-06-26T07:08:58.4547050Z +++ extract_trap_cmd 2024-06-26T07:08:58.4549215Z +++ printf '%s\n' '' 2024-06-26T07:08:58.4549729Z ++ printf '%s\n' cleanup_workspace 2024-06-26T07:08:58.4550191Z + trap -- ' 2024-06-26T07:08:58.4550555Z cleanup_workspace' EXIT 2024-06-26T07:08:58.4551020Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-06-26T07:08:58.9838300Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-06-26T07:08:58.9850895Z + echo 'Environment variables:' 2024-06-26T07:08:58.9851318Z Environment variables: 2024-06-26T07:08:58.9851649Z + env 2024-06-26T07:08:58.9857286Z INSTALLED_DB=yes 2024-06-26T07:08:58.9857780Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-06-26T07:08:58.9858220Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:08:58.9858690Z NV_NVML_DEV_VERSION=12.1.105-1 2024-06-26T07:08:58.9859829Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:08:58.9860502Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:08:58.9861037Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-06-26T07:08:58.9861566Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:08:58.9862140Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:08:58.9862647Z HOSTNAME=9c6f49d01630 2024-06-26T07:08:58.9863548Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_06c398d1-db9b-49f7-90c0-3cb311bffd17 2024-06-26T07:08:58.9864379Z GITHUB_ACTION=__self 2024-06-26T07:08:58.9864740Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:08:58.9869087Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-06-26T07:08:58.9873105Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-06-26T07:08:58.9873619Z NV_NVTX_VERSION=12.1.105-1 2024-06-26T07:08:58.9874239Z GITHUB_RUN_NUMBER=76977 2024-06-26T07:08:58.9874603Z TEST_CONFIG=dynamic_inductor_timm 2024-06-26T07:08:58.9875013Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:08:58.9875497Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:08:58.9875964Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-06-26T07:08:58.9876413Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-06-26T07:08:58.9876834Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-06-26T07:08:58.9877280Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:08:58.9877827Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:08:58.9878306Z GITHUB_REF_TYPE=tag 2024-06-26T07:08:58.9878638Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:08:58.9879013Z NCCL_VERSION=2.17.1-1 2024-06-26T07:08:58.9879401Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:08:58.9879967Z XLA_CUDA= 2024-06-26T07:08:58.9880478Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:08:58.9882367Z *** 2024-06-26T07:08:58.9882665Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:08:58.9883106Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:08:58.9883502Z GITHUB_ACTIONS=true 2024-06-26T07:08:58.9883833Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:58.9884345Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-06-26T07:08:58.9884884Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-06-26T07:08:58.9885348Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:08:58.9885862Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:08:58.9886349Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:08:58.9887147Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:08:58.9887862Z UCC_HOME=/usr 2024-06-26T07:08:58.9888200Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-06-26T07:08:58.9888603Z VERBOSE_TEST_LOGS=False 2024-06-26T07:08:58.9888956Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:08:58.9889408Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-06-26T07:08:58.9889915Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:08:58.9890386Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-06-26T07:08:58.9890773Z SHARD_NUMBER=2 2024-06-26T07:08:58.9891080Z GITHUB_REF_PROTECTED=false 2024-06-26T07:08:58.9891434Z HOME=/var/lib/jenkins 2024-06-26T07:08:58.9891809Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:08:58.9892549Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:08:58.9893039Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:08:58.9893528Z SCCACHE_S3_KEY_PREFIX=inductor 2024-06-26T07:08:58.9893975Z CUDA_VERSION=12.1.1 2024-06-26T07:08:58.9894399Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-06-26T07:08:58.9894846Z NUM_TEST_SHARDS=2 2024-06-26T07:08:58.9895152Z UCX_HOME=/usr 2024-06-26T07:08:58.9895665Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-06-26T07:08:58.9896748Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_06c398d1-db9b-49f7-90c0-3cb311bffd17 2024-06-26T07:08:58.9897987Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:08:58.9899229Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_06c398d1-db9b-49f7-90c0-3cb311bffd17 2024-06-26T07:08:58.9900359Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:08:58.9901020Z GITHUB_EVENT_NAME=push 2024-06-26T07:08:58.9901364Z DASHBOARD_TAG= 2024-06-26T07:08:58.9901673Z GITHUB_RUN_ID=9673646808 2024-06-26T07:08:58.9902132Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-06-26T07:08:58.9902668Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-06-26T07:08:58.9903667Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_06c398d1-db9b-49f7-90c0-3cb311bffd17 2024-06-26T07:08:58.9904583Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:08:58.9904999Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-06-26T07:08:58.9905377Z PR_NUMBER= 2024-06-26T07:08:58.9905703Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:08:58.9906274Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:08:58.9906816Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:08:58.9907451Z TERM=xterm 2024-06-26T07:08:58.9907915Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-06-26T07:08:58.9908486Z INSTALLED_VISION=yes 2024-06-26T07:08:58.9908876Z BRANCH= 2024-06-26T07:08:58.9909232Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:08:58.9909729Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:08:58.9910253Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:08:58.9911181Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:08:58.9912162Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:08:58.9912869Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:08:58.9913627Z REENABLED_ISSUES= 2024-06-26T07:08:58.9914037Z SHLVL=1 2024-06-26T07:08:58.9914560Z MAX_JOBS=14 2024-06-26T07:08:58.9915028Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-06-26T07:08:58.9915543Z NVARCH=x86_64 2024-06-26T07:08:58.9916005Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:08:58.9916691Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:08:58.9917505Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:08:58.9918262Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-06-26T07:08:58.9919284Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:08:58.9920172Z GITHUB_JOB=test 2024-06-26T07:08:58.9920792Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-06-26T07:08:58.9921634Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:08:58.9922416Z NO_TEST_TIMEOUT=False 2024-06-26T07:08:58.9922916Z TD_DISTRIBUTED=False 2024-06-26T07:08:58.9923503Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-06-26T07:08:58.9924186Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:08:58.9924849Z NV_NVPROF_VERSION=12.1.105-1 2024-06-26T07:08:58.9925421Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:08:58.9925966Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:08:58.9926508Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:08:58.9929492Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:08:58.9931140Z GITHUB_BASE_REF= 2024-06-26T07:08:58.9931620Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:08:58.9932503Z CI=true 2024-06-26T07:08:58.9933001Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:08:58.9933763Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:08:58.9934329Z JOB_ID=26688911835 2024-06-26T07:08:58.9934809Z INSTALLED_PROTOBUF=yes 2024-06-26T07:08:58.9935312Z GITHUB_HEAD_REF= 2024-06-26T07:08:58.9935757Z GITHUB_ACTION_REF= 2024-06-26T07:08:58.9936438Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:08:58.9937169Z GITHUB_WORKFLOW=inductor 2024-06-26T07:08:58.9937712Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:08:58.9939120Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_06c398d1-db9b-49f7-90c0-3cb311bffd17 2024-06-26T07:08:58.9940399Z NO_TD=False 2024-06-26T07:08:58.9940861Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:08:58.9941424Z _=/usr/bin/env 2024-06-26T07:08:58.9942153Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-06-26T07:08:59.0005449Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-06-26T07:08:59.0006873Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-06-26T07:08:59.0008230Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-06-26T07:08:59.0009610Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-06-26T07:08:59.0010562Z + BUILD_DIR=build 2024-06-26T07:08:59.0011072Z + BUILD_RENAMED_DIR=build_renamed 2024-06-26T07:08:59.0011676Z + BUILD_BIN_DIR=build/bin 2024-06-26T07:08:59.0012436Z + SHARD_NUMBER=2 2024-06-26T07:08:59.0012917Z + NUM_TEST_SHARDS=2 2024-06-26T07:08:59.0013407Z + export VALGRIND=ON 2024-06-26T07:08:59.0014015Z + VALGRIND=ON 2024-06-26T07:08:59.0014941Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]] 2024-06-26T07:08:59.0015702Z + [[ 0 == \1 ]] 2024-06-26T07:08:59.0016158Z + [[ False == \1 ]] 2024-06-26T07:08:59.0016884Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]] 2024-06-26T07:08:59.0017698Z ++ realpath build/custom_test_artifacts 2024-06-26T07:08:59.0021645Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-06-26T07:08:59.0022726Z + [[ -n '' ]] 2024-06-26T07:08:59.0023258Z + echo 'Environment variables' 2024-06-26T07:08:59.0023848Z Environment variables 2024-06-26T07:08:59.0024321Z + env 2024-06-26T07:08:59.0027659Z INSTALLED_DB=yes 2024-06-26T07:08:59.0028141Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-06-26T07:08:59.0028624Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:08:59.0029020Z NV_NVML_DEV_VERSION=12.1.105-1 2024-06-26T07:08:59.0029767Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:08:59.0030359Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:08:59.0031009Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-06-26T07:08:59.0031723Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:08:59.0032508Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:08:59.0033159Z HOSTNAME=9c6f49d01630 2024-06-26T07:08:59.0034231Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_06c398d1-db9b-49f7-90c0-3cb311bffd17 2024-06-26T07:08:59.0035116Z GITHUB_ACTION=__self 2024-06-26T07:08:59.0035477Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:08:59.0039592Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-06-26T07:08:59.0043628Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-06-26T07:08:59.0044141Z NV_NVTX_VERSION=12.1.105-1 2024-06-26T07:08:59.0044498Z GITHUB_RUN_NUMBER=76977 2024-06-26T07:08:59.0044870Z TEST_CONFIG=dynamic_inductor_timm 2024-06-26T07:08:59.0045287Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:08:59.0045758Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:08:59.0046233Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-06-26T07:08:59.0046682Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-06-26T07:08:59.0047123Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-06-26T07:08:59.0047563Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:08:59.0048068Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:08:59.0048548Z GITHUB_REF_TYPE=tag 2024-06-26T07:08:59.0048876Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:08:59.0049262Z NCCL_VERSION=2.17.1-1 2024-06-26T07:08:59.0049653Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:08:59.0050104Z XLA_CUDA= 2024-06-26T07:08:59.0050583Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:08:59.0051034Z *** 2024-06-26T07:08:59.0051313Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:08:59.0051719Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:08:59.0053212Z GITHUB_ACTIONS=true 2024-06-26T07:08:59.0053560Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:08:59.0054214Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-06-26T07:08:59.0054749Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-06-26T07:08:59.0055222Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:08:59.0055731Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:08:59.0056218Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:08:59.0057024Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:08:59.0057878Z UCC_HOME=/usr 2024-06-26T07:08:59.0058217Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-06-26T07:08:59.0058621Z VERBOSE_TEST_LOGS=False 2024-06-26T07:08:59.0058969Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:08:59.0059429Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-06-26T07:08:59.0059929Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:08:59.0060400Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-06-26T07:08:59.0060785Z SHARD_NUMBER=2 2024-06-26T07:08:59.0061093Z GITHUB_REF_PROTECTED=false 2024-06-26T07:08:59.0077097Z HOME=/var/lib/jenkins 2024-06-26T07:08:59.0077522Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:08:59.0078003Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:08:59.0078615Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:08:59.0079117Z SCCACHE_S3_KEY_PREFIX=inductor 2024-06-26T07:08:59.0079497Z CUDA_VERSION=12.1.1 2024-06-26T07:08:59.0079934Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-06-26T07:08:59.0080393Z NUM_TEST_SHARDS=2 2024-06-26T07:08:59.0080699Z UCX_HOME=/usr 2024-06-26T07:08:59.0081211Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-06-26T07:08:59.0082285Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_06c398d1-db9b-49f7-90c0-3cb311bffd17 2024-06-26T07:08:59.0083522Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:08:59.0084752Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_06c398d1-db9b-49f7-90c0-3cb311bffd17 2024-06-26T07:08:59.0085878Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:08:59.0086546Z GITHUB_EVENT_NAME=push 2024-06-26T07:08:59.0086882Z DASHBOARD_TAG= 2024-06-26T07:08:59.0087191Z GITHUB_RUN_ID=9673646808 2024-06-26T07:08:59.0087649Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-06-26T07:08:59.0088190Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-06-26T07:08:59.0089185Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_06c398d1-db9b-49f7-90c0-3cb311bffd17 2024-06-26T07:08:59.0090105Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:08:59.0090520Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-06-26T07:08:59.0090896Z PR_NUMBER= 2024-06-26T07:08:59.0091187Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:08:59.0091511Z VALGRIND=ON 2024-06-26T07:08:59.0091815Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:08:59.0092500Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:08:59.0092995Z TERM=xterm 2024-06-26T07:08:59.0093360Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-06-26T07:08:59.0093877Z INSTALLED_VISION=yes 2024-06-26T07:08:59.0094194Z BRANCH= 2024-06-26T07:08:59.0094490Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:08:59.0094900Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:08:59.0095323Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:08:59.0096078Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:08:59.0096850Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:08:59.0097340Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:08:59.0097804Z REENABLED_ISSUES= 2024-06-26T07:08:59.0098109Z SHLVL=1 2024-06-26T07:08:59.0098371Z MAX_JOBS=14 2024-06-26T07:08:59.0098679Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-06-26T07:08:59.0099044Z NVARCH=x86_64 2024-06-26T07:08:59.0099344Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:08:59.0099796Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:08:59.0100325Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:08:59.0100815Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-06-26T07:08:59.0101475Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:08:59.0102058Z GITHUB_JOB=test 2024-06-26T07:08:59.0102446Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-06-26T07:08:59.0103036Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:08:59.0103689Z NO_TEST_TIMEOUT=False 2024-06-26T07:08:59.0104023Z TD_DISTRIBUTED=False 2024-06-26T07:08:59.0104396Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-06-26T07:08:59.0104837Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:08:59.0105257Z NV_NVPROF_VERSION=12.1.105-1 2024-06-26T07:08:59.0105616Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:08:59.0105969Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:08:59.0106322Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:08:59.0107386Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:08:59.0108463Z GITHUB_BASE_REF= 2024-06-26T07:08:59.0108789Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:08:59.0109158Z CI=true 2024-06-26T07:08:59.0109551Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:08:59.0109968Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:08:59.0110348Z JOB_ID=26688911835 2024-06-26T07:08:59.0110671Z INSTALLED_PROTOBUF=yes 2024-06-26T07:08:59.0110999Z GITHUB_HEAD_REF= 2024-06-26T07:08:59.0111311Z GITHUB_ACTION_REF= 2024-06-26T07:08:59.0111747Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:08:59.0112222Z GITHUB_WORKFLOW=inductor 2024-06-26T07:08:59.0112576Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:08:59.0113479Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_06c398d1-db9b-49f7-90c0-3cb311bffd17 2024-06-26T07:08:59.0114297Z NO_TD=False 2024-06-26T07:08:59.0114604Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:08:59.0114962Z _=/usr/bin/env 2024-06-26T07:08:59.0115293Z + echo 'Testing pytorch' 2024-06-26T07:08:59.0115637Z Testing pytorch 2024-06-26T07:08:59.0115960Z + export LANG=C.UTF-8 2024-06-26T07:08:59.0116289Z + LANG=C.UTF-8 2024-06-26T07:08:59.0116581Z + PR_NUMBER= 2024-06-26T07:08:59.0116937Z + [[ dynamic_inductor_timm == \d\e\f\a\u\l\t ]] 2024-06-26T07:08:59.0117456Z + [[ dynamic_inductor_timm == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-06-26T07:08:59.0117972Z + [[ dynamic_inductor_timm == \s\l\o\w ]] 2024-06-26T07:08:59.0118596Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-06-26T07:08:59.0119283Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]] 2024-06-26T07:08:59.0119826Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:08:59.0120304Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:08:59.0120759Z + [[ dynamic_inductor_timm == *crossref* ]] 2024-06-26T07:08:59.0121329Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-06-26T07:08:59.0121954Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]] 2024-06-26T07:08:59.0122692Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:08:59.0123255Z + pip_install --user ninja==1.10.2 2024-06-26T07:08:59.0123801Z + pip install --progress-bar off --user ninja==1.10.2 2024-06-26T07:08:59.3733776Z Collecting ninja==1.10.2 2024-06-26T07:08:59.3894226Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-06-26T07:08:59.4040056Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-06-26T07:09:00.6936912Z Installing collected packages: ninja 2024-06-26T07:09:00.7003990Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-06-26T07:09:00.7005285Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-06-26T07:09:00.7060475Z Successfully installed ninja-1.10.2 2024-06-26T07:09:00.7891495Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:09:00.7894053Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:09:00.7896824Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-06-26T07:09:00.7897467Z + install_tlparse 2024-06-26T07:09:00.7897865Z + pip_install --user tlparse==0.3.7 2024-06-26T07:09:00.7898411Z + pip install --progress-bar off --user tlparse==0.3.7 2024-06-26T07:09:01.1386173Z Collecting tlparse==0.3.7 2024-06-26T07:09:01.1550608Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-06-26T07:09:01.1654517Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-06-26T07:09:02.4807587Z Installing collected packages: tlparse 2024-06-26T07:09:02.5230473Z Successfully installed tlparse-0.3.7 2024-06-26T07:09:02.6052348Z ++ python -m site --user-base 2024-06-26T07:09:02.6205628Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:09:02.6207539Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]] 2024-06-26T07:09:02.6208279Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]] 2024-06-26T07:09:02.6209059Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:09:02.6210010Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-06-26T07:09:02.6211176Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass 2024-06-26T07:09:02.6211903Z + cd test 2024-06-26T07:09:02.6212639Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-06-26T07:09:04.0930111Z + [[ dynamic_inductor_timm == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-06-26T07:09:04.0930755Z + [[ dynamic_inductor_timm == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-06-26T07:09:04.0931583Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:09:04.0932394Z + pushd test 2024-06-26T07:09:04.0932755Z ~/workspace/test ~/workspace 2024-06-26T07:09:04.0933429Z ++ python -c 'import torch; print(torch.version.cuda)' 2024-06-26T07:09:05.5604451Z + CUDA_VERSION=12.1 2024-06-26T07:09:05.5605076Z + '[' 12.1 == 12.4 ']' 2024-06-26T07:09:05.5605418Z + ISCUDA124= 2024-06-26T07:09:05.5605708Z + popd 2024-06-26T07:09:05.5606025Z ~/workspace 2024-06-26T07:09:05.5607958Z + DYNAMO_BENCHMARK_FLAGS=() 2024-06-26T07:09:05.5608530Z + [[ dynamic_inductor_timm == *dynamo_eager* ]] 2024-06-26T07:09:05.5609046Z + [[ dynamic_inductor_timm == *aot_eager* ]] 2024-06-26T07:09:05.5609606Z + [[ dynamic_inductor_timm == *aot_inductor* ]] 2024-06-26T07:09:05.5610148Z + [[ dynamic_inductor_timm == *inductor* ]] 2024-06-26T07:09:05.5610619Z + [[ dynamic_inductor_timm != *perf* ]] 2024-06-26T07:09:05.5611128Z + DYNAMO_BENCHMARK_FLAGS+=(--inductor) 2024-06-26T07:09:05.5611604Z + [[ dynamic_inductor_timm == *dynamic* ]] 2024-06-26T07:09:05.5612505Z + DYNAMO_BENCHMARK_FLAGS+=(--dynamic-shapes --dynamic-batch-only) 2024-06-26T07:09:05.5613115Z + [[ dynamic_inductor_timm == *cpu_inductor* ]] 2024-06-26T07:09:05.5613732Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-06-26T07:09:05.5641947Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-06-26T07:09:05.5642634Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-06-26T07:09:05.5644740Z + cd test 2024-06-26T07:09:05.5645196Z + python -c 'import torch; print(torch.__config__.show())' 2024-06-26T07:09:06.8952304Z PyTorch built with: 2024-06-26T07:09:06.8952987Z - GCC 9.4 2024-06-26T07:09:06.8953430Z - C++ Version: 201703 2024-06-26T07:09:06.8954450Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:09:06.8955679Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:09:06.8956447Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:09:06.8956969Z - LAPACK is enabled (usually provided by MKL) 2024-06-26T07:09:06.8957544Z - NNPACK is enabled 2024-06-26T07:09:06.8958327Z - CPU capability usage: AVX2 2024-06-26T07:09:06.8958752Z - CUDA Runtime 12.1 2024-06-26T07:09:06.8959337Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-06-26T07:09:06.8960022Z - CuDNN 90.1 (built against CUDA 12.4) 2024-06-26T07:09:06.8960481Z - Magma 2.6.1 2024-06-26T07:09:06.8968255Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-06-26T07:09:06.8975857Z 2024-06-26T07:09:07.1733768Z + cd test 2024-06-26T07:09:07.1734460Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-06-26T07:09:08.3848428Z ATen/Parallel: 2024-06-26T07:09:08.3848981Z at::get_num_threads() : 8 2024-06-26T07:09:08.3849519Z at::get_num_interop_threads() : 16 2024-06-26T07:09:08.3850031Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:09:08.3850467Z omp_get_max_threads() : 8 2024-06-26T07:09:08.3851509Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:09:08.3852744Z mkl_get_max_threads() : 8 2024-06-26T07:09:08.3853471Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:09:08.3854161Z std::thread::hardware_concurrency() : 16 2024-06-26T07:09:08.3854662Z Environment variables: 2024-06-26T07:09:08.3855021Z OMP_NUM_THREADS : [not set] 2024-06-26T07:09:08.3855460Z MKL_NUM_THREADS : [not set] 2024-06-26T07:09:08.3855882Z ATen parallel backend: OpenMP 2024-06-26T07:09:08.3856234Z 2024-06-26T07:09:08.6278758Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-06-26T07:09:08.6279496Z + [[ dynamic_inductor_timm == *backward* ]] 2024-06-26T07:09:08.6279987Z + [[ dynamic_inductor_timm == *xla* ]] 2024-06-26T07:09:08.6280470Z + [[ dynamic_inductor_timm == *executorch* ]] 2024-06-26T07:09:08.6280982Z + [[ dynamic_inductor_timm == \j\i\t\_\l\e\g\a\c\y ]] 2024-06-26T07:09:08.6281735Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-06-26T07:09:08.6282304Z + [[ dynamic_inductor_timm == distributed ]] 2024-06-26T07:09:08.6282832Z + [[ dynamic_inductor_timm == *inductor_distributed* ]] 2024-06-26T07:09:08.6283421Z + [[ dynamic_inductor_timm == *inductor-halide* ]] 2024-06-26T07:09:08.6284044Z + [[ dynamic_inductor_timm == *inductor-micro-benchmark* ]] 2024-06-26T07:09:08.6284605Z + [[ dynamic_inductor_timm == *huggingface* ]] 2024-06-26T07:09:08.6285082Z + [[ dynamic_inductor_timm == *timm* ]] 2024-06-26T07:09:08.6285499Z + install_torchvision 2024-06-26T07:09:08.6285841Z + local orig_preload 2024-06-26T07:09:08.6286168Z + local commit 2024-06-26T07:09:08.6286489Z ++ get_pinned_commit vision 2024-06-26T07:09:08.6286881Z ++ cat .github/ci_commit_pins/vision.txt 2024-06-26T07:09:08.6294024Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:09:08.6294641Z + orig_preload= 2024-06-26T07:09:08.6295013Z + '[' -n '' ']' 2024-06-26T07:09:08.6296131Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:09:08.6297588Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:09:08.9371564Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:09:08.9375585Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-dq8ctpp9 2024-06-26T07:09:08.9391132Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-dq8ctpp9 2024-06-26T07:09:10.2605395Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-06-26T07:09:10.2623916Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:09:11.4617305Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:09:11.6866688Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:09:13.9704828Z Preparing metadata (setup.py) ... [?25l- \ done 2024-06-26T07:09:13.9752669Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2) 2024-06-26T07:09:13.9755441Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+gitb8c4c54) 2024-06-26T07:09:13.9760446Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-06-26T07:09:13.9965415Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-06-26T07:09:13.9969617Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-06-26T07:09:13.9971640Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.12.1) 2024-06-26T07:09:13.9974850Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-06-26T07:09:13.9978068Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-06-26T07:09:13.9980253Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.2.0) 2024-06-26T07:09:14.0404544Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-06-26T07:09:14.0560228Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-06-26T07:09:14.0635001Z Building wheels for collected packages: torchvision 2024-06-26T07:10:27.3468452Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2024-06-26T07:10:27.3494594Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2112709 sha256=ac745da9df9a20161f34edf5cc89293eeaae4b8106c3c281edffcacafc0e4334 2024-06-26T07:10:27.3497957Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-06-26T07:10:27.3531467Z Successfully built torchvision 2024-06-26T07:10:28.5320627Z Installing collected packages: torchvision 2024-06-26T07:10:28.9131068Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-06-26T07:10:29.0236901Z + '[' -n '' ']' 2024-06-26T07:10:29.0237239Z + id=1 2024-06-26T07:10:29.0237569Z + test_dynamo_benchmark timm_models 1 2024-06-26T07:10:29.0241144Z ++ pwd 2024-06-26T07:10:29.0243858Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:10:29.0244504Z + local suite=timm_models 2024-06-26T07:10:29.0244849Z + shift 2024-06-26T07:10:29.0245133Z + local shard_id=1 2024-06-26T07:10:29.0245443Z + shift 2024-06-26T07:10:29.0245788Z + [[ dynamic_inductor_timm == *perf_compare* ]] 2024-06-26T07:10:29.0246836Z + [[ dynamic_inductor_timm == *perf* ]] 2024-06-26T07:10:29.0247482Z + [[ dynamic_inductor_timm == *cpu_inductor* ]] 2024-06-26T07:10:29.0248444Z + [[ dynamic_inductor_timm == *aot_inductor* ]] 2024-06-26T07:10:29.0249347Z + test_single_dynamo_benchmark inference timm_models 1 --inference --bfloat16 2024-06-26T07:10:29.0250013Z ++ pwd 2024-06-26T07:10:29.0250504Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:10:29.0251308Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:10:29.0262008Z + local name=inference 2024-06-26T07:10:29.0262469Z + shift 2024-06-26T07:10:29.0262836Z + local suite=timm_models 2024-06-26T07:10:29.0263294Z + shift 2024-06-26T07:10:29.0263583Z + local shard_id=1 2024-06-26T07:10:29.0263958Z + shift 2024-06-26T07:10:29.0264248Z + partition_flags=() 2024-06-26T07:10:29.0264587Z + local partition_flags 2024-06-26T07:10:29.0264961Z + [[ -n 2 ]] 2024-06-26T07:10:29.0265276Z + [[ -n 1 ]] 2024-06-26T07:10:29.0265870Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-06-26T07:10:29.0266572Z + [[ dynamic_inductor_timm == *perf_compare* ]] 2024-06-26T07:10:29.0267065Z + [[ dynamic_inductor_timm == *perf* ]] 2024-06-26T07:10:29.0267539Z + [[ dynamic_inductor_timm == *aot_inductor* ]] 2024-06-26T07:10:29.0269378Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --inductor --dynamic-shapes --dynamic-batch-only --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 1 --output /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv 2024-06-26T07:10:34.8491835Z 2024-06-26T07:10:34.9946417Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:10:34.9947071Z 2024-06-26T07:10:35.0581680Z model.safetensors: 0% 0.00/29.6M [00:00 will be ignored 2024-06-26T07:33:35.2608757Z pass 2024-06-26T07:33:35.2654441Z TIMING: entire_frame_compile:98.85439 code_gen:28.22483 inductor_compile:57.29641 backend_compile:84.07768 2024-06-26T07:33:35.2656884Z STATS: call_* op count: 797 | FakeTensor.__torch_dispatch__:16631 | FakeTensorMode.__torch_dispatch__:91510 | attempt fast:1800 | fast is_contiguous:1800 | ProxyTorchDispatchMode.__torch_dispatch__:17697 2024-06-26T07:33:35.2658269Z Dynamo produced 4 graphs covering 797 ops with 7 graph breaks (5 unique) 2024-06-26T07:33:43.2464209Z 2024-06-26T07:33:43.9227369Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:33:43.9227845Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:33:43.9231822Z cuda train mobilenetv2_100 2024-06-26T07:34:52.7660329Z W0626 07:34:52.765000 139690397848192 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:35:37.1734143Z pass 2024-06-26T07:35:37.1779956Z TIMING: entire_frame_compile:105.20873 code_gen:27.11501 inductor_compile:59.99732 backend_compile:89.31598 2024-06-26T07:35:37.1782497Z STATS: call_* op count: 796 | FakeTensor.__torch_dispatch__:17704 | FakeTensorMode.__torch_dispatch__:98341 | attempt fast:1936 | fast is_contiguous:1936 | ProxyTorchDispatchMode.__torch_dispatch__:17713 2024-06-26T07:35:37.1783884Z Dynamo produced 3 graphs covering 796 ops with 7 graph breaks (5 unique) 2024-06-26T07:35:45.6579871Z 2024-06-26T07:35:46.4076895Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:35:46.4077370Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:35:46.4081138Z cuda train mobilenetv3_large_100 2024-06-26T07:37:02.5630286Z W0626 07:37:02.562000 140522143117952 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:37:53.7543648Z pass 2024-06-26T07:37:53.7611466Z TIMING: entire_frame_compile:118.74999 code_gen:31.9351 inductor_compile:69.23465 backend_compile:101.75218 2024-06-26T07:37:53.7613513Z STATS: call_* op count: 855 | FakeTensor.__torch_dispatch__:19022 | FakeTensorMode.__torch_dispatch__:105653 | attempt fast:2386 | fast is_contiguous:2386 | ProxyTorchDispatchMode.__torch_dispatch__:19569 2024-06-26T07:37:53.7614870Z Dynamo produced 3 graphs covering 855 ops with 7 graph breaks (5 unique) 2024-06-26T07:38:02.6985140Z 2024-06-26T07:38:03.4542698Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:38:03.4543181Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:38:03.4543622Z cuda train mobilevit_s 2024-06-26T07:40:17.4784332Z pass 2024-06-26T07:40:17.4862215Z TIMING: entire_frame_compile:123.8247 code_gen:47.1963 inductor_compile:84.68821 backend_compile:112.61704 2024-06-26T07:40:17.4864346Z STATS: call_* op count: 542 | FakeTensor.__torch_dispatch__:11299 | FakeTensorMode.__torch_dispatch__:77918 | attempt fast:3155 | fast is_contiguous:3143 | ProxyTorchDispatchMode.__torch_dispatch__:16472 | slow no contiguity match:12 2024-06-26T07:40:17.4866257Z Dynamo produced 3 graphs covering 542 ops with 6 graph breaks (5 unique) 2024-06-26T07:40:25.5869141Z 2024-06-26T07:40:26.6710478Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:40:26.6711117Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:40:26.6711783Z cuda train nfnet_l0 2024-06-26T07:41:40.1775237Z W0626 07:41:40.176000 140400538305152 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:42:45.7373590Z pass 2024-06-26T07:42:45.7604621Z TIMING: entire_frame_compile:129.7191 code_gen:32.65939 inductor_compile:68.76178 backend_compile:107.71208 2024-06-26T07:42:45.7608701Z STATS: call_* op count: 1232 | FakeTensorMode.__torch_dispatch__:126149 | attempt fast:1242 | fast is_contiguous:1242 | FakeTensor.__torch_dispatch__:21296 | ProxyTorchDispatchMode.__torch_dispatch__:25953 2024-06-26T07:42:45.7610083Z Dynamo produced 3 graphs covering 1232 ops with 7 graph breaks (5 unique) 2024-06-26T07:42:55.0586881Z 2024-06-26T07:42:55.9977096Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:42:55.9977572Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:42:55.9978013Z cuda train pit_b_224 2024-06-26T07:44:04.3232898Z pass 2024-06-26T07:44:04.3705298Z TIMING: entire_frame_compile:61.15451 code_gen:12.4246 inductor_compile:34.13495 backend_compile:53.96926 2024-06-26T07:44:04.3707780Z STATS: call_* op count: 438 | FakeTensor.__torch_dispatch__:7228 | FakeTensorMode.__torch_dispatch__:56768 | attempt fast:2576 | fast is_contiguous:2576 | ProxyTorchDispatchMode.__torch_dispatch__:12561 2024-06-26T07:44:04.3709152Z Dynamo produced 2 graphs covering 438 ops with 6 graph breaks (5 unique) 2024-06-26T07:44:10.8898228Z 2024-06-26T07:44:12.4431793Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:44:12.4432291Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:44:12.4435487Z cuda train pnasnet5large 2024-06-26T07:50:21.5307550Z pass 2024-06-26T07:50:21.6447188Z TIMING: entire_frame_compile:343.17342 code_gen:160.10963 inductor_compile:246.37616 backend_compile:318.91931 2024-06-26T07:50:21.6450738Z STATS: call_* op count: 963 | FakeTensor.__torch_dispatch__:38522 | FakeTensorMode.__torch_dispatch__:240517 | attempt fast:7265 | fast is_contiguous:7217 | ProxyTorchDispatchMode.__torch_dispatch__:49820 | slow no contiguity match:48 2024-06-26T07:50:21.6452520Z Dynamo produced 2 graphs covering 963 ops with 5 graph breaks (4 unique) 2024-06-26T07:50:37.7780741Z 2024-06-26T07:50:39.0203335Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:50:39.0203803Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:50:39.0206446Z cuda train poolformer_m36 2024-06-26T07:55:01.9957424Z pass 2024-06-26T07:55:02.0360992Z TIMING: entire_frame_compile:251.30103 code_gen:136.10104 inductor_compile:191.10557 backend_compile:236.40505 2024-06-26T07:55:02.0364271Z STATS: call_* op count: 741 | FakeTensor.__torch_dispatch__:16736 | FakeTensorMode.__torch_dispatch__:108619 | attempt fast:7695 | fast is_contiguous:7551 | ProxyTorchDispatchMode.__torch_dispatch__:22827 | slow both tensors nontrivially broadcast:144 2024-06-26T07:55:02.0366400Z Dynamo produced 2 graphs covering 741 ops with 6 graph breaks (5 unique) 2024-06-26T07:55:13.3573635Z 2024-06-26T07:55:14.0499676Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:55:14.0500394Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:55:14.0501060Z cuda train regnety_002 2024-06-26T07:56:21.9506670Z pass 2024-06-26T07:56:21.9542577Z TIMING: entire_frame_compile:62.93409 code_gen:14.49094 inductor_compile:35.81092 backend_compile:55.00827 2024-06-26T07:56:21.9544887Z STATS: call_* op count: 369 | FakeTensor.__torch_dispatch__:9014 | FakeTensorMode.__torch_dispatch__:57115 | attempt fast:1826 | fast is_contiguous:1826 | ProxyTorchDispatchMode.__torch_dispatch__:12209 2024-06-26T07:56:21.9546268Z Dynamo produced 2 graphs covering 369 ops with 6 graph breaks (5 unique) 2024-06-26T07:56:28.5410047Z 2024-06-26T07:56:29.4973858Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:56:29.4974333Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:56:29.4974775Z cuda train repvgg_a2 2024-06-26T07:57:32.9099524Z W0626 07:57:32.909000 139959086105216 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:58:18.9026221Z pass 2024-06-26T07:58:18.9208963Z TIMING: entire_frame_compile:100.2363 code_gen:27.91489 inductor_compile:57.88565 backend_compile:85.43086 2024-06-26T07:58:18.9211321Z STATS: call_* op count: 920 | FakeTensor.__torch_dispatch__:17643 | FakeTensorMode.__torch_dispatch__:96457 | attempt fast:2269 | fast is_contiguous:2269 | ProxyTorchDispatchMode.__torch_dispatch__:18975 2024-06-26T07:58:18.9213068Z Dynamo produced 3 graphs covering 920 ops with 7 graph breaks (5 unique) 2024-06-26T07:58:26.9547152Z 2024-06-26T07:58:28.3009795Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:58:28.3010302Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:58:28.3010907Z cuda train res2net101_26w_4s 2024-06-26T08:01:58.9994727Z pass 2024-06-26T08:01:59.0323685Z TIMING: entire_frame_compile:195.24609 code_gen:44.64545 inductor_compile:118.06938 backend_compile:181.56084 2024-06-26T08:01:59.0325377Z STATS: call_* op count: 807 | FakeTensor.__torch_dispatch__:33467 | FakeTensorMode.__torch_dispatch__:193519 | attempt fast:6719 | fast is_contiguous:6313 | slow no contiguity match:406 | ProxyTorchDispatchMode.__torch_dispatch__:37834 2024-06-26T08:01:59.0328114Z Dynamo produced 2 graphs covering 807 ops with 6 graph breaks (5 unique) 2024-06-26T08:02:12.1973873Z 2024-06-26T08:02:13.2784642Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:02:13.2785196Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:02:13.2785821Z cuda train res2net50_14w_8s 2024-06-26T08:05:27.0795694Z pass 2024-06-26T08:05:27.0989230Z TIMING: entire_frame_compile:180.66646 code_gen:44.57523 inductor_compile:110.32473 backend_compile:168.52168 2024-06-26T08:05:27.0992331Z STATS: call_* op count: 703 | FakeTensor.__torch_dispatch__:29651 | FakeTensorMode.__torch_dispatch__:176919 | attempt fast:6023 | fast is_contiguous:5519 | slow no contiguity match:504 | ProxyTorchDispatchMode.__torch_dispatch__:33247 2024-06-26T08:05:27.0994570Z Dynamo produced 2 graphs covering 703 ops with 6 graph breaks (5 unique) 2024-06-26T08:05:39.4089417Z 2024-06-26T08:05:40.3548393Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:05:40.3548908Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:05:40.3549358Z cuda train res2next50 2024-06-26T08:07:32.0848099Z pass 2024-06-26T08:07:32.1017497Z TIMING: entire_frame_compile:103.20406 code_gen:28.10539 inductor_compile:64.90878 backend_compile:96.26769 2024-06-26T08:07:32.1019898Z STATS: call_* op count: 399 | FakeTensor.__torch_dispatch__:16673 | FakeTensorMode.__torch_dispatch__:96198 | attempt fast:3319 | fast is_contiguous:3151 | slow no contiguity match:168 | ProxyTorchDispatchMode.__torch_dispatch__:18879 2024-06-26T08:07:32.1021694Z Dynamo produced 2 graphs covering 399 ops with 6 graph breaks (5 unique) 2024-06-26T08:07:40.4959558Z 2024-06-26T08:07:41.1022934Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:07:41.1023543Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:07:41.1023993Z cuda train resmlp_12_224 2024-06-26T08:08:21.1491662Z pass 2024-06-26T08:08:21.1615061Z TIMING: entire_frame_compile:36.29944 code_gen:7.64423 inductor_compile:18.81152 backend_compile:31.5868 2024-06-26T08:08:21.1618834Z STATS: call_* op count: 216 | FakeTensor.__torch_dispatch__:4403 | FakeTensorMode.__torch_dispatch__:34997 | attempt fast:1512 | slow no contiguity match:552 | fast is_contiguous:960 | ProxyTorchDispatchMode.__torch_dispatch__:8345 2024-06-26T08:08:21.1621307Z Dynamo produced 2 graphs covering 216 ops with 6 graph breaks (5 unique) 2024-06-26T08:08:26.3910035Z 2024-06-26T08:08:27.7850865Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:08:27.7851458Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:08:27.7851972Z cuda train resnest101e 2024-06-26T08:12:05.8208688Z pass 2024-06-26T08:12:05.8604209Z TIMING: entire_frame_compile:201.20128 code_gen:49.54706 inductor_compile:123.24981 backend_compile:185.98246 2024-06-26T08:12:05.8605901Z STATS: call_* op count: 1121 | FakeTensor.__torch_dispatch__:29732 | FakeTensorMode.__torch_dispatch__:184094 | attempt fast:5851 | fast is_contiguous:5719 | ProxyTorchDispatchMode.__torch_dispatch__:38715 | slow no contiguity match:132 2024-06-26T08:12:05.8610912Z Dynamo produced 2 graphs covering 1121 ops with 6 graph breaks (5 unique) 2024-06-26T08:12:19.1424273Z 2024-06-26T08:12:20.0440823Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:12:20.0441387Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:12:20.0441865Z cuda train rexnet_100 2024-06-26T08:14:12.0390611Z W0626 08:14:12.038000 139671134315136 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:15:18.5361547Z pass 2024-06-26T08:15:18.5441537Z TIMING: entire_frame_compile:165.36397 code_gen:49.96019 inductor_compile:100.19318 backend_compile:142.01233 2024-06-26T08:15:18.5444291Z STATS: call_* op count: 1104 | FakeTensor.__torch_dispatch__:24915 | FakeTensorMode.__torch_dispatch__:139418 | attempt fast:2729 | fast is_contiguous:2652 | slow no contiguity match:77 | ProxyTorchDispatchMode.__torch_dispatch__:26051 2024-06-26T08:15:18.5445846Z Dynamo produced 3 graphs covering 1104 ops with 7 graph breaks (5 unique) 2024-06-26T08:15:29.6359745Z 2024-06-26T08:15:30.5409209Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:15:30.5409810Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:15:30.5410261Z cuda train sebotnet33ts_256 2024-06-26T08:17:11.7893044Z pass 2024-06-26T08:17:11.8016711Z TIMING: entire_frame_compile:91.22909 code_gen:23.3848 inductor_compile:52.0287 backend_compile:80.94008 2024-06-26T08:17:11.8030720Z STATS: call_* op count: 510 | FakeTensor.__torch_dispatch__:9495 | FakeTensorMode.__torch_dispatch__:65697 | attempt fast:2204 | fast is_contiguous:2171 | slow no contiguity match:33 | ProxyTorchDispatchMode.__torch_dispatch__:13430 2024-06-26T08:17:11.8032287Z Dynamo produced 3 graphs covering 510 ops with 6 graph breaks (5 unique) 2024-06-26T08:17:19.3709859Z 2024-06-26T08:17:20.3884374Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:17:20.3884857Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:17:20.3885294Z cuda train selecsls42b 2024-06-26T08:18:12.9083318Z pass 2024-06-26T08:18:12.9348928Z TIMING: entire_frame_compile:47.30598 code_gen:13.31792 inductor_compile:29.9581 backend_compile:43.79582 2024-06-26T08:18:12.9350604Z STATS: call_* op count: 137 | FakeTensor.__torch_dispatch__:7417 | FakeTensorMode.__torch_dispatch__:42313 | attempt fast:1421 | fast is_contiguous:1391 | ProxyTorchDispatchMode.__torch_dispatch__:8917 | slow no contiguity match:30 2024-06-26T08:18:12.9352335Z Dynamo produced 2 graphs covering 137 ops with 6 graph breaks (5 unique) 2024-06-26T08:18:18.6860397Z 2024-06-26T08:18:19.4009855Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:18:19.4010736Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:18:19.4011268Z cuda train spnasnet_100 2024-06-26T08:19:29.8550701Z W0626 08:19:29.854000 140619583541888 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:20:23.9405250Z pass 2024-06-26T08:20:23.9446901Z TIMING: entire_frame_compile:115.82728 code_gen:30.04312 inductor_compile:66.04377 backend_compile:99.19426 2024-06-26T08:20:23.9448842Z STATS: call_* op count: 976 | FakeTensor.__torch_dispatch__:20374 | FakeTensorMode.__torch_dispatch__:112111 | attempt fast:2224 | fast is_contiguous:2224 | ProxyTorchDispatchMode.__torch_dispatch__:21710 2024-06-26T08:20:23.9452250Z Dynamo produced 3 graphs covering 976 ops with 7 graph breaks (5 unique) 2024-06-26T08:20:32.8394468Z 2024-06-26T08:20:34.5293939Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:20:34.5294418Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:20:34.5294876Z cuda train swin_base_patch4_window7_224 2024-06-26T08:24:00.5748068Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T08:24:00.5750634Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 371, in torch_dynamo_resume_in_forward_and_backward_pass_at_369 2024-06-26T08:24:00.5751746Z pred = mod(*cloned_inputs) 2024-06-26T08:24:00.5752767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:24:00.5753630Z return forward_call(*args, **kwargs) 2024-06-26T08:24:00.5754652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 617, in forward 2024-06-26T08:24:00.5757440Z x = self.forward_features(x) 2024-06-26T08:24:00.5758573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 609, in forward_features 2024-06-26T08:24:00.5759561Z x = self.layers(x) 2024-06-26T08:24:00.5760629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:24:00.5761575Z return forward_call(*args, **kwargs) 2024-06-26T08:24:00.5762637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 443, in forward 2024-06-26T08:24:00.5763514Z x = self.blocks(x) 2024-06-26T08:24:00.5764427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:24:00.5765398Z return forward_call(*args, **kwargs) 2024-06-26T08:24:00.5766393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 323, in forward 2024-06-26T08:24:00.5767378Z x = x + self.drop_path1(self._attn(self.norm1(x))) 2024-06-26T08:24:00.5768456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 307, in _attn 2024-06-26T08:24:00.5769604Z attn_windows = self.attn(x_windows, mask=self.attn_mask) # nW*B, window_size*window_size, C 2024-06-26T08:24:00.5770921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:24:00.5771826Z return forward_call(*args, **kwargs) 2024-06-26T08:24:00.5773229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 172, in forward 2024-06-26T08:24:00.5774244Z attn = attn + self._get_rel_pos_bias() 2024-06-26T08:24:00.5775359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 143, in _get_rel_pos_bias 2024-06-26T08:24:00.5776469Z relative_position_bias = self.relative_position_bias_table[ 2024-06-26T08:24:00.5776884Z 2024-06-26T08:24:07.8453178Z W0626 08:24:07.844000 139812089270912 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:25:35.7973749Z pass 2024-06-26T08:25:35.8416639Z TIMING: entire_frame_compile:283.2952 code_gen:63.61365 inductor_compile:152.2846 backend_compile:247.93583 2024-06-26T08:25:35.8419235Z STATS: call_* op count: 2715 | FakeTensor.__torch_dispatch__:31967 | FakeTensorMode.__torch_dispatch__:245272 | attempt fast:6318 | fast is_contiguous:6174 | slow no contiguity match:144 | ProxyTorchDispatchMode.__torch_dispatch__:50112 2024-06-26T08:25:35.8420785Z Dynamo produced 3 graphs covering 2715 ops with 7 graph breaks (5 unique) 2024-06-26T08:25:53.3110390Z 2024-06-26T08:25:56.2921934Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:25:56.2924693Z loading model: 0it [00:02, ?it/s] 2024-06-26T08:25:56.2925900Z cuda train swsl_resnext101_32x16d 2024-06-26T08:27:53.5391498Z pass 2024-06-26T08:27:53.6729403Z TIMING: entire_frame_compile:99.29758 code_gen:21.70941 inductor_compile:58.71225 backend_compile:92.04283 2024-06-26T08:27:53.6730949Z STATS: call_* op count: 415 | FakeTensor.__torch_dispatch__:18037 | FakeTensorMode.__torch_dispatch__:101806 | attempt fast:3607 | fast is_contiguous:3607 | ProxyTorchDispatchMode.__torch_dispatch__:22001 2024-06-26T08:27:53.6736743Z Dynamo produced 2 graphs covering 415 ops with 6 graph breaks (5 unique) 2024-06-26T08:28:02.1456718Z 2024-06-26T08:28:02.9361348Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:28:02.9361836Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:28:02.9362292Z cuda train tf_efficientnet_b0 2024-06-26T08:29:38.2799637Z pass 2024-06-26T08:29:38.2879892Z TIMING: entire_frame_compile:87.86961 code_gen:22.02855 inductor_compile:51.77986 backend_compile:77.51989 2024-06-26T08:29:38.2883580Z STATS: call_* op count: 371 | FakeTensorMode.__torch_dispatch__:71403 | attempt fast:2747 | fast is_contiguous:2747 | FakeTensor.__torch_dispatch__:11897 | ProxyTorchDispatchMode.__torch_dispatch__:14649 2024-06-26T08:29:38.2886325Z Dynamo produced 2 graphs covering 371 ops with 6 graph breaks (5 unique) 2024-06-26T08:29:45.9904336Z 2024-06-26T08:29:46.8670463Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:29:46.8670957Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:29:46.8671448Z cuda train tf_mixnet_l 2024-06-26T08:31:53.2952034Z pass 2024-06-26T08:31:53.3037739Z TIMING: entire_frame_compile:117.20096 code_gen:27.99544 inductor_compile:68.48776 backend_compile:103.39615 2024-06-26T08:31:53.3040615Z STATS: call_* op count: 692 | FakeTensorMode.__torch_dispatch__:101019 | attempt fast:3078 | fast is_contiguous:3078 | FakeTensor.__torch_dispatch__:15291 | ProxyTorchDispatchMode.__torch_dispatch__:19823 2024-06-26T08:31:53.3042032Z Dynamo produced 2 graphs covering 692 ops with 6 graph breaks (5 unique) 2024-06-26T08:32:02.5376128Z 2024-06-26T08:32:03.3829750Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:32:03.3830228Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:32:03.3833640Z cuda train tinynet_a 2024-06-26T08:33:50.2564474Z pass 2024-06-26T08:33:50.2645219Z TIMING: entire_frame_compile:99.45433 code_gen:22.87125 inductor_compile:57.97365 backend_compile:87.86979 2024-06-26T08:33:50.2647200Z STATS: call_* op count: 435 | FakeTensor.__torch_dispatch__:14091 | FakeTensorMode.__torch_dispatch__:83778 | attempt fast:3260 | fast is_contiguous:3260 | ProxyTorchDispatchMode.__torch_dispatch__:17283 2024-06-26T08:33:50.2648575Z Dynamo produced 2 graphs covering 435 ops with 6 graph breaks (5 unique) 2024-06-26T08:33:58.5938115Z 2024-06-26T08:33:59.4677417Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:33:59.4677882Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:33:59.4678364Z cuda train tnt_s_patch16_224 2024-06-26T08:36:50.9777340Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T08:36:50.9779675Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 371, in torch_dynamo_resume_in_forward_and_backward_pass_at_369 2024-06-26T08:36:50.9781577Z pred = mod(*cloned_inputs) 2024-06-26T08:36:50.9782754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:36:50.9783628Z return forward_call(*args, **kwargs) 2024-06-26T08:36:50.9784537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/tnt.py", line 333, in forward 2024-06-26T08:36:50.9785446Z x = self.forward_features(x) 2024-06-26T08:36:50.9786537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/tnt.py", line 309, in forward_features 2024-06-26T08:36:50.9787438Z pixel_embed = self.pixel_embed(x, self.pixel_pos) 2024-06-26T08:36:50.9788444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:36:50.9789466Z return forward_call(*args, **kwargs) 2024-06-26T08:36:50.9790327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/tnt.py", line 182, in forward 2024-06-26T08:36:50.9791144Z x = self.unfold(x) 2024-06-26T08:36:50.9791370Z 2024-06-26T08:36:57.1004716Z W0626 08:36:57.099000 139906097341056 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:38:28.0704195Z pass 2024-06-26T08:38:28.0942952Z TIMING: entire_frame_compile:254.51689 code_gen:52.17072 inductor_compile:137.06691 backend_compile:222.65771 2024-06-26T08:38:28.0945165Z STATS: call_* op count: 2000 | FakeTensor.__torch_dispatch__:32624 | FakeTensorMode.__torch_dispatch__:237350 | attempt fast:6398 | fast is_contiguous:6182 | slow no contiguity match:216 | ProxyTorchDispatchMode.__torch_dispatch__:49260 2024-06-26T08:38:28.0946724Z Dynamo produced 3 graphs covering 2000 ops with 7 graph breaks (5 unique) 2024-06-26T08:38:45.2074117Z 2024-06-26T08:38:46.4279591Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:38:46.4280064Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:38:46.4285194Z cuda train twins_pcpvt_base 2024-06-26T08:41:37.8549237Z W0626 08:41:37.854000 139764197114496 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:44:00.1301900Z pass 2024-06-26T08:44:00.1601979Z TIMING: entire_frame_compile:292.48415 code_gen:71.87045 inductor_compile:161.3196 backend_compile:252.03291 2024-06-26T08:44:00.1604854Z STATS: call_* op count: 2583 | FakeTensor.__torch_dispatch__:47391 | FakeTensorMode.__torch_dispatch__:299762 | attempt fast:7156 | fast is_contiguous:7032 | slow no contiguity match:118 | ProxyTorchDispatchMode.__torch_dispatch__:59071 | fast channels_last:6 2024-06-26T08:44:00.1606508Z Dynamo produced 3 graphs covering 2583 ops with 7 graph breaks (5 unique) 2024-06-26T08:44:17.7876898Z 2024-06-26T08:44:18.8588502Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:44:18.8588971Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:44:18.8589406Z cuda train visformer_small 2024-06-26T08:45:20.8611835Z W0626 08:45:20.860000 139866463994496 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:45:57.2604537Z pass 2024-06-26T08:45:57.2850530Z TIMING: entire_frame_compile:90.22231 code_gen:19.35436 inductor_compile:48.58195 backend_compile:78.23976 2024-06-26T08:45:57.2853220Z STATS: call_* op count: 715 | FakeTensor.__torch_dispatch__:13912 | FakeTensorMode.__torch_dispatch__:82479 | attempt fast:2305 | fast is_contiguous:2305 | ProxyTorchDispatchMode.__torch_dispatch__:15868 2024-06-26T08:45:57.2854669Z Dynamo produced 3 graphs covering 715 ops with 7 graph breaks (5 unique) 2024-06-26T08:46:04.9438885Z 2024-06-26T08:46:06.3492787Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:46:06.3493305Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:46:06.3493858Z cuda train vit_base_patch16_224 2024-06-26T08:47:01.5340763Z W0626 08:47:01.533000 140158203626112 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:47:41.7363081Z pass 2024-06-26T08:47:41.7852841Z TIMING: entire_frame_compile:85.58976 code_gen:21.99803 inductor_compile:48.87034 backend_compile:74.5452 2024-06-26T08:47:41.7855480Z STATS: call_* op count: 855 | FakeTensor.__torch_dispatch__:13987 | FakeTensorMode.__torch_dispatch__:89467 | attempt fast:2319 | fast is_contiguous:2319 | ProxyTorchDispatchMode.__torch_dispatch__:17807 2024-06-26T08:47:41.7856837Z Dynamo produced 3 graphs covering 855 ops with 7 graph breaks (5 unique) 2024-06-26T08:47:49.3021297Z 2024-06-26T08:47:50.3187083Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:47:50.3187590Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:47:50.3188035Z cuda train volo_d1_224 2024-06-26T08:49:24.5455492Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T08:49:24.5456999Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 371, in torch_dynamo_resume_in_forward_and_backward_pass_at_369 2024-06-26T08:49:24.5458262Z pred = mod(*cloned_inputs) 2024-06-26T08:49:24.5460665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:49:24.5461714Z return forward_call(*args, **kwargs) 2024-06-26T08:49:24.5462848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 725, in forward 2024-06-26T08:49:24.5463690Z x = self.forward_features(x) 2024-06-26T08:49:24.5464585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 698, in forward_features 2024-06-26T08:49:24.5465421Z x = self.forward_tokens(x) 2024-06-26T08:49:24.5466292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 625, in forward_tokens 2024-06-26T08:49:24.5467103Z x = block(x) 2024-06-26T08:49:24.5467922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:49:24.5468797Z return forward_call(*args, **kwargs) 2024-06-26T08:49:24.5469655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 134, in forward 2024-06-26T08:49:24.5470538Z x = x + self.drop_path(self.attn(self.norm1(x))) 2024-06-26T08:49:24.5471526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:49:24.5472381Z return forward_call(*args, **kwargs) 2024-06-26T08:49:24.5473237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 88, in forward 2024-06-26T08:49:24.5474329Z x = F.fold(x, output_size=(H, W), kernel_size=self.kernel_size, padding=self.padding, stride=self.stride) 2024-06-26T08:49:24.5474946Z 2024-06-26T08:50:10.7622531Z W0626 08:50:10.761000 140474235609728 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:51:18.0839482Z pass 2024-06-26T08:51:18.0967154Z TIMING: entire_frame_compile:198.14874 code_gen:51.2258 inductor_compile:110.72588 backend_compile:175.17275 2024-06-26T08:51:18.0969919Z STATS: call_* op count: 1454 | FakeTensor.__torch_dispatch__:23400 | FakeTensorMode.__torch_dispatch__:177479 | attempt fast:4516 | fast is_contiguous:4234 | slow no contiguity match:282 | ProxyTorchDispatchMode.__torch_dispatch__:35802 2024-06-26T08:51:18.0971467Z Dynamo produced 3 graphs covering 1454 ops with 7 graph breaks (5 unique) 2024-06-26T08:51:30.7022764Z 2024-06-26T08:51:33.8522066Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:51:33.8522555Z loading model: 0it [00:03, ?it/s] 2024-06-26T08:51:33.8526430Z cuda train xcit_large_24_p8_224 2024-06-26T08:51:33.9411645Z WARNING:common:fp64 golden ref were not generated for xcit_large_24_p8_224. Setting accuracy check to cosine 2024-06-26T08:55:46.4940046Z W0626 08:55:46.493000 139797164540544 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:58:34.8142318Z pass_due_to_skip 2024-06-26T08:58:34.8560184Z TIMING: entire_frame_compile:390.43942 code_gen:85.66301 inductor_compile:208.42995 backend_compile:341.20814 2024-06-26T08:58:34.8561974Z STATS: call_* op count: 3216 | FakeTensor.__torch_dispatch__:60933 | FakeTensorMode.__torch_dispatch__:393015 | attempt fast:10604 | fast is_contiguous:7432 | slow no contiguity match:2234 | fast channels_last:938 | ProxyTorchDispatchMode.__torch_dispatch__:78435 2024-06-26T08:58:34.8564244Z Dynamo produced 3 graphs covering 3216 ops with 7 graph breaks (5 unique) 2024-06-26T08:58:55.2584135Z accuracy pass_rate=96.77% 2024-06-26T08:58:55.2588044Z calls_captured gmean=759.70x mean=959.806x 2024-06-26T08:58:55.2592493Z unique_graphs gmean=2.49x mean=2.548x 2024-06-26T08:58:55.2596003Z graph_breaks gmean=6.39x mean=6.419x 2024-06-26T08:58:55.2600335Z unique_graph_breaks gmean=4.96x mean=4.968x 2024-06-26T08:58:55.2604033Z autograd_captures gmean=0.00x mean=0.000x 2024-06-26T08:58:55.2608538Z autograd_compiles gmean=0.00x mean=0.000x 2024-06-26T08:58:55.2612685Z cudagraph_skips gmean=0.00x mean=0.097x 2024-06-26T08:58:56.9037397Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy//dynamic_inductor_timm_training.csv 2024-06-26T08:58:57.1409739Z mixnet_l PASS 2024-06-26T08:58:57.1414694Z mnasnet_100 PASS 2024-06-26T08:58:57.1419022Z mobilenetv2_100 PASS 2024-06-26T08:58:57.1423359Z mobilenetv3_large_100 PASS 2024-06-26T08:58:57.1427943Z mobilevit_s PASS 2024-06-26T08:58:57.1432423Z nfnet_l0 PASS 2024-06-26T08:58:57.1436840Z pit_b_224 PASS 2024-06-26T08:58:57.1441180Z pnasnet5large PASS 2024-06-26T08:58:57.1445529Z poolformer_m36 PASS 2024-06-26T08:58:57.1449975Z regnety_002 PASS 2024-06-26T08:58:57.1454495Z repvgg_a2 PASS 2024-06-26T08:58:57.1459003Z res2net101_26w_4s PASS 2024-06-26T08:58:57.1463327Z res2net50_14w_8s PASS 2024-06-26T08:58:57.1467673Z res2next50 PASS 2024-06-26T08:58:57.1472036Z resmlp_12_224 PASS 2024-06-26T08:58:57.1476490Z resnest101e PASS 2024-06-26T08:58:57.1480920Z rexnet_100 PASS 2024-06-26T08:58:57.1485556Z sebotnet33ts_256 PASS 2024-06-26T08:58:57.1490015Z selecsls42b PASS 2024-06-26T08:58:57.1494709Z spnasnet_100 PASS 2024-06-26T08:58:57.1499309Z swin_base_patch4_window7_224 PASS 2024-06-26T08:58:57.1503590Z swsl_resnext101_32x16d PASS 2024-06-26T08:58:57.1507952Z tf_efficientnet_b0 PASS 2024-06-26T08:58:57.1512301Z tf_mixnet_l PASS 2024-06-26T08:58:57.1516655Z tinynet_a PASS 2024-06-26T08:58:57.1521029Z tnt_s_patch16_224 PASS 2024-06-26T08:58:57.1525418Z twins_pcpvt_base PASS 2024-06-26T08:58:57.1529769Z visformer_small PASS 2024-06-26T08:58:57.1534156Z vit_base_patch16_224 PASS 2024-06-26T08:58:57.1538966Z volo_d1_224 PASS 2024-06-26T08:58:57.1543301Z xcit_large_24_p8_224 XFAIL 2024-06-26T08:58:57.1927155Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy//dynamic_inductor_timm_training.csv 2024-06-26T08:58:57.4483995Z mixnet_l PASS 2024-06-26T08:58:57.4488276Z mnasnet_100 PASS 2024-06-26T08:58:57.4494102Z mobilenetv2_100 PASS 2024-06-26T08:58:57.4499498Z mobilenetv3_large_100 PASS 2024-06-26T08:58:57.4504569Z mobilevit_s PASS 2024-06-26T08:58:57.4509536Z nfnet_l0 PASS 2024-06-26T08:58:57.4514554Z pit_b_224 PASS 2024-06-26T08:58:57.4519632Z pnasnet5large PASS 2024-06-26T08:58:57.4524128Z poolformer_m36 PASS 2024-06-26T08:58:57.4528838Z regnety_002 PASS 2024-06-26T08:58:57.4534022Z repvgg_a2 PASS 2024-06-26T08:58:57.4538839Z res2net101_26w_4s PASS 2024-06-26T08:58:57.4543651Z res2net50_14w_8s PASS 2024-06-26T08:58:57.4548338Z res2next50 PASS 2024-06-26T08:58:57.4553273Z resmlp_12_224 PASS 2024-06-26T08:58:57.4557944Z resnest101e PASS 2024-06-26T08:58:57.4562710Z rexnet_100 PASS 2024-06-26T08:58:57.4567700Z sebotnet33ts_256 PASS 2024-06-26T08:58:57.4572496Z selecsls42b PASS 2024-06-26T08:58:57.4578169Z spnasnet_100 PASS 2024-06-26T08:58:57.4583091Z swin_base_patch4_window7_224 PASS 2024-06-26T08:58:57.4587985Z swsl_resnext101_32x16d PASS 2024-06-26T08:58:57.4592974Z tf_efficientnet_b0 PASS 2024-06-26T08:58:57.4597979Z tf_mixnet_l PASS 2024-06-26T08:58:57.4602782Z tinynet_a PASS 2024-06-26T08:58:57.4607582Z tnt_s_patch16_224 PASS 2024-06-26T08:58:57.4612376Z twins_pcpvt_base PASS 2024-06-26T08:58:57.4618006Z visformer_small PASS 2024-06-26T08:58:57.4622660Z vit_base_patch16_224 PASS 2024-06-26T08:58:57.4627415Z volo_d1_224 PASS 2024-06-26T08:58:57.4632429Z xcit_large_24_p8_224 PASS 2024-06-26T08:58:57.5010371Z + cleanup_workspace 2024-06-26T08:58:57.5011369Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-06-26T08:58:57.5013002Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-06-26T08:58:57.5014196Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-06-26T08:58:57.5015023Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-06-26T08:58:57.5015863Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-06-26T08:58:57.5016774Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-06-26T08:58:57.5017478Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-06-26T08:58:58.2109600Z ##[group]Run cat test/**/*_toprint.log || true 2024-06-26T08:58:58.2110122Z cat test/**/*_toprint.log || true 2024-06-26T08:58:58.2117766Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:58:58.2118269Z env: 2024-06-26T08:58:58.2118556Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:58:58.2119015Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:58:58.2119761Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:58:58.2120405Z ##[endgroup] 2024-06-26T08:58:58.2184596Z cat: test/**/*_toprint.log: No such file or directory 2024-06-26T08:58:58.2238179Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-06-26T08:58:58.2238643Z kill "$MONITOR_SCRIPT_PID" 2024-06-26T08:58:58.2246221Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:58:58.2246877Z env: 2024-06-26T08:58:58.2247224Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:58:58.2247690Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:58:58.2248440Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:58:58.2249110Z MONITOR_SCRIPT_PID: 26259 2024-06-26T08:58:58.2249463Z ##[endgroup] 2024-06-26T08:58:58.2440271Z Prepare all required actions 2024-06-26T08:58:58.2440884Z Getting action download info 2024-06-26T08:58:58.4130352Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-06-26T08:58:58.5703157Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-06-26T08:58:58.5703632Z with: 2024-06-26T08:58:58.5704176Z file-suffix: test-dynamic_inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688911835 2024-06-26T08:58:58.5704853Z s3-bucket: gha-artifacts 2024-06-26T08:58:58.5705201Z env: 2024-06-26T08:58:58.5705472Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:58:58.5705937Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:58:58.5706686Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:58:58.5707328Z ##[endgroup] 2024-06-26T08:58:58.5732231Z ##[group]Run # Remove any previous test jsons if they exist 2024-06-26T08:58:58.5732928Z # Remove any previous test jsons if they exist 2024-06-26T08:58:58.5733458Z rm -f test-jsons-*.zip 2024-06-26T08:58:58.5734072Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-06-26T08:58:58.5745268Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:58:58.5745780Z env: 2024-06-26T08:58:58.5746065Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:58:58.5746519Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:58:58.5747270Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:58:58.5748187Z FILE_SUFFIX: test-dynamic_inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688911835 2024-06-26T08:58:58.5748837Z ##[endgroup] 2024-06-26T08:58:58.5924977Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-06-26T08:58:58.5954225Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-06-26T08:58:58.5955036Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-06-26T08:58:58.5960841Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-06-26T08:58:58.5986916Z ##[group]Run # Remove any previous test reports if they exist 2024-06-26T08:58:58.5987574Z # Remove any previous test reports if they exist 2024-06-26T08:58:58.5988114Z rm -f test-reports-*.zip 2024-06-26T08:58:58.5988718Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-06-26T08:58:58.5996358Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:58:58.5996862Z env: 2024-06-26T08:58:58.5997147Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:58:58.5997614Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:58:58.5998365Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:58:58.5999283Z FILE_SUFFIX: test-dynamic_inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688911835 2024-06-26T08:58:58.5999937Z ##[endgroup] 2024-06-26T08:58:58.6164992Z adding: test/test-reports/inference_timm_models.csv (deflated 68%) 2024-06-26T08:58:58.6165877Z adding: test/test-reports/training_timm_models.csv (deflated 66%) 2024-06-26T08:58:58.6169623Z adding: test/test-reports/training_timm_model_graph_breaks.csv (deflated 97%) 2024-06-26T08:58:58.6170587Z adding: test/test-reports/training_timm_model_graph_break_deduped.csv (deflated 74%) 2024-06-26T08:58:58.6196150Z ##[group]Run # Remove any previous usage logs if they exist 2024-06-26T08:58:58.6196787Z # Remove any previous usage logs if they exist 2024-06-26T08:58:58.6197294Z rm -f logs-*.zip 2024-06-26T08:58:58.6197977Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-06-26T08:58:58.6198781Z # so check to see if the file exists first 2024-06-26T08:58:58.6199293Z if [ -f 'usage_log.txt' ]; then 2024-06-26T08:58:58.6199830Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-06-26T08:58:58.6200324Z fi 2024-06-26T08:58:58.6200806Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-06-26T08:58:58.6201491Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-06-26T08:58:58.6201991Z fi 2024-06-26T08:58:58.6209198Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:58:58.6209703Z env: 2024-06-26T08:58:58.6209978Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:58:58.6210441Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:58:58.6211189Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:58:58.6212394Z FILE_SUFFIX: test-dynamic_inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688911835 2024-06-26T08:58:58.6213073Z ##[endgroup] 2024-06-26T08:58:58.6295578Z adding: usage_log.txt (deflated 92%) 2024-06-26T08:58:58.6359859Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-06-26T08:58:58.6360587Z # Remove any previous debugging artifacts if they exist 2024-06-26T08:58:58.6361158Z rm -f debug-*.zip 2024-06-26T08:58:58.6361563Z if [ -d 'test/debug' ]; then 2024-06-26T08:58:58.6362068Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-06-26T08:58:58.6362551Z fi 2024-06-26T08:58:58.6369353Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:58:58.6369863Z env: 2024-06-26T08:58:58.6370136Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:58:58.6370597Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:58:58.6371355Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:58:58.6372564Z FILE_SUFFIX: test-dynamic_inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688911835 2024-06-26T08:58:58.6373219Z ##[endgroup] 2024-06-26T08:58:58.6446799Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T08:58:58.6447232Z with: 2024-06-26T08:58:58.6447514Z s3-bucket: gha-artifacts 2024-06-26T08:58:58.6447952Z s3-prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:58:58.6448426Z retention-days: 14 2024-06-26T08:58:58.6448767Z if-no-files-found: warn 2024-06-26T08:58:58.6449129Z path: test-jsons-*.zip 2024-06-26T08:58:58.6449471Z name: artifact 2024-06-26T08:58:58.6449771Z region: us-east-1 2024-06-26T08:58:58.6450068Z env: 2024-06-26T08:58:58.6450341Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:58:58.6450794Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:58:58.6451535Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:58:58.6452363Z ##[endgroup] 2024-06-26T08:58:58.9475382Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T08:58:58.9476628Z With the provided path, there will be 1 file uploaded 2024-06-26T08:58:58.9477914Z Uploading to s3 prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:58:58.9502445Z Starting upload of test-jsons-test-dynamic_inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688911835.zip 2024-06-26T08:58:59.0406490Z Finished upload of test-jsons-test-dynamic_inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688911835.zip 2024-06-26T08:58:59.0535353Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T08:58:59.0535805Z with: 2024-06-26T08:58:59.0536082Z s3-bucket: gha-artifacts 2024-06-26T08:58:59.0536513Z s3-prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:58:59.0536992Z retention-days: 14 2024-06-26T08:58:59.0537326Z if-no-files-found: error 2024-06-26T08:58:59.0537699Z path: test-reports-*.zip 2024-06-26T08:58:59.0538058Z name: artifact 2024-06-26T08:58:59.0538364Z region: us-east-1 2024-06-26T08:58:59.0538657Z env: 2024-06-26T08:58:59.0538936Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:58:59.0539392Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:58:59.0540142Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:58:59.0540793Z ##[endgroup] 2024-06-26T08:58:59.3588177Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T08:58:59.3589208Z With the provided path, there will be 1 file uploaded 2024-06-26T08:58:59.3589977Z Uploading to s3 prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:58:59.3615971Z Starting upload of test-reports-test-dynamic_inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688911835.zip 2024-06-26T08:58:59.4689931Z Finished upload of test-reports-test-dynamic_inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688911835.zip 2024-06-26T08:58:59.4817918Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T08:58:59.4818365Z with: 2024-06-26T08:58:59.4818651Z s3-bucket: gha-artifacts 2024-06-26T08:58:59.4819084Z s3-prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:58:59.4819566Z retention-days: 14 2024-06-26T08:58:59.4819901Z if-no-files-found: ignore 2024-06-26T08:58:59.4820268Z path: logs-*.zip 2024-06-26T08:58:59.4820581Z name: artifact 2024-06-26T08:58:59.4820885Z region: us-east-1 2024-06-26T08:58:59.4821186Z env: 2024-06-26T08:58:59.4821466Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:58:59.4821941Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:58:59.4822700Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:58:59.4823351Z ##[endgroup] 2024-06-26T08:58:59.7945820Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T08:58:59.7946442Z With the provided path, there will be 1 file uploaded 2024-06-26T08:58:59.7947073Z Uploading to s3 prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:58:59.7974152Z Starting upload of logs-test-dynamic_inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688911835.zip 2024-06-26T08:58:59.8792695Z Finished upload of logs-test-dynamic_inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_26688911835.zip 2024-06-26T08:58:59.8924008Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T08:58:59.8924451Z with: 2024-06-26T08:58:59.8924744Z s3-bucket: gha-artifacts 2024-06-26T08:58:59.8925180Z s3-prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:58:59.8925670Z retention-days: 14 2024-06-26T08:58:59.8926016Z if-no-files-found: ignore 2024-06-26T08:58:59.8926381Z path: debug-*.zip 2024-06-26T08:58:59.8926698Z name: artifact 2024-06-26T08:58:59.8927003Z region: us-east-1 2024-06-26T08:58:59.8927305Z env: 2024-06-26T08:58:59.8927581Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:58:59.8928040Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:58:59.8928784Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:58:59.8929432Z ##[endgroup] 2024-06-26T08:59:00.1967648Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-06-26T08:59:00.2105156Z ##[group]Run # shellcheck disable=SC2156 2024-06-26T08:59:00.2105671Z # shellcheck disable=SC2156 2024-06-26T08:59:00.2106496Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-06-26T08:59:00.2114424Z shell: /usr/bin/bash -e {0} 2024-06-26T08:59:00.2114781Z env: 2024-06-26T08:59:00.2115082Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:59:00.2115549Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:59:00.2116296Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:59:00.2116950Z ##[endgroup] 2024-06-26T08:59:00.3739934Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-06-26T08:59:00.3740503Z with: 2024-06-26T08:59:00.3740767Z env: 2024-06-26T08:59:00.3741048Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:59:00.3741505Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:59:00.3742253Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:59:00.3742900Z ##[endgroup] 2024-06-26T08:59:00.3760659Z ##[group]Run set -eou pipefail 2024-06-26T08:59:00.3761072Z set -eou pipefail 2024-06-26T08:59:00.3761427Z  2024-06-26T08:59:00.3761946Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-06-26T08:59:00.3762738Z for _ in $(seq 1440); do 2024-06-26T08:59:00.3763219Z  # Break if no ssh session exists anymore 2024-06-26T08:59:00.3763724Z  if [ "$(who)" = "" ]; then 2024-06-26T08:59:00.3764139Z  break 2024-06-26T08:59:00.3764489Z  fi 2024-06-26T08:59:00.3764787Z  echo "." 2024-06-26T08:59:00.3765127Z  sleep 5 2024-06-26T08:59:00.3765450Z done 2024-06-26T08:59:00.3773014Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:59:00.3773598Z env: 2024-06-26T08:59:00.3773880Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:59:00.3774339Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:59:00.3775084Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:59:00.3775755Z ##[endgroup] 2024-06-26T08:59:00.3795620Z Holding runner for 2 hours until all ssh sessions have logged out 2024-06-26T08:59:00.3832256Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T08:59:00.3833041Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T08:59:00.3833644Z # shellcheck disable=SC2046 2024-06-26T08:59:00.3834133Z docker stop $(docker ps -q) || true 2024-06-26T08:59:00.3834623Z # Prune all of the docker images 2024-06-26T08:59:00.3835083Z docker system prune -af 2024-06-26T08:59:00.3842237Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:59:00.3842746Z env: 2024-06-26T08:59:00.3843024Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:59:00.3843490Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:59:00.3844247Z DOCKER_CONTAINER_ID: 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:59:00.3844897Z ##[endgroup] 2024-06-26T08:59:00.7001830Z 9c6f49d01630 2024-06-26T08:59:01.4455868Z Deleted Containers: 2024-06-26T08:59:01.4456415Z 9c6f49d016309f4914aeaebba36d1b7424b35d0e0fb4f7d9c8a2823aba3f6627 2024-06-26T08:59:01.4456906Z 2024-06-26T08:59:06.5162149Z Deleted Images: 2024-06-26T08:59:06.5163979Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T08:59:06.5166654Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb 2024-06-26T08:59:06.5168594Z deleted: sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd 2024-06-26T08:59:06.5169446Z deleted: sha256:754ead7012f0b7691885a9660f02e80be6cee8c5665486fb995c1fb06adb6e92 2024-06-26T08:59:06.5170287Z deleted: sha256:fbe4496d8830e11be86d8f222522e7956f32f85bfc06b05ea53961f34a58af74 2024-06-26T08:59:06.5171121Z deleted: sha256:1c83814aa1a1a7e0cc998e3479bdab8e8766b975560cbf00690acf39ed899a80 2024-06-26T08:59:06.5171970Z deleted: sha256:e9ea8e61942c7f8ae4aab97b8d102b01d6bb4b6cf2a605808cd34130a17d8a0f 2024-06-26T08:59:06.5173032Z deleted: sha256:acc5fc32588cd95e082a6ce35e48d0f303ae8f96e3ab65c4e375b85e1f3c4a92 2024-06-26T08:59:06.5173921Z deleted: sha256:a93e53400904d29e4185d3be942ea34f19572ea438962788ae4e9b3d9bf4c498 2024-06-26T08:59:06.5174902Z deleted: sha256:c97092b88b5e4c77c68bbc4b51afab64e38719af1ff58013ec49a696cdd40a6e 2024-06-26T08:59:06.5175738Z deleted: sha256:b132889c90d67ffa53655846603e3fcc2372c23f04dbdbc54690a39d304eb89b 2024-06-26T08:59:06.5176552Z deleted: sha256:915d54081ed607710624abe3f2490114c745858831f4fc13396bd3b4b9a121bc 2024-06-26T08:59:06.5177370Z deleted: sha256:bfa8d446c498c87f535aef102695564de1b22c755e6c63944ae2f5073a2b31ec 2024-06-26T08:59:06.5178190Z deleted: sha256:e71460fb363c29e70bbe357d680aaa7ae694f094ef2f2713609158ad90611744 2024-06-26T08:59:06.5179016Z deleted: sha256:84079bb1b4ee1d0119ae3d3e701de460a1324912297c675425c9a36d30fcdb0f 2024-06-26T08:59:06.5179845Z deleted: sha256:d86ac3ca148d4758cb03b493fdd49109d8f1a2c7c85c4b755493c3c4529553e5 2024-06-26T08:59:06.5180799Z deleted: sha256:532b3ab97c44a673a0d5f62ee46fa6a83248a84de8877754a612355be56ed0a3 2024-06-26T08:59:06.5181621Z deleted: sha256:617e2a35f25ae24500c88c0c9b8475f0687fbcbed3d200013b4c99c6d8941aff 2024-06-26T08:59:06.5182451Z deleted: sha256:b88120bcd5a26a660c09c5df13176e0ef098fb2c8cdb57d74ab0796a3d1194d6 2024-06-26T08:59:06.5183275Z deleted: sha256:c016b2e7764735243205fb76fa9a273bb3dd4bf05532288e3a613ce022131ac3 2024-06-26T08:59:06.5184112Z deleted: sha256:bcaca462c390deddd2995de2d245a7495e7d2e1e94f0edad30a573c83d0588fd 2024-06-26T08:59:06.5184946Z deleted: sha256:9e260eba003d5c25d7b410e5c4c3093845d9e46611896840901c689d7fbf8ee6 2024-06-26T08:59:06.5185753Z deleted: sha256:3e62b86f8b911a9b59c62d55f063e71683ae6a01c22418db5d27930712adaad6 2024-06-26T08:59:06.5186602Z deleted: sha256:905920ccb1d88449b8dde686c34a4b2a8b6f59e78ac44916b189ea3593b9fbbb 2024-06-26T08:59:06.5187419Z deleted: sha256:961201237e03acd5061406238dd7a3c1343ac888d7583212cd0ab5ae1b4f0540 2024-06-26T08:59:06.5188249Z deleted: sha256:70047a2aac62159fa28fb4dbd4567ec2ffbab68fe046967e09acb0eb3e5cb9cb 2024-06-26T08:59:06.5189091Z deleted: sha256:1ec680ad781436ae65fa987f5fa1ed032ecd5c82a6e35eea6707de501f62c0f4 2024-06-26T08:59:06.5189927Z deleted: sha256:ef859623a50e85abf6e6c1dbd6188aa765a66cc7b2bd618a1cbe4825b7b2bd91 2024-06-26T08:59:06.5190780Z deleted: sha256:404ade5cacff59773b93ab438bfa64cd3a171d07f6f168bdc4a5d4bfa0b43526 2024-06-26T08:59:06.5191627Z deleted: sha256:5b27de6ee3d9f2f18217bf18dec725f7b4954c0e48c85d35752634c22d62209b 2024-06-26T08:59:06.5192461Z deleted: sha256:498e39c575b3fa069acaac7e46b91b4e54ae3a05f2ada5c4347091f346f25d4c 2024-06-26T08:59:06.5193418Z deleted: sha256:1405919d2694d65669b1f4c842282021afe89208ce86570e96464903733da2d0 2024-06-26T08:59:06.5194335Z deleted: sha256:d5b41e5d70896c0472fc3e6da0516a17d89cd6635dc75010ecddc31ce9690fad 2024-06-26T08:59:06.5195172Z deleted: sha256:18e9379b62b9d0bb40293d28a652eafaee66a499f66f0776bf14e37a513101b7 2024-06-26T08:59:06.5196016Z deleted: sha256:a781f80ed00ff2926a699fd01a309bb0b4fb7d5655eb652c205bb9c7f811035f 2024-06-26T08:59:06.5196837Z deleted: sha256:c0430973b7cf759075330ee29c79bfd1312c91e6033d9dd883b4b1c29743a171 2024-06-26T08:59:06.5197682Z deleted: sha256:e733b3d66efb9342e5a283376c0e1ddffa47a42fecc26e61ecaf0a3472885d97 2024-06-26T08:59:06.5198526Z deleted: sha256:8929c0b2047f74b84c82c39eff9077a92ae1a828cde00c9ab78fb3e0ed5e0283 2024-06-26T08:59:06.5199354Z deleted: sha256:f5b56155fb58a0a60fe6391d3581cf91610ed69d4f0a321257c19944da296a52 2024-06-26T08:59:06.5200171Z deleted: sha256:b54d2cb355238d1aa91e58df93064447b80f53706d1ca62823acc834f5ac54fe 2024-06-26T08:59:06.5200998Z deleted: sha256:e7311285af9eeddb51aa5629aa98386e89f1456ff32843b6aee6443247edf66c 2024-06-26T08:59:06.5201826Z deleted: sha256:67657ceec347da8dc8085d5731633efeac115ba677fa4b7c5966c4260f7abab0 2024-06-26T08:59:06.5202661Z deleted: sha256:b496e50409dbf414930fc9b4437aa0aa4ce27bc7840e050ae459aaffcf9aca86 2024-06-26T08:59:06.5203508Z deleted: sha256:8f5276170cc4a60d36ad7b964a502306b29546254cedcbcb101ea9123497d2c0 2024-06-26T08:59:06.5204645Z deleted: sha256:53d524d1564745e96a7e515462171b73184d9f1ff5454b32fcfbbca39d361467 2024-06-26T08:59:06.5205654Z deleted: sha256:2acbadb00d69b6aca575f04baba4ea26ae2b5c2d85f09945004ad93a78a8eb75 2024-06-26T08:59:06.5206615Z deleted: sha256:02665efeff878aeb603c1edb2e21cb700c171c025d64f5aae1b2b20b12f3964a 2024-06-26T08:59:06.5207470Z deleted: sha256:19bd23e9a51dfe883a8b8d2ff6d66ff1b3476b8912856406bb79b81762aa6d4b 2024-06-26T08:59:06.5208303Z deleted: sha256:73aa8637bf9299d6298dabd858480c274a62117388f2ca1d5e9eb15c6d1e4155 2024-06-26T08:59:06.5209131Z deleted: sha256:5f8968406305f348b1caebc0e035187d6e9965ace7b9d5f2f21114d37c9fe2e1 2024-06-26T08:59:06.5209964Z deleted: sha256:d827104914db0c503cf5c81f13e2456bf4efce42d4cf1f8d5d70838207278bee 2024-06-26T08:59:06.5210800Z deleted: sha256:867b219724f2f1aaf56f42e3e9f03a5d64bdca8a6a72407efd10328aeea5d510 2024-06-26T08:59:06.5211640Z deleted: sha256:b9ff2a7cceb9237b26bcb17728edf8a558eec5a76f63626e84f80541dfc0f1d4 2024-06-26T08:59:06.5212722Z deleted: sha256:be73f509106819007ce2b0f5a1301ac56ae583cb6c1fb69bee93efa0840368f1 2024-06-26T08:59:06.5213649Z deleted: sha256:2a11c07c915c3fdbb4e73096dea73ea950c970d50fe1a5764060bd62b7b7cf2c 2024-06-26T08:59:06.5214493Z deleted: sha256:5e9989622e2abc1c72f8184343507b45d75e4ff971edb68b96cbcd75ae484ff5 2024-06-26T08:59:06.5215332Z deleted: sha256:05ce642bd9cf5a9205bff107df28b1d6cef7d5f6ab3ea944d28c032ec61270df 2024-06-26T08:59:06.5216168Z deleted: sha256:017702617f6c07b73f2088678f4710b794cfe897a213a5cf122a0e2519eae0ac 2024-06-26T08:59:06.5216990Z deleted: sha256:44b3b37bb1ca39dc5cf356de41a23a5c12f605a1992813e408087fb2b0ad6d1c 2024-06-26T08:59:06.5217824Z deleted: sha256:8c387982e9f27db357a1d149dc149cabc0bd04668ad6956387a5072682dada67 2024-06-26T08:59:06.5218661Z deleted: sha256:10d415dca4e3fbb88a747b40317f3c6333c041ba8f73855f67fb9fe981fae5ee 2024-06-26T08:59:06.5219496Z deleted: sha256:6c65469bf0f53dcd625faefe363b8f3136c08388a6cf64d9158e8576e4ee89b1 2024-06-26T08:59:06.5220353Z deleted: sha256:027db5d8b76deeefbd8c34b485ab7ff14ed2a3e6800425c1ce0d77c2e9fdd4ab 2024-06-26T08:59:06.5221192Z deleted: sha256:24941234ad28cf5c8174d7cf18326c10b4e17f2c2137672ea0322df461597dd7 2024-06-26T08:59:06.5222026Z deleted: sha256:240ea2eebfa8519d6c2d88f4c99e66ce4e038d000f93ce8d03726c01d81180ef 2024-06-26T08:59:06.5222857Z deleted: sha256:09e823cb391a69f6a195e3ca8a41624b5f8b34eee14c6eb36277de814a91966e 2024-06-26T08:59:06.5223678Z deleted: sha256:730479d695169a6593bf0ac16b377491dce65175a6d158e012422367d04577fd 2024-06-26T08:59:06.5224495Z deleted: sha256:b0a30f9f7bd5459ff662453b9346a7ba8ea8db1191528345c1953eab4aa2f54e 2024-06-26T08:59:06.5225329Z deleted: sha256:75cefa87effaa3ad361c74642839923101f0f9982f814f3c9e2ed4ad52f2a008 2024-06-26T08:59:06.5226148Z deleted: sha256:6f97db87ba9512e615e871c6982000602982e1e5a5be334a56e7292251e84e00 2024-06-26T08:59:06.5226976Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-06-26T08:59:06.5227814Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-06-26T08:59:06.5228693Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-06-26T08:59:06.5229516Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-06-26T08:59:06.5230365Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-06-26T08:59:06.5231205Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-06-26T08:59:06.5232062Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-06-26T08:59:06.5232922Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-06-26T08:59:06.5233771Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-06-26T08:59:06.5234631Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-06-26T08:59:06.5235467Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-06-26T08:59:06.5235979Z 2024-06-26T08:59:06.5236142Z Total reclaimed space: 36.29GB 2024-06-26T08:59:06.5300449Z Post job cleanup. 2024-06-26T08:59:06.5348610Z Post job cleanup. 2024-06-26T08:59:06.6163612Z [command]/usr/bin/git version 2024-06-26T08:59:06.6200029Z git version 2.40.1 2024-06-26T08:59:06.6231365Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/611378b1-ade1-4760-b3cc-4c768928601d/.gitconfig' 2024-06-26T08:59:06.6242300Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/611378b1-ade1-4760-b3cc-4c768928601d' before making global git config changes 2024-06-26T08:59:06.6243565Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T08:59:06.6246918Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T08:59:06.6274872Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T08:59:06.6304916Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T08:59:06.6523349Z Entering 'android/libs/fbjni' 2024-06-26T08:59:06.6561691Z Entering 'third_party/FP16' 2024-06-26T08:59:06.6598652Z Entering 'third_party/FXdiv' 2024-06-26T08:59:06.6634836Z Entering 'third_party/NNPACK' 2024-06-26T08:59:06.6673379Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T08:59:06.6709873Z Entering 'third_party/XNNPACK' 2024-06-26T08:59:06.6757644Z Entering 'third_party/benchmark' 2024-06-26T08:59:06.6794749Z Entering 'third_party/cpp-httplib' 2024-06-26T08:59:06.6831553Z Entering 'third_party/cpuinfo' 2024-06-26T08:59:06.6868301Z Entering 'third_party/cudnn_frontend' 2024-06-26T08:59:06.6904314Z Entering 'third_party/cutlass' 2024-06-26T08:59:06.6945625Z Entering 'third_party/eigen' 2024-06-26T08:59:06.6983238Z Entering 'third_party/fbgemm' 2024-06-26T08:59:06.7019371Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T08:59:06.7053881Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T08:59:06.7087477Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T08:59:06.7124480Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T08:59:06.7156631Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T08:59:06.7190834Z Entering 'third_party/flatbuffers' 2024-06-26T08:59:06.7230343Z Entering 'third_party/fmt' 2024-06-26T08:59:06.7262648Z Entering 'third_party/foxi' 2024-06-26T08:59:06.7296249Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T08:59:06.7329403Z Entering 'third_party/gloo' 2024-06-26T08:59:06.7361830Z Entering 'third_party/googletest' 2024-06-26T08:59:06.7393795Z Entering 'third_party/ideep' 2024-06-26T08:59:06.7424523Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T08:59:06.7461427Z Entering 'third_party/ittapi' 2024-06-26T08:59:06.7493102Z Entering 'third_party/kineto' 2024-06-26T08:59:06.7525641Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T08:59:06.7560920Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T08:59:06.7595259Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T08:59:06.7630957Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T08:59:06.7663917Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T08:59:06.7694984Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T08:59:06.7727185Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T08:59:06.7759172Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T08:59:06.7794054Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T08:59:06.7828024Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T08:59:06.7862101Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T08:59:06.7893969Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T08:59:06.7928758Z Entering 'third_party/mimalloc' 2024-06-26T08:59:06.7961906Z Entering 'third_party/nccl/nccl' 2024-06-26T08:59:06.7993963Z Entering 'third_party/nlohmann' 2024-06-26T08:59:06.8028400Z Entering 'third_party/onnx' 2024-06-26T08:59:06.8075480Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T08:59:06.8114874Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T08:59:06.8150604Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T08:59:06.8185590Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T08:59:06.8221203Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T08:59:06.8255100Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T08:59:06.8287730Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T08:59:06.8323931Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T08:59:06.8356244Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T08:59:06.8388499Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T08:59:06.8421038Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T08:59:06.8456187Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T08:59:06.8491403Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T08:59:06.8540932Z Entering 'third_party/pocketfft' 2024-06-26T08:59:06.8576647Z Entering 'third_party/protobuf' 2024-06-26T08:59:06.8610357Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T08:59:06.8644415Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T08:59:06.8680509Z Entering 'third_party/psimd' 2024-06-26T08:59:06.8716937Z Entering 'third_party/pthreadpool' 2024-06-26T08:59:06.8750815Z Entering 'third_party/pybind11' 2024-06-26T08:59:06.8787386Z Entering 'third_party/python-peachpy' 2024-06-26T08:59:06.8819466Z Entering 'third_party/sleef' 2024-06-26T08:59:06.8854549Z Entering 'third_party/tensorpipe' 2024-06-26T08:59:06.8888032Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T08:59:06.8922447Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T08:59:06.8956679Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T08:59:06.8992401Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T08:59:06.9025754Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T08:59:06.9076960Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T08:59:06.9095417Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9103378Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-06-26T08:59:06.9131725Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T08:59:06.9342136Z Entering 'android/libs/fbjni' 2024-06-26T08:59:06.9364781Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9389267Z Entering 'third_party/FP16' 2024-06-26T08:59:06.9410610Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9435147Z Entering 'third_party/FXdiv' 2024-06-26T08:59:06.9456084Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9480229Z Entering 'third_party/NNPACK' 2024-06-26T08:59:06.9503385Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9526604Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T08:59:06.9551711Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9576772Z Entering 'third_party/XNNPACK' 2024-06-26T08:59:06.9597640Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9632819Z Entering 'third_party/benchmark' 2024-06-26T08:59:06.9654536Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9676575Z Entering 'third_party/cpp-httplib' 2024-06-26T08:59:06.9698586Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9722096Z Entering 'third_party/cpuinfo' 2024-06-26T08:59:06.9743465Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9767402Z Entering 'third_party/cudnn_frontend' 2024-06-26T08:59:06.9789420Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9814487Z Entering 'third_party/cutlass' 2024-06-26T08:59:06.9836524Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9866311Z Entering 'third_party/eigen' 2024-06-26T08:59:06.9892602Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9918388Z Entering 'third_party/fbgemm' 2024-06-26T08:59:06.9940156Z http.https://github.com/.extraheader 2024-06-26T08:59:06.9963674Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T08:59:06.9985388Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0008277Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T08:59:07.0031204Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0053677Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T08:59:07.0075165Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0102840Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T08:59:07.0123528Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0146636Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T08:59:07.0166965Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0191843Z Entering 'third_party/flatbuffers' 2024-06-26T08:59:07.0214065Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0239940Z Entering 'third_party/fmt' 2024-06-26T08:59:07.0262626Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0286141Z Entering 'third_party/foxi' 2024-06-26T08:59:07.0308288Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0332744Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T08:59:07.0355119Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0377167Z Entering 'third_party/gloo' 2024-06-26T08:59:07.0398675Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0421157Z Entering 'third_party/googletest' 2024-06-26T08:59:07.0443487Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0466220Z Entering 'third_party/ideep' 2024-06-26T08:59:07.0487679Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0510241Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T08:59:07.0531168Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0559742Z Entering 'third_party/ittapi' 2024-06-26T08:59:07.0582502Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0605536Z Entering 'third_party/kineto' 2024-06-26T08:59:07.0627233Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0651011Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T08:59:07.0672614Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0696956Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T08:59:07.0718433Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0742697Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T08:59:07.0763747Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0788357Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T08:59:07.0808601Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0832695Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T08:59:07.0854333Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0875819Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T08:59:07.0898097Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0923449Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T08:59:07.0945283Z http.https://github.com/.extraheader 2024-06-26T08:59:07.0968555Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T08:59:07.0990752Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1015413Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T08:59:07.1035265Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1059933Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T08:59:07.1079706Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1102051Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T08:59:07.1123312Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1146563Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T08:59:07.1167468Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1192172Z Entering 'third_party/mimalloc' 2024-06-26T08:59:07.1215453Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1238270Z Entering 'third_party/nccl/nccl' 2024-06-26T08:59:07.1259802Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1283289Z Entering 'third_party/nlohmann' 2024-06-26T08:59:07.1304723Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1328325Z Entering 'third_party/onnx' 2024-06-26T08:59:07.1349513Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1384254Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T08:59:07.1405236Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1429323Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T08:59:07.1449617Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1474118Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T08:59:07.1497018Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1521221Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T08:59:07.1542731Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1565500Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T08:59:07.1587283Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1610490Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T08:59:07.1632117Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1656148Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T08:59:07.1677667Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1700227Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T08:59:07.1720515Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1743783Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T08:59:07.1764754Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1788350Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T08:59:07.1809175Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1832903Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T08:59:07.1852695Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1877895Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T08:59:07.1899644Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1924981Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T08:59:07.1946782Z http.https://github.com/.extraheader 2024-06-26T08:59:07.1985307Z Entering 'third_party/pocketfft' 2024-06-26T08:59:07.2008308Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2032821Z Entering 'third_party/protobuf' 2024-06-26T08:59:07.2054170Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2080395Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T08:59:07.2101400Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2124149Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T08:59:07.2145750Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2170844Z Entering 'third_party/psimd' 2024-06-26T08:59:07.2192434Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2217469Z Entering 'third_party/pthreadpool' 2024-06-26T08:59:07.2239710Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2264515Z Entering 'third_party/pybind11' 2024-06-26T08:59:07.2286443Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2309566Z Entering 'third_party/python-peachpy' 2024-06-26T08:59:07.2330495Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2354203Z Entering 'third_party/sleef' 2024-06-26T08:59:07.2375154Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2399462Z Entering 'third_party/tensorpipe' 2024-06-26T08:59:07.2420569Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2443616Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T08:59:07.2464970Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2488067Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T08:59:07.2509561Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2532878Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T08:59:07.2554669Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2577549Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T08:59:07.2597857Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2620850Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T08:59:07.2641622Z http.https://github.com/.extraheader 2024-06-26T08:59:07.2739848Z A job completed hook has been configured by the self-hosted runner administrator 2024-06-26T08:59:07.2757257Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T08:59:07.2763935Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:59:07.2764469Z ##[endgroup] 2024-06-26T08:59:08.0694808Z Cleaning up orphan processes