2024-08-20T22:09:58.7605843Z Current runner version: '2.319.1' 2024-08-20T22:09:58.7612005Z Runner name: 'i-0a41c091cbed476e8' 2024-08-20T22:09:58.7612778Z Runner group name: 'Default' 2024-08-20T22:09:58.7613700Z Machine name: 'ip-10-0-62-214' 2024-08-20T22:09:58.7630796Z Testing runner upgrade compatibility 2024-08-20T22:09:59.0287168Z ##[group]GITHUB_TOKEN Permissions 2024-08-20T22:09:59.0289146Z Actions: read 2024-08-20T22:09:59.0289661Z Attestations: read 2024-08-20T22:09:59.0290301Z Checks: read 2024-08-20T22:09:59.0290793Z Contents: read 2024-08-20T22:09:59.0291301Z Deployments: read 2024-08-20T22:09:59.0291873Z Discussions: read 2024-08-20T22:09:59.0292388Z Issues: read 2024-08-20T22:09:59.0292929Z Metadata: read 2024-08-20T22:09:59.0293468Z Packages: read 2024-08-20T22:09:59.0294030Z Pages: read 2024-08-20T22:09:59.0294543Z PullRequests: read 2024-08-20T22:09:59.0295155Z RepositoryProjects: read 2024-08-20T22:09:59.0295723Z SecurityEvents: read 2024-08-20T22:09:59.0296292Z Statuses: read 2024-08-20T22:09:59.0297017Z ##[endgroup] 2024-08-20T22:09:59.0300225Z Secret source: Actions 2024-08-20T22:09:59.0300938Z Prepare workflow directory 2024-08-20T22:09:59.1226406Z Prepare all required actions 2024-08-20T22:09:59.1391577Z Getting action download info 2024-08-20T22:09:59.3658450Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-20T22:09:59.9116198Z Download action repository 'pytorch/pytorch@main' (SHA:1ae5d5bb62141d7c9b1b0b66c66a462b4e10b1f2) 2024-08-20T22:10:03.9228837Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-20T22:10:04.1187412Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-20T22:10:04.3535167Z Getting action download info 2024-08-20T22:10:04.4538493Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-20T22:10:04.6430295Z Getting action download info 2024-08-20T22:10:04.7426690Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-20T22:10:04.9135901Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/133712 (40ec5f6ddd9787aca0449b24128343ff4c4a88b3) 2024-08-20T22:10:04.9138017Z ##[group] Inputs 2024-08-20T22:10:04.9138492Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:10:04.9148298Z test-matrix: {"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:10:04.9158616Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:10:04.9159796Z sync-tag: 2024-08-20T22:10:04.9160662Z timeout-minutes: 240 2024-08-20T22:10:04.9161003Z use-gha: 2024-08-20T22:10:04.9161302Z dashboard-tag: 2024-08-20T22:10:04.9161626Z s3-bucket: gha-artifacts 2024-08-20T22:10:04.9161984Z aws-role-to-assume: 2024-08-20T22:10:04.9162323Z ##[endgroup] 2024-08-20T22:10:04.9163470Z Complete job name: cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:10:04.9891383Z A job started hook has been configured by the self-hosted runner administrator 2024-08-20T22:10:05.0032571Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-20T22:10:05.0044611Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:10:05.0045165Z ##[endgroup] 2024-08-20T22:10:06.1454316Z Runner Type: amz2023.linux.g5.4xlarge.nvidia.gpu 2024-08-20T22:10:06.1454877Z Instance Type: g5.4xlarge 2024-08-20T22:10:06.1455568Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-20T22:10:06.1456128Z AMI ID: ami-06c68f701d8090592 2024-08-20T22:10:11.8975151Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-20T22:10:11.8975704Z with: 2024-08-20T22:10:11.8976432Z github-secret: *** 2024-08-20T22:10:11.8977383Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:10:11.8978380Z activate-with-label: false 2024-08-20T22:10:11.8978743Z label: with-ssh 2024-08-20T22:10:11.8979061Z remove-existing-keys: true 2024-08-20T22:10:11.8979417Z fail-silently: true 2024-08-20T22:10:11.8979735Z env: 2024-08-20T22:10:11.8980007Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:10:11.8980348Z ##[endgroup] 2024-08-20T22:10:11.9861806Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-20T22:10:11.9864574Z ciflow reference detected, attempting to extract PR number 2024-08-20T22:10:12.2870004Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-08-20T22:10:12.3565461Z No SSH keys found for user pytorch-bot[bot] 2024-08-20T22:10:12.3566083Z Grabbing public ssh keys from https://github.com/XuehaiPan.keys 2024-08-20T22:10:12.4276908Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-20T22:10:12.4291607Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-20T22:10:12.4314683Z Login using: ssh ec2-user@ec2-52-91-98-187.compute-1.amazonaws.com 2024-08-20T22:10:12.4315462Z All testing is done inside the container, to start an interactive session run: 2024-08-20T22:10:12.4316281Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:10:12.4444602Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-20T22:10:12.4445168Z with: 2024-08-20T22:10:12.4445447Z submodules: recursive 2024-08-20T22:10:12.4446104Z fetch-depth: 0 2024-08-20T22:10:12.4446462Z env: 2024-08-20T22:10:12.4446793Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:10:12.4447222Z ##[endgroup] 2024-08-20T22:10:12.4636274Z ##[group]Run retry () { 2024-08-20T22:10:12.4636664Z retry () { 2024-08-20T22:10:12.4637383Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-20T22:10:12.4637954Z } 2024-08-20T22:10:12.4638264Z echo "${GITHUB_WORKSPACE}" 2024-08-20T22:10:12.4638693Z if [ -z "${NO_SUDO}" ]; then 2024-08-20T22:10:12.4639184Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:10:12.4639650Z else 2024-08-20T22:10:12.4640000Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:10:12.4640437Z fi 2024-08-20T22:10:12.4640790Z mkdir "${GITHUB_WORKSPACE}" 2024-08-20T22:10:12.4653406Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:10:12.4653924Z env: 2024-08-20T22:10:12.4654205Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:10:12.4654548Z NO_SUDO: 2024-08-20T22:10:12.4654824Z ##[endgroup] 2024-08-20T22:10:12.4688254Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:10:12.6180358Z ##[group]Run malfet/checkout@silent-checkout 2024-08-20T22:10:12.6180802Z with: 2024-08-20T22:10:12.6181128Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:10:12.6181570Z fetch-depth: 0 2024-08-20T22:10:12.6182005Z submodules: recursive 2024-08-20T22:10:12.6182337Z quiet-checkout: true 2024-08-20T22:10:12.6182680Z repository: pytorch/pytorch 2024-08-20T22:10:12.6183155Z token: *** 2024-08-20T22:10:12.6183441Z ssh-strict: true 2024-08-20T22:10:12.6183757Z persist-credentials: true 2024-08-20T22:10:12.6184110Z clean: true 2024-08-20T22:10:12.6184439Z sparse-checkout-cone-mode: true 2024-08-20T22:10:12.6184833Z lfs: false 2024-08-20T22:10:12.6185136Z set-safe-directory: true 2024-08-20T22:10:12.6185478Z env: 2024-08-20T22:10:12.6185755Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:10:12.6186096Z ##[endgroup] 2024-08-20T22:10:12.7149125Z Syncing repository: pytorch/pytorch 2024-08-20T22:10:12.7150658Z ##[group]Getting Git version info 2024-08-20T22:10:12.7151443Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:10:12.7152329Z [command]/usr/bin/git version 2024-08-20T22:10:12.7152689Z git version 2.40.1 2024-08-20T22:10:12.7154548Z ##[endgroup] 2024-08-20T22:10:12.7168840Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/876236f6-d47c-40c8-b645-f3ff099f4477' before making global git config changes 2024-08-20T22:10:12.7170090Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T22:10:12.7173623Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:10:12.7222615Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:10:12.7226443Z ##[group]Initializing the repository 2024-08-20T22:10:12.7229138Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:10:12.7272013Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-20T22:10:12.7273021Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-20T22:10:12.7273885Z hint: of your new repositories, which will suppress this warning, call: 2024-08-20T22:10:12.7274454Z hint: 2024-08-20T22:10:12.7274909Z hint: git config --global init.defaultBranch 2024-08-20T22:10:12.7275388Z hint: 2024-08-20T22:10:12.7275910Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-20T22:10:12.7276761Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-20T22:10:12.7277661Z hint: 2024-08-20T22:10:12.7278002Z hint: git branch -m 2024-08-20T22:10:12.7278759Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-20T22:10:12.7286197Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-20T22:10:12.7329709Z ##[endgroup] 2024-08-20T22:10:12.7330280Z ##[group]Disabling automatic garbage collection 2024-08-20T22:10:12.7332589Z [command]/usr/bin/git config --local gc.auto 0 2024-08-20T22:10:12.7373284Z ##[endgroup] 2024-08-20T22:10:12.7373797Z ##[group]Setting up auth 2024-08-20T22:10:12.7379094Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T22:10:12.7422664Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T22:10:12.7793965Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T22:10:12.7834430Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T22:10:12.8187741Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:10:12.8246975Z ##[endgroup] 2024-08-20T22:10:12.8247568Z ##[group]Fetching the repository 2024-08-20T22:10:12.8253261Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-20T22:10:15.8393919Z remote: Enumerating objects: 1008509 2024-08-20T22:10:15.8394684Z remote: Enumerating objects: 1010229, done. 2024-08-20T22:10:15.8398526Z remote: Counting objects: 0% (1/1720) 2024-08-20T22:10:15.8399301Z remote: Counting objects: 1% (18/1720) 2024-08-20T22:10:15.8400138Z remote: Counting objects: 2% (35/1720) 2024-08-20T22:10:15.8400960Z remote: Counting objects: 3% (52/1720) 2024-08-20T22:10:15.8401849Z remote: Counting objects: 4% (69/1720) 2024-08-20T22:10:15.8402362Z remote: Counting objects: 5% (86/1720) 2024-08-20T22:10:15.8402982Z remote: Counting objects: 6% (104/1720) 2024-08-20T22:10:15.8403764Z remote: Counting objects: 7% (121/1720) 2024-08-20T22:10:15.8404281Z remote: Counting objects: 8% (138/1720) 2024-08-20T22:10:15.8405061Z remote: Counting objects: 9% (155/1720) 2024-08-20T22:10:15.8405841Z remote: Counting objects: 10% (172/1720) 2024-08-20T22:10:15.8406402Z remote: Counting objects: 11% (190/1720) 2024-08-20T22:10:15.8406898Z remote: Counting objects: 12% (207/1720) 2024-08-20T22:10:15.8407388Z remote: Counting objects: 13% (224/1720) 2024-08-20T22:10:15.8407881Z remote: Counting objects: 14% (241/1720) 2024-08-20T22:10:15.8408417Z remote: Counting objects: 15% (258/1720) 2024-08-20T22:10:15.8408907Z remote: Counting objects: 16% (276/1720) 2024-08-20T22:10:15.8409402Z remote: Counting objects: 17% (293/1720) 2024-08-20T22:10:15.8410115Z remote: Counting objects: 18% (310/1720) 2024-08-20T22:10:15.8410840Z remote: Counting objects: 19% (327/1720) 2024-08-20T22:10:15.8411489Z remote: Counting objects: 20% (344/1720) 2024-08-20T22:10:15.8412287Z remote: Counting objects: 21% (362/1720) 2024-08-20T22:10:15.8413099Z remote: Counting objects: 22% (379/1720) 2024-08-20T22:10:15.8413845Z remote: Counting objects: 23% (396/1720) 2024-08-20T22:10:15.8414616Z remote: Counting objects: 24% (413/1720) 2024-08-20T22:10:15.8415123Z remote: Counting objects: 25% (430/1720) 2024-08-20T22:10:15.8415616Z remote: Counting objects: 26% (448/1720) 2024-08-20T22:10:15.8416117Z remote: Counting objects: 27% (465/1720) 2024-08-20T22:10:15.8417092Z remote: Counting objects: 28% (482/1720) 2024-08-20T22:10:15.8417617Z remote: Counting objects: 29% (499/1720) 2024-08-20T22:10:15.8418113Z remote: Counting objects: 30% (516/1720) 2024-08-20T22:10:15.8418616Z remote: Counting objects: 31% (534/1720) 2024-08-20T22:10:15.8419109Z remote: Counting objects: 32% (551/1720) 2024-08-20T22:10:15.8419598Z remote: Counting objects: 33% (568/1720) 2024-08-20T22:10:15.8420247Z remote: Counting objects: 34% (585/1720) 2024-08-20T22:10:15.8420744Z remote: Counting objects: 35% (602/1720) 2024-08-20T22:10:15.8421240Z remote: Counting objects: 36% (620/1720) 2024-08-20T22:10:15.8421729Z remote: Counting objects: 37% (637/1720) 2024-08-20T22:10:15.8422222Z remote: Counting objects: 38% (654/1720) 2024-08-20T22:10:15.8422716Z remote: Counting objects: 39% (671/1720) 2024-08-20T22:10:15.8423207Z remote: Counting objects: 40% (688/1720) 2024-08-20T22:10:15.8423700Z remote: Counting objects: 41% (706/1720) 2024-08-20T22:10:15.8424194Z remote: Counting objects: 42% (723/1720) 2024-08-20T22:10:15.8424685Z remote: Counting objects: 43% (740/1720) 2024-08-20T22:10:15.8425175Z remote: Counting objects: 44% (757/1720) 2024-08-20T22:10:15.8425660Z remote: Counting objects: 45% (774/1720) 2024-08-20T22:10:15.8426156Z remote: Counting objects: 46% (792/1720) 2024-08-20T22:10:15.8426658Z remote: Counting objects: 47% (809/1720) 2024-08-20T22:10:15.8427154Z remote: Counting objects: 48% (826/1720) 2024-08-20T22:10:15.8427637Z remote: Counting objects: 49% (843/1720) 2024-08-20T22:10:15.8428133Z remote: Counting objects: 50% (860/1720) 2024-08-20T22:10:15.8428630Z remote: Counting objects: 51% (878/1720) 2024-08-20T22:10:15.8429126Z remote: Counting objects: 52% (895/1720) 2024-08-20T22:10:15.8429638Z remote: Counting objects: 53% (912/1720) 2024-08-20T22:10:15.8430130Z remote: Counting objects: 54% (929/1720) 2024-08-20T22:10:15.8430625Z remote: Counting objects: 55% (946/1720) 2024-08-20T22:10:15.8431120Z remote: Counting objects: 56% (964/1720) 2024-08-20T22:10:15.8431611Z remote: Counting objects: 57% (981/1720) 2024-08-20T22:10:15.8432093Z remote: Counting objects: 58% (998/1720) 2024-08-20T22:10:15.8432603Z remote: Counting objects: 59% (1015/1720) 2024-08-20T22:10:15.8433129Z remote: Counting objects: 60% (1032/1720) 2024-08-20T22:10:15.8433647Z remote: Counting objects: 61% (1050/1720) 2024-08-20T22:10:15.8434170Z remote: Counting objects: 62% (1067/1720) 2024-08-20T22:10:15.8434699Z remote: Counting objects: 63% (1084/1720) 2024-08-20T22:10:15.8435215Z remote: Counting objects: 64% (1101/1720) 2024-08-20T22:10:15.8435730Z remote: Counting objects: 65% (1118/1720) 2024-08-20T22:10:15.8436250Z remote: Counting objects: 66% (1136/1720) 2024-08-20T22:10:15.8436756Z remote: Counting objects: 67% (1153/1720) 2024-08-20T22:10:15.8437274Z remote: Counting objects: 68% (1170/1720) 2024-08-20T22:10:15.8437789Z remote: Counting objects: 69% (1187/1720) 2024-08-20T22:10:15.8438292Z remote: Counting objects: 70% (1204/1720) 2024-08-20T22:10:15.8438807Z remote: Counting objects: 71% (1222/1720) 2024-08-20T22:10:15.8439318Z remote: Counting objects: 72% (1239/1720) 2024-08-20T22:10:15.8439836Z remote: Counting objects: 73% (1256/1720) 2024-08-20T22:10:15.8440339Z remote: Counting objects: 74% (1273/1720) 2024-08-20T22:10:15.8440846Z remote: Counting objects: 75% (1290/1720) 2024-08-20T22:10:15.8441365Z remote: Counting objects: 76% (1308/1720) 2024-08-20T22:10:15.8441875Z remote: Counting objects: 77% (1325/1720) 2024-08-20T22:10:15.8442382Z remote: Counting objects: 78% (1342/1720) 2024-08-20T22:10:15.8442994Z remote: Counting objects: 79% (1359/1720) 2024-08-20T22:10:15.8443588Z remote: Counting objects: 80% (1376/1720) 2024-08-20T22:10:15.8444095Z remote: Counting objects: 81% (1394/1720) 2024-08-20T22:10:15.8444602Z remote: Counting objects: 82% (1411/1720) 2024-08-20T22:10:15.8445117Z remote: Counting objects: 83% (1428/1720) 2024-08-20T22:10:15.8445879Z remote: Counting objects: 84% (1445/1720) 2024-08-20T22:10:15.8446401Z remote: Counting objects: 85% (1462/1720) 2024-08-20T22:10:15.8447037Z remote: Counting objects: 86% (1480/1720) 2024-08-20T22:10:15.8447550Z remote: Counting objects: 87% (1497/1720) 2024-08-20T22:10:15.8448056Z remote: Counting objects: 88% (1514/1720) 2024-08-20T22:10:15.8448569Z remote: Counting objects: 89% (1531/1720) 2024-08-20T22:10:15.8449074Z remote: Counting objects: 90% (1548/1720) 2024-08-20T22:10:15.8449587Z remote: Counting objects: 91% (1566/1720) 2024-08-20T22:10:15.8450102Z remote: Counting objects: 92% (1583/1720) 2024-08-20T22:10:15.8450613Z remote: Counting objects: 93% (1600/1720) 2024-08-20T22:10:15.8451116Z remote: Counting objects: 94% (1617/1720) 2024-08-20T22:10:15.8451623Z remote: Counting objects: 95% (1634/1720) 2024-08-20T22:10:15.8452129Z remote: Counting objects: 96% (1652/1720) 2024-08-20T22:10:15.8452640Z remote: Counting objects: 97% (1669/1720) 2024-08-20T22:10:15.8453149Z remote: Counting objects: 98% (1686/1720) 2024-08-20T22:10:15.8453657Z remote: Counting objects: 99% (1703/1720) 2024-08-20T22:10:15.8454175Z remote: Counting objects: 100% (1720/1720) 2024-08-20T22:10:15.8454730Z remote: Counting objects: 100% (1720/1720), done. 2024-08-20T22:10:15.8572621Z remote: Compressing objects: 0% (1/836) 2024-08-20T22:10:15.8846014Z remote: Compressing objects: 1% (9/836) 2024-08-20T22:10:15.9158992Z remote: Compressing objects: 2% (17/836) 2024-08-20T22:10:15.9598428Z remote: Compressing objects: 3% (26/836) 2024-08-20T22:10:16.0414333Z remote: Compressing objects: 4% (34/836) 2024-08-20T22:10:16.0882073Z remote: Compressing objects: 5% (42/836) 2024-08-20T22:10:16.1564663Z remote: Compressing objects: 6% (51/836) 2024-08-20T22:10:16.1905740Z remote: Compressing objects: 7% (59/836) 2024-08-20T22:10:16.2324430Z remote: Compressing objects: 8% (67/836) 2024-08-20T22:10:16.2633000Z remote: Compressing objects: 9% (76/836) 2024-08-20T22:10:16.2906460Z remote: Compressing objects: 10% (84/836) 2024-08-20T22:10:16.3143055Z remote: Compressing objects: 11% (92/836) 2024-08-20T22:10:16.3325729Z remote: Compressing objects: 12% (101/836) 2024-08-20T22:10:16.3524799Z remote: Compressing objects: 13% (109/836) 2024-08-20T22:10:16.3680069Z remote: Compressing objects: 14% (118/836) 2024-08-20T22:10:16.3790300Z remote: Compressing objects: 15% (126/836) 2024-08-20T22:10:16.3894772Z remote: Compressing objects: 16% (134/836) 2024-08-20T22:10:16.3955390Z remote: Compressing objects: 17% (143/836) 2024-08-20T22:10:16.3998899Z remote: Compressing objects: 18% (151/836) 2024-08-20T22:10:16.4049989Z remote: Compressing objects: 19% (159/836) 2024-08-20T22:10:16.4083717Z remote: Compressing objects: 20% (168/836) 2024-08-20T22:10:16.4107102Z remote: Compressing objects: 21% (176/836) 2024-08-20T22:10:16.4113529Z remote: Compressing objects: 22% (184/836) 2024-08-20T22:10:16.4123248Z remote: Compressing objects: 23% (193/836) 2024-08-20T22:10:16.4136331Z remote: Compressing objects: 24% (201/836) 2024-08-20T22:10:16.4145756Z remote: Compressing objects: 25% (209/836) 2024-08-20T22:10:16.4159148Z remote: Compressing objects: 26% (218/836) 2024-08-20T22:10:16.4166611Z remote: Compressing objects: 27% (226/836) 2024-08-20T22:10:16.4190338Z remote: Compressing objects: 28% (235/836) 2024-08-20T22:10:16.4204898Z remote: Compressing objects: 29% (243/836) 2024-08-20T22:10:16.4217086Z remote: Compressing objects: 30% (251/836) 2024-08-20T22:10:16.4231348Z remote: Compressing objects: 31% (260/836) 2024-08-20T22:10:16.4237069Z remote: Compressing objects: 32% (268/836) 2024-08-20T22:10:16.4249973Z remote: Compressing objects: 33% (276/836) 2024-08-20T22:10:16.4258293Z remote: Compressing objects: 34% (285/836) 2024-08-20T22:10:16.4265654Z remote: Compressing objects: 35% (293/836) 2024-08-20T22:10:16.4275121Z remote: Compressing objects: 36% (301/836) 2024-08-20T22:10:16.4282962Z remote: Compressing objects: 37% (310/836) 2024-08-20T22:10:16.4295118Z remote: Compressing objects: 38% (318/836) 2024-08-20T22:10:16.4302818Z remote: Compressing objects: 39% (327/836) 2024-08-20T22:10:16.4310136Z remote: Compressing objects: 40% (335/836) 2024-08-20T22:10:16.4319002Z remote: Compressing objects: 41% (343/836) 2024-08-20T22:10:16.4327252Z remote: Compressing objects: 42% (352/836) 2024-08-20T22:10:16.4333411Z remote: Compressing objects: 43% (360/836) 2024-08-20T22:10:16.4342159Z remote: Compressing objects: 44% (368/836) 2024-08-20T22:10:16.4348432Z remote: Compressing objects: 45% (377/836) 2024-08-20T22:10:16.4353709Z remote: Compressing objects: 46% (385/836) 2024-08-20T22:10:16.4358194Z remote: Compressing objects: 47% (393/836) 2024-08-20T22:10:16.4362406Z remote: Compressing objects: 48% (402/836) 2024-08-20T22:10:16.4368138Z remote: Compressing objects: 49% (410/836) 2024-08-20T22:10:16.4371739Z remote: Compressing objects: 50% (418/836) 2024-08-20T22:10:16.4375542Z remote: Compressing objects: 51% (427/836) 2024-08-20T22:10:16.4378965Z remote: Compressing objects: 52% (435/836) 2024-08-20T22:10:16.4381104Z remote: Compressing objects: 53% (444/836) 2024-08-20T22:10:16.4386278Z remote: Compressing objects: 54% (452/836) 2024-08-20T22:10:16.4386810Z remote: Compressing objects: 55% (460/836) 2024-08-20T22:10:16.4389197Z remote: Compressing objects: 56% (469/836) 2024-08-20T22:10:16.4390389Z remote: Compressing objects: 57% (477/836) 2024-08-20T22:10:16.4391642Z remote: Compressing objects: 58% (485/836) 2024-08-20T22:10:16.4393236Z remote: Compressing objects: 59% (494/836) 2024-08-20T22:10:16.4393773Z remote: Compressing objects: 60% (502/836) 2024-08-20T22:10:16.4394662Z remote: Compressing objects: 61% (510/836) 2024-08-20T22:10:16.4395869Z remote: Compressing objects: 62% (519/836) 2024-08-20T22:10:16.4396908Z remote: Compressing objects: 63% (527/836) 2024-08-20T22:10:16.4400634Z remote: Compressing objects: 64% (536/836) 2024-08-20T22:10:16.4407657Z remote: Compressing objects: 65% (544/836) 2024-08-20T22:10:16.4415452Z remote: Compressing objects: 66% (552/836) 2024-08-20T22:10:16.4419899Z remote: Compressing objects: 67% (561/836) 2024-08-20T22:10:16.4424549Z remote: Compressing objects: 68% (569/836) 2024-08-20T22:10:16.4429890Z remote: Compressing objects: 69% (577/836) 2024-08-20T22:10:16.4434400Z remote: Compressing objects: 70% (586/836) 2024-08-20T22:10:16.4439248Z remote: Compressing objects: 71% (594/836) 2024-08-20T22:10:16.4445039Z remote: Compressing objects: 72% (602/836) 2024-08-20T22:10:16.4448582Z remote: Compressing objects: 73% (611/836) 2024-08-20T22:10:16.4454240Z remote: Compressing objects: 74% (619/836) 2024-08-20T22:10:16.4458196Z remote: Compressing objects: 75% (627/836) 2024-08-20T22:10:16.4461215Z remote: Compressing objects: 76% (636/836) 2024-08-20T22:10:16.4471674Z remote: Compressing objects: 77% (644/836) 2024-08-20T22:10:16.4474836Z remote: Compressing objects: 78% (653/836) 2024-08-20T22:10:16.4476222Z remote: Compressing objects: 79% (661/836) 2024-08-20T22:10:16.4480423Z remote: Compressing objects: 80% (669/836) 2024-08-20T22:10:16.4483891Z remote: Compressing objects: 81% (678/836) 2024-08-20T22:10:16.4487329Z remote: Compressing objects: 82% (686/836) 2024-08-20T22:10:16.4489929Z remote: Compressing objects: 83% (694/836) 2024-08-20T22:10:16.4493422Z remote: Compressing objects: 84% (703/836) 2024-08-20T22:10:16.4494554Z remote: Compressing objects: 85% (711/836) 2024-08-20T22:10:16.4497001Z remote: Compressing objects: 86% (719/836) 2024-08-20T22:10:16.4499660Z remote: Compressing objects: 87% (728/836) 2024-08-20T22:10:16.4503399Z remote: Compressing objects: 88% (736/836) 2024-08-20T22:10:16.4507908Z remote: Compressing objects: 89% (745/836) 2024-08-20T22:10:16.4509922Z remote: Compressing objects: 90% (753/836) 2024-08-20T22:10:16.4511030Z remote: Compressing objects: 91% (761/836) 2024-08-20T22:10:16.4513077Z remote: Compressing objects: 92% (770/836) 2024-08-20T22:10:16.4513611Z remote: Compressing objects: 93% (778/836) 2024-08-20T22:10:16.4516684Z remote: Compressing objects: 94% (786/836) 2024-08-20T22:10:16.4521753Z remote: Compressing objects: 95% (795/836) 2024-08-20T22:10:16.4522285Z remote: Compressing objects: 96% (803/836) 2024-08-20T22:10:16.4525797Z remote: Compressing objects: 97% (811/836) 2024-08-20T22:10:16.4526343Z remote: Compressing objects: 98% (820/836) 2024-08-20T22:10:16.4528854Z remote: Compressing objects: 99% (828/836) 2024-08-20T22:10:16.4529561Z remote: Compressing objects: 100% (836/836) 2024-08-20T22:10:16.4530138Z remote: Compressing objects: 100% (836/836), done. 2024-08-20T22:10:40.3681644Z remote: Total 1010229 (delta 1121), reused 1351 (delta 881), pack-reused 1008509 (from 1) 2024-08-20T22:11:01.6559660Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-20T22:11:01.6599961Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:11:01.6604314Z ##[endgroup] 2024-08-20T22:11:01.6604864Z ##[group]Determining the checkout info 2024-08-20T22:11:01.6605492Z ##[endgroup] 2024-08-20T22:11:01.6606146Z ##[group]Checking out the ref 2024-08-20T22:11:01.6608817Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:11:03.3822375Z ##[endgroup] 2024-08-20T22:11:03.3822970Z ##[group]Setting up auth for fetching submodules 2024-08-20T22:11:03.3827223Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:11:03.3897556Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-20T22:11:03.3938602Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-20T22:11:03.3980253Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-20T22:11:03.4018790Z ##[endgroup] 2024-08-20T22:11:03.4020675Z ##[group]Fetching submodules 2024-08-20T22:11:03.4021830Z [command]/usr/bin/git submodule sync --recursive 2024-08-20T22:11:03.4411384Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-20T22:11:03.4777868Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-20T22:11:03.4780018Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-20T22:11:03.4784149Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-20T22:11:03.4788240Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-20T22:11:03.4793592Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-20T22:11:03.4796609Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-20T22:11:03.4801508Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-20T22:11:03.4806363Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-20T22:11:03.4810965Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-20T22:11:03.4815858Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-20T22:11:03.4820572Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-20T22:11:03.4825444Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-20T22:11:03.4830310Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-20T22:11:03.4835486Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-20T22:11:03.4840350Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-20T22:11:03.4848013Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:11:03.4856074Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-20T22:11:03.4861263Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-20T22:11:03.4866468Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-20T22:11:03.4871996Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-20T22:11:03.4877457Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-20T22:11:03.4883120Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-20T22:11:03.4888902Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-20T22:11:03.4894538Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-20T22:11:03.4900145Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-20T22:11:03.4906416Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-20T22:11:03.4912209Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-20T22:11:03.4918327Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-20T22:11:03.4924461Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-20T22:11:03.4930809Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-20T22:11:03.4936957Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-20T22:11:03.4943560Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-20T22:11:03.4950184Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-20T22:11:03.4959893Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-20T22:11:03.4996665Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-20T22:11:03.8016071Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-20T22:11:03.9816489Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-20T22:11:04.1681195Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-20T22:11:04.4380577Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-20T22:11:06.3862898Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-20T22:11:17.5866550Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-20T22:11:17.9983928Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-20T22:11:18.4251509Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-20T22:11:19.0349863Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-20T22:11:20.2861453Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-20T22:11:22.4702006Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-20T22:11:28.3243456Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-20T22:11:29.7497794Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-20T22:11:31.5080557Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-20T22:11:32.8168987Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-20T22:11:33.2491119Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-20T22:11:33.6274901Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-20T22:11:34.7802057Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-20T22:11:35.1501625Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-20T22:11:35.4436609Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-20T22:11:37.0589617Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-20T22:11:37.7684970Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-20T22:11:38.5056603Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-20T22:11:44.4116577Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-20T22:11:46.4688535Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-20T22:11:50.7349530Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-20T22:11:50.9648580Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-20T22:12:00.0676587Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-20T22:12:00.2634297Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-20T22:12:00.4654560Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-20T22:12:01.3975592Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-20T22:12:01.6977730Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-20T22:12:02.3843358Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-20T22:12:02.8148091Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-20T22:12:02.8300554Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-20T22:12:02.8416388Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-20T22:12:02.8728386Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-20T22:12:02.9183625Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-20T22:12:04.0885624Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-20T22:12:04.1168856Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-20T22:12:04.1700899Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-20T22:12:04.2837467Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-20T22:12:04.3246340Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-20T22:12:04.9484459Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-20T22:12:05.2348870Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-20T22:12:05.3287978Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-20T22:12:05.3307412Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:12:05.3310883Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:12:05.3314322Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:12:05.3317770Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:12:05.3321528Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:12:05.3352972Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-20T22:12:06.2716312Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-20T22:12:06.8757048Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-20T22:12:09.0511281Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-20T22:12:10.0707955Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-20T22:12:10.4177674Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-20T22:12:10.5283055Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-20T22:12:11.0367209Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-20T22:12:11.1076655Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-20T22:12:11.1226376Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-20T22:12:11.2740612Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-20T22:12:11.3190628Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-20T22:12:11.3645382Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-20T22:12:11.3970286Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-20T22:12:11.4510659Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:12:11.4669749Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-20T22:12:11.4689481Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-20T22:12:11.4719533Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-20T22:12:24.8270791Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-20T22:12:24.8486574Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-20T22:12:24.9484732Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-20T22:12:24.9508641Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:12:24.9511376Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:12:24.9515011Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:12:24.9547574Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-20T22:12:25.4993928Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-20T22:12:26.7124440Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-20T22:12:27.9115355Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-20T22:12:27.9135827Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:12:27.9138958Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:12:27.9142736Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:12:27.9146546Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:12:27.9150688Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:12:27.9154512Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:12:27.9158296Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:12:27.9162111Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:12:27.9193862Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-20T22:12:28.9067791Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-20T22:12:29.2802800Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-20T22:12:30.5269231Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-20T22:12:30.8035122Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-20T22:12:31.3577037Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-20T22:12:32.4664982Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-20T22:12:39.9070468Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-20T22:12:40.3045208Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-20T22:12:40.3274138Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-20T22:12:40.3707745Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-20T22:12:40.3873571Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-20T22:12:40.3892950Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:12:40.3924579Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-20T22:12:40.7247067Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-20T22:12:40.7475502Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-20T22:12:40.7958039Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-20T22:12:40.9192776Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-20T22:12:40.9398558Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-20T22:12:40.9856203Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-20T22:12:41.0516863Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-20T22:12:41.0985294Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-20T22:12:41.1281675Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-20T22:12:41.2538517Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-20T22:12:41.7644794Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-20T22:12:41.7682637Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-20T22:12:41.7685839Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-20T22:12:41.7717937Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-20T22:12:42.2251689Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-20T22:12:43.1777758Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-20T22:12:43.2191828Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-20T22:12:43.3098615Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-20T22:12:43.3119645Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:12:43.3122991Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:12:43.3126576Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:12:43.3130195Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:12:43.3133986Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:12:43.3137703Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:12:43.3142049Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:12:43.3145858Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:12:43.3182814Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-20T22:12:43.7607183Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-20T22:12:44.7981521Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-20T22:12:45.1298683Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-20T22:12:51.3903196Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-20T22:12:51.6570155Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-20T22:12:51.8721407Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-20T22:12:52.1803581Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-20T22:12:58.9241762Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-20T22:12:58.9723581Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-20T22:12:58.9912894Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-20T22:12:59.1139666Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-20T22:12:59.1301680Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-20T22:12:59.1486966Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-20T22:12:59.1692686Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-20T22:12:59.1711370Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:12:59.1714820Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:12:59.1744890Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-20T22:13:01.0730982Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-20T22:13:02.4142542Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-20T22:13:02.4674921Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:13:03.1391545Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-20T22:13:03.1534773Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-20T22:13:03.4700752Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-20T22:13:03.4722829Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:13:03.4726266Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-20T22:13:03.4757729Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-20T22:13:03.9789587Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-20T22:13:05.0367705Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-20T22:13:05.1182983Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-20T22:13:05.1302442Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-20T22:13:05.1459471Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-20T22:13:05.1908768Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-20T22:13:05.2252143Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-20T22:13:05.2735676Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-20T22:13:05.3074592Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-20T22:13:05.3094098Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:13:05.3097281Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:13:05.3100906Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:13:05.3104217Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:13:05.3135519Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-20T22:13:06.3984601Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-20T22:13:06.6271117Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-20T22:13:07.8673156Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-20T22:13:08.8887990Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-20T22:13:08.9081399Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-20T22:13:08.9866203Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-20T22:13:09.0214481Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-20T22:13:09.0234495Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:13:09.0266986Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-20T22:13:09.2614890Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-20T22:13:09.2661612Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-20T22:13:09.3024337Z Entering 'android/libs/fbjni' 2024-08-20T22:13:09.3076416Z Entering 'third_party/FP16' 2024-08-20T22:13:09.3128705Z Entering 'third_party/FXdiv' 2024-08-20T22:13:09.3181120Z Entering 'third_party/NNPACK' 2024-08-20T22:13:09.3235322Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:13:09.3288769Z Entering 'third_party/XNNPACK' 2024-08-20T22:13:09.3357392Z Entering 'third_party/benchmark' 2024-08-20T22:13:09.3409929Z Entering 'third_party/cpp-httplib' 2024-08-20T22:13:09.3462094Z Entering 'third_party/cpuinfo' 2024-08-20T22:13:09.3514069Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:13:09.3568386Z Entering 'third_party/cutlass' 2024-08-20T22:13:09.3626930Z Entering 'third_party/eigen' 2024-08-20T22:13:09.3682103Z Entering 'third_party/fbgemm' 2024-08-20T22:13:09.3732641Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:13:09.3784237Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:13:09.3834826Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:13:09.3892848Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:13:09.3945156Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:13:09.4000368Z Entering 'third_party/flatbuffers' 2024-08-20T22:13:09.4056369Z Entering 'third_party/fmt' 2024-08-20T22:13:09.4107281Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:13:09.4159739Z Entering 'third_party/gloo' 2024-08-20T22:13:09.4212437Z Entering 'third_party/googletest' 2024-08-20T22:13:09.4264557Z Entering 'third_party/ideep' 2024-08-20T22:13:09.4314696Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:13:09.4373579Z Entering 'third_party/ittapi' 2024-08-20T22:13:09.4425311Z Entering 'third_party/kineto' 2024-08-20T22:13:09.4476153Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:13:09.4525764Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:13:09.4579334Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:13:09.4631357Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:13:09.4682622Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:13:09.4730927Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:13:09.4793854Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:13:09.4844164Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:13:09.4896917Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:13:09.4949032Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:13:09.5002974Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:13:09.5054052Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:13:09.5106954Z Entering 'third_party/mimalloc' 2024-08-20T22:13:09.5162087Z Entering 'third_party/nccl/nccl' 2024-08-20T22:13:09.5214922Z Entering 'third_party/nlohmann' 2024-08-20T22:13:09.5268553Z Entering 'third_party/onnx' 2024-08-20T22:13:09.5334144Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:13:09.5385689Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:13:09.5440998Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:13:09.5495103Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:13:09.5544951Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:13:09.5599073Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:13:09.5647679Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:13:09.5702895Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:13:09.5752862Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:13:09.5802426Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:13:09.5851946Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:13:09.5904669Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:13:09.5958117Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:13:09.6034237Z Entering 'third_party/pocketfft' 2024-08-20T22:13:09.6086405Z Entering 'third_party/protobuf' 2024-08-20T22:13:09.6140297Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:13:09.6191250Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:13:09.6244816Z Entering 'third_party/psimd' 2024-08-20T22:13:09.6297671Z Entering 'third_party/pthreadpool' 2024-08-20T22:13:09.6349835Z Entering 'third_party/pybind11' 2024-08-20T22:13:09.6402007Z Entering 'third_party/python-peachpy' 2024-08-20T22:13:09.6455150Z Entering 'third_party/sleef' 2024-08-20T22:13:09.6506344Z Entering 'third_party/tensorpipe' 2024-08-20T22:13:09.6557062Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:13:09.6606782Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:13:09.6656197Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:13:09.6707938Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:13:09.6756278Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:13:09.6832775Z ##[endgroup] 2024-08-20T22:13:09.6834025Z ##[group]Persisting credentials for submodules 2024-08-20T22:13:09.6837138Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-20T22:13:09.7195423Z Entering 'android/libs/fbjni' 2024-08-20T22:13:09.7261391Z Entering 'third_party/FP16' 2024-08-20T22:13:09.7329673Z Entering 'third_party/FXdiv' 2024-08-20T22:13:09.7397299Z Entering 'third_party/NNPACK' 2024-08-20T22:13:09.7468447Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:13:09.7534523Z Entering 'third_party/XNNPACK' 2024-08-20T22:13:09.7616766Z Entering 'third_party/benchmark' 2024-08-20T22:13:09.7685307Z Entering 'third_party/cpp-httplib' 2024-08-20T22:13:09.7750824Z Entering 'third_party/cpuinfo' 2024-08-20T22:13:09.7817860Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:13:09.7884536Z Entering 'third_party/cutlass' 2024-08-20T22:13:09.7961344Z Entering 'third_party/eigen' 2024-08-20T22:13:09.8028818Z Entering 'third_party/fbgemm' 2024-08-20T22:13:09.8097757Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:13:09.8165360Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:13:09.8231354Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:13:09.8304332Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:13:09.8370748Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:13:09.8439680Z Entering 'third_party/flatbuffers' 2024-08-20T22:13:09.8508940Z Entering 'third_party/fmt' 2024-08-20T22:13:09.8580565Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:13:09.8649644Z Entering 'third_party/gloo' 2024-08-20T22:13:09.8715506Z Entering 'third_party/googletest' 2024-08-20T22:13:09.8784926Z Entering 'third_party/ideep' 2024-08-20T22:13:09.8850070Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:13:09.8926569Z Entering 'third_party/ittapi' 2024-08-20T22:13:09.8994090Z Entering 'third_party/kineto' 2024-08-20T22:13:09.9058716Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:13:09.9123633Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:13:09.9192571Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:13:09.9260553Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:13:09.9326935Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:13:09.9396424Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:13:09.9471761Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:13:09.9538821Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:13:09.9606569Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:13:09.9680153Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:13:09.9754354Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:13:09.9820893Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:13:09.9890638Z Entering 'third_party/mimalloc' 2024-08-20T22:13:09.9957036Z Entering 'third_party/nccl/nccl' 2024-08-20T22:13:10.0024439Z Entering 'third_party/nlohmann' 2024-08-20T22:13:10.0092217Z Entering 'third_party/onnx' 2024-08-20T22:13:10.0171146Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:13:10.0240388Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:13:10.0314046Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:13:10.0381256Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:13:10.0447277Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:13:10.0513884Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:13:10.0579354Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:13:10.0646233Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:13:10.0712048Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:13:10.0778434Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:13:10.0842629Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:13:10.0914206Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:13:10.0984170Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:13:10.1070880Z Entering 'third_party/pocketfft' 2024-08-20T22:13:10.1139168Z Entering 'third_party/protobuf' 2024-08-20T22:13:10.1207913Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:13:10.1276348Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:13:10.1345746Z Entering 'third_party/psimd' 2024-08-20T22:13:10.1417502Z Entering 'third_party/pthreadpool' 2024-08-20T22:13:10.1483805Z Entering 'third_party/pybind11' 2024-08-20T22:13:10.1551743Z Entering 'third_party/python-peachpy' 2024-08-20T22:13:10.1618914Z Entering 'third_party/sleef' 2024-08-20T22:13:10.1686967Z Entering 'third_party/tensorpipe' 2024-08-20T22:13:10.1757238Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:13:10.1822089Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:13:10.1887150Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:13:10.1953672Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:13:10.2016975Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:13:10.2108469Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-20T22:13:10.2468938Z Entering 'android/libs/fbjni' 2024-08-20T22:13:10.2530497Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-20T22:13:10.2552807Z Entering 'third_party/FP16' 2024-08-20T22:13:10.2615706Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-20T22:13:10.2637750Z Entering 'third_party/FXdiv' 2024-08-20T22:13:10.2700892Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-20T22:13:10.2722164Z Entering 'third_party/NNPACK' 2024-08-20T22:13:10.2783680Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-20T22:13:10.2805606Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:13:10.2868082Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-20T22:13:10.2889977Z Entering 'third_party/XNNPACK' 2024-08-20T22:13:10.2952634Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-20T22:13:10.2991560Z Entering 'third_party/benchmark' 2024-08-20T22:13:10.3054202Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:13:10.3075932Z Entering 'third_party/cpp-httplib' 2024-08-20T22:13:10.3147107Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-20T22:13:10.3160350Z Entering 'third_party/cpuinfo' 2024-08-20T22:13:10.3222257Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:13:10.3244151Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:13:10.3305666Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-20T22:13:10.3326980Z Entering 'third_party/cutlass' 2024-08-20T22:13:10.3389041Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:13:10.3417712Z Entering 'third_party/eigen' 2024-08-20T22:13:10.3479736Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-20T22:13:10.3503252Z Entering 'third_party/fbgemm' 2024-08-20T22:13:10.3566807Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-20T22:13:10.3586655Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:13:10.3648485Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-20T22:13:10.3671963Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:13:10.3733968Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:13:10.3754551Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:13:10.3815045Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:13:10.3841296Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:13:10.3901508Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:13:10.3922246Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:13:10.3984828Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-20T22:13:10.4007084Z Entering 'third_party/flatbuffers' 2024-08-20T22:13:10.4069101Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-20T22:13:10.4093139Z Entering 'third_party/fmt' 2024-08-20T22:13:10.4154406Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:13:10.4176371Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:13:10.4238295Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-20T22:13:10.4261143Z Entering 'third_party/gloo' 2024-08-20T22:13:10.4324261Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-20T22:13:10.4346089Z Entering 'third_party/googletest' 2024-08-20T22:13:10.4407865Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:13:10.4429698Z Entering 'third_party/ideep' 2024-08-20T22:13:10.4493946Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-20T22:13:10.4512780Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:13:10.4573847Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-20T22:13:10.4603006Z Entering 'third_party/ittapi' 2024-08-20T22:13:10.4665347Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-20T22:13:10.4687381Z Entering 'third_party/kineto' 2024-08-20T22:13:10.4750050Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-20T22:13:10.4770357Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:13:10.4833327Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-20T22:13:10.4857426Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:13:10.4919320Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-20T22:13:10.4940952Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:13:10.5003689Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-20T22:13:10.5023608Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:13:10.5086281Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:13:10.5106737Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:13:10.5169771Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-20T22:13:10.5187593Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:13:10.5250803Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-20T22:13:10.5275871Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:13:10.5338290Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-20T22:13:10.5358584Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:13:10.5420148Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:13:10.5440377Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:13:10.5502449Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-20T22:13:10.5523252Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:13:10.5585610Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-20T22:13:10.5608892Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:13:10.5669391Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-20T22:13:10.5690463Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:13:10.5751990Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-20T22:13:10.5775328Z Entering 'third_party/mimalloc' 2024-08-20T22:13:10.5837155Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-20T22:13:10.5859530Z Entering 'third_party/nccl/nccl' 2024-08-20T22:13:10.5920942Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-20T22:13:10.5942758Z Entering 'third_party/nlohmann' 2024-08-20T22:13:10.6004202Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-20T22:13:10.6026892Z Entering 'third_party/onnx' 2024-08-20T22:13:10.6091443Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-20T22:13:10.6125871Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:13:10.6187349Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:13:10.6209261Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:13:10.6271224Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:13:10.6297840Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:13:10.6360565Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-20T22:13:10.6382035Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:13:10.6442707Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:13:10.6465288Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:13:10.6525961Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:13:10.6545813Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:13:10.6607631Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-20T22:13:10.6628980Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:13:10.6689923Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-20T22:13:10.6711053Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:13:10.6772953Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-20T22:13:10.6792498Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:13:10.6853658Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-20T22:13:10.6873279Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:13:10.6933856Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-20T22:13:10.6952175Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:13:10.7013101Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-20T22:13:10.7034664Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:13:10.7096150Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-20T22:13:10.7118670Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:13:10.7178840Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-20T22:13:10.7219805Z Entering 'third_party/pocketfft' 2024-08-20T22:13:10.7282939Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-20T22:13:10.7304471Z Entering 'third_party/protobuf' 2024-08-20T22:13:10.7367241Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-20T22:13:10.7390215Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:13:10.7451134Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:13:10.7472159Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:13:10.7533111Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:13:10.7558006Z Entering 'third_party/psimd' 2024-08-20T22:13:10.7619680Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-20T22:13:10.7641467Z Entering 'third_party/pthreadpool' 2024-08-20T22:13:10.7703742Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-20T22:13:10.7725590Z Entering 'third_party/pybind11' 2024-08-20T22:13:10.7789442Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:13:10.7811596Z Entering 'third_party/python-peachpy' 2024-08-20T22:13:10.7875492Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-20T22:13:10.7897229Z Entering 'third_party/sleef' 2024-08-20T22:13:10.7958876Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-20T22:13:10.7980571Z Entering 'third_party/tensorpipe' 2024-08-20T22:13:10.8042598Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-20T22:13:10.8064198Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:13:10.8124129Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:13:10.8148810Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:13:10.8208423Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-20T22:13:10.8227823Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:13:10.8288711Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-20T22:13:10.8309345Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:13:10.8369899Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:13:10.8387683Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:13:10.8449567Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-20T22:13:10.9553701Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-20T22:13:10.9916289Z Entering 'android/libs/fbjni' 2024-08-20T22:13:10.9967856Z Entering 'third_party/FP16' 2024-08-20T22:13:11.0019480Z Entering 'third_party/FXdiv' 2024-08-20T22:13:11.0071906Z Entering 'third_party/NNPACK' 2024-08-20T22:13:11.0125170Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:13:11.0177355Z Entering 'third_party/XNNPACK' 2024-08-20T22:13:11.0244845Z Entering 'third_party/benchmark' 2024-08-20T22:13:11.0297321Z Entering 'third_party/cpp-httplib' 2024-08-20T22:13:11.0349043Z Entering 'third_party/cpuinfo' 2024-08-20T22:13:11.0401016Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:13:11.0455015Z Entering 'third_party/cutlass' 2024-08-20T22:13:11.0513690Z Entering 'third_party/eigen' 2024-08-20T22:13:11.0568601Z Entering 'third_party/fbgemm' 2024-08-20T22:13:11.0621186Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:13:11.0675240Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:13:11.0731295Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:13:11.0790970Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:13:11.0840621Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:13:11.0893339Z Entering 'third_party/flatbuffers' 2024-08-20T22:13:11.0950378Z Entering 'third_party/fmt' 2024-08-20T22:13:11.1000852Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:13:11.1052941Z Entering 'third_party/gloo' 2024-08-20T22:13:11.1103774Z Entering 'third_party/googletest' 2024-08-20T22:13:11.1155579Z Entering 'third_party/ideep' 2024-08-20T22:13:11.1205428Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:13:11.1269220Z Entering 'third_party/ittapi' 2024-08-20T22:13:11.1321073Z Entering 'third_party/kineto' 2024-08-20T22:13:11.1372413Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:13:11.1421664Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:13:11.1475220Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:13:11.1526145Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:13:11.1578136Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:13:11.1628785Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:13:11.1686944Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:13:11.1738191Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:13:11.1791361Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:13:11.1843257Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:13:11.1897323Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:13:11.1947420Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:13:11.2000912Z Entering 'third_party/mimalloc' 2024-08-20T22:13:11.2054412Z Entering 'third_party/nccl/nccl' 2024-08-20T22:13:11.2107257Z Entering 'third_party/nlohmann' 2024-08-20T22:13:11.2161470Z Entering 'third_party/onnx' 2024-08-20T22:13:11.2227369Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:13:11.2283748Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:13:11.2340124Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:13:11.2394710Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:13:11.2443927Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:13:11.2494585Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:13:11.2544223Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:13:11.2595467Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:13:11.2645017Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:13:11.2696969Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:13:11.2747462Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:13:11.2804817Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:13:11.2858721Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:13:11.2930133Z Entering 'third_party/pocketfft' 2024-08-20T22:13:11.2982536Z Entering 'third_party/protobuf' 2024-08-20T22:13:11.3036315Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:13:11.3091995Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:13:11.3147319Z Entering 'third_party/psimd' 2024-08-20T22:13:11.3199881Z Entering 'third_party/pthreadpool' 2024-08-20T22:13:11.3252250Z Entering 'third_party/pybind11' 2024-08-20T22:13:11.3304736Z Entering 'third_party/python-peachpy' 2024-08-20T22:13:11.3358678Z Entering 'third_party/sleef' 2024-08-20T22:13:11.3411128Z Entering 'third_party/tensorpipe' 2024-08-20T22:13:11.3463173Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:13:11.3513359Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:13:11.3562896Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:13:11.3612895Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:13:11.3662269Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:13:11.3738017Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-20T22:13:11.4103377Z Entering 'android/libs/fbjni' 2024-08-20T22:13:11.4157285Z Entering 'third_party/FP16' 2024-08-20T22:13:11.4209809Z Entering 'third_party/FXdiv' 2024-08-20T22:13:11.4262083Z Entering 'third_party/NNPACK' 2024-08-20T22:13:11.4314782Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:13:11.4367896Z Entering 'third_party/XNNPACK' 2024-08-20T22:13:11.4437490Z Entering 'third_party/benchmark' 2024-08-20T22:13:11.4490620Z Entering 'third_party/cpp-httplib' 2024-08-20T22:13:11.4542294Z Entering 'third_party/cpuinfo' 2024-08-20T22:13:11.4595221Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:13:11.4647797Z Entering 'third_party/cutlass' 2024-08-20T22:13:11.4706736Z Entering 'third_party/eigen' 2024-08-20T22:13:11.4760525Z Entering 'third_party/fbgemm' 2024-08-20T22:13:11.4811300Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:13:11.4863127Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:13:11.4913386Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:13:11.4970948Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:13:11.5021128Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:13:11.5074453Z Entering 'third_party/flatbuffers' 2024-08-20T22:13:11.5128371Z Entering 'third_party/fmt' 2024-08-20T22:13:11.5179542Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:13:11.5231452Z Entering 'third_party/gloo' 2024-08-20T22:13:11.5285332Z Entering 'third_party/googletest' 2024-08-20T22:13:11.5336588Z Entering 'third_party/ideep' 2024-08-20T22:13:11.5385771Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:13:11.5450802Z Entering 'third_party/ittapi' 2024-08-20T22:13:11.5501255Z Entering 'third_party/kineto' 2024-08-20T22:13:11.5551629Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:13:11.5602606Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:13:11.5654598Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:13:11.5704349Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:13:11.5754494Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:13:11.5802783Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:13:11.5858047Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:13:11.5907294Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:13:11.5957488Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:13:11.6008983Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:13:11.6062742Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:13:11.6113055Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:13:11.6166338Z Entering 'third_party/mimalloc' 2024-08-20T22:13:11.6218999Z Entering 'third_party/nccl/nccl' 2024-08-20T22:13:11.6275746Z Entering 'third_party/nlohmann' 2024-08-20T22:13:11.6328969Z Entering 'third_party/onnx' 2024-08-20T22:13:11.6394830Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:13:11.6445494Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:13:11.6506756Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:13:11.6601924Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:13:11.6654837Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:13:11.6704996Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:13:11.6755165Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:13:11.6811676Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:13:11.6861750Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:13:11.6912405Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:13:11.6960883Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:13:11.7013629Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:13:11.7069814Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:13:11.7143430Z Entering 'third_party/pocketfft' 2024-08-20T22:13:11.7198246Z Entering 'third_party/protobuf' 2024-08-20T22:13:11.7252916Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:13:11.7304160Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:13:11.7359734Z Entering 'third_party/psimd' 2024-08-20T22:13:11.7412579Z Entering 'third_party/pthreadpool' 2024-08-20T22:13:11.7465317Z Entering 'third_party/pybind11' 2024-08-20T22:13:11.7517978Z Entering 'third_party/python-peachpy' 2024-08-20T22:13:11.7570859Z Entering 'third_party/sleef' 2024-08-20T22:13:11.7623089Z Entering 'third_party/tensorpipe' 2024-08-20T22:13:11.7675443Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:13:11.7724938Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:13:11.7775149Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:13:11.7826248Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:13:11.7875440Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:13:11.7951149Z ##[endgroup] 2024-08-20T22:13:11.8000900Z [command]/usr/bin/git log -1 --format='%H' 2024-08-20T22:13:11.8036291Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-20T22:13:11.8269667Z Prepare all required actions 2024-08-20T22:13:11.8270138Z Getting action download info 2024-08-20T22:13:11.9630330Z ##[group]Run ./.github/actions/setup-linux 2024-08-20T22:13:11.9630759Z env: 2024-08-20T22:13:11.9631036Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:11.9631391Z ##[endgroup] 2024-08-20T22:13:11.9694847Z ##[group]Run set -euo pipefail 2024-08-20T22:13:11.9695282Z set -euo pipefail 2024-08-20T22:13:11.9695679Z function get_ec2_metadata() { 2024-08-20T22:13:11.9696230Z  # Pulled from instance metadata endpoint for EC2 2024-08-20T22:13:11.9697138Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-20T22:13:11.9697929Z  category=$1 2024-08-20T22:13:11.9698482Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-20T22:13:11.9699132Z  runner_name_str=i-0a41c091cbed476e8 2024-08-20T22:13:11.9699639Z  if [[ -f /.inarc ]]; then 2024-08-20T22:13:11.9700138Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-20T22:13:11.9700715Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-20T22:13:11.9701411Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-20T22:13:11.9702022Z  else 2024-08-20T22:13:11.9702517Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-20T22:13:11.9703108Z  fi 2024-08-20T22:13:11.9703399Z } 2024-08-20T22:13:11.9703757Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-20T22:13:11.9704349Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-20T22:13:11.9705019Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-20T22:13:11.9705601Z echo "system info $(uname -a)" 2024-08-20T22:13:11.9716905Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:11.9717397Z env: 2024-08-20T22:13:11.9717675Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:11.9718030Z ##[endgroup] 2024-08-20T22:13:11.9825952Z ami-id: ami-06c68f701d8090592 2024-08-20T22:13:11.9886517Z instance-id: i-0a41c091cbed476e8 2024-08-20T22:13:11.9942187Z instance-type: g5.4xlarge 2024-08-20T22:13:11.9956586Z system info Linux ip-10-0-62-214.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-20T22:13:11.9979961Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:13:11.9980934Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:13:11.9990615Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:11.9991267Z env: 2024-08-20T22:13:11.9991551Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:11.9991902Z ##[endgroup] 2024-08-20T22:13:12.0070245Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-20T22:13:12.0070873Z if systemctl is-active --quiet docker; then 2024-08-20T22:13:12.0071495Z  echo "Docker daemon is running..."; 2024-08-20T22:13:12.0071967Z else 2024-08-20T22:13:12.0072468Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-20T22:13:12.0073061Z fi 2024-08-20T22:13:12.0082214Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:12.0082714Z env: 2024-08-20T22:13:12.0082999Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:12.0083350Z ##[endgroup] 2024-08-20T22:13:12.0174356Z Docker daemon is running... 2024-08-20T22:13:12.0232230Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:13:12.0232798Z with: 2024-08-20T22:13:12.0233081Z shell: bash 2024-08-20T22:13:12.0233368Z timeout_minutes: 5 2024-08-20T22:13:12.0233689Z max_attempts: 3 2024-08-20T22:13:12.0234003Z retry_wait_seconds: 30 2024-08-20T22:13:12.0237200Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-20T22:13:12.0240282Z polling_interval_seconds: 1 2024-08-20T22:13:12.0240652Z warning_on_retry: true 2024-08-20T22:13:12.0241014Z continue_on_error: false 2024-08-20T22:13:12.0241357Z env: 2024-08-20T22:13:12.0241638Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:12.0241991Z AWS_RETRY_MODE: standard 2024-08-20T22:13:12.0242343Z AWS_MAX_ATTEMPTS: 5 2024-08-20T22:13:12.0242695Z AWS_DEFAULT_REGION: us-east-1 2024-08-20T22:13:12.0243087Z ##[endgroup] 2024-08-20T22:13:13.2032383Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:13:13.2033289Z Configure a credential helper to remove this warning. See 2024-08-20T22:13:13.2034159Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:13:13.2034678Z 2024-08-20T22:13:13.2034797Z Login Succeeded 2024-08-20T22:13:14.0794886Z Command completed after 1 attempt(s). 2024-08-20T22:13:14.0859178Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:13:14.0859914Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:13:14.0860579Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:13:14.0871503Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:14.0872006Z env: 2024-08-20T22:13:14.0872293Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:14.0872650Z ##[endgroup] 2024-08-20T22:13:14.0965167Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:13:14.0965934Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:13:14.0966527Z # shellcheck disable=SC2046 2024-08-20T22:13:14.0966986Z docker stop $(docker ps -q) || true 2024-08-20T22:13:14.0967467Z # Prune all of the docker images 2024-08-20T22:13:14.0967913Z docker system prune -af 2024-08-20T22:13:14.0976274Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:14.0976771Z env: 2024-08-20T22:13:14.0977045Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:14.0977379Z ##[endgroup] 2024-08-20T22:13:14.1286835Z "docker stop" requires at least 1 argument. 2024-08-20T22:13:14.1287730Z See 'docker stop --help'. 2024-08-20T22:13:14.1288308Z 2024-08-20T22:13:14.1288560Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-20T22:13:14.1288951Z 2024-08-20T22:13:14.1289100Z Stop one or more running containers 2024-08-20T22:13:14.1782661Z Total reclaimed space: 0B 2024-08-20T22:13:14.1828994Z ##[group]Run set +e 2024-08-20T22:13:14.1829365Z set +e 2024-08-20T22:13:14.1829678Z set -x 2024-08-20T22:13:14.1829977Z  2024-08-20T22:13:14.1830299Z PT_DOMAIN=download.pytorch.org 2024-08-20T22:13:14.1831098Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-20T22:13:14.1832205Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-20T22:13:14.1832975Z # one is returned at random 2024-08-20T22:13:14.1833531Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-20T22:13:14.1834045Z  2024-08-20T22:13:14.1834377Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:13:14.1835008Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-20T22:13:14.1835970Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-20T22:13:14.1836538Z  2024-08-20T22:13:14.1836873Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:13:14.1837435Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-20T22:13:14.1837953Z  exit 1 2024-08-20T22:13:14.1838263Z  fi 2024-08-20T22:13:14.1838551Z fi 2024-08-20T22:13:14.1838831Z  2024-08-20T22:13:14.1839193Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-20T22:13:14.1839712Z  # Clean up any old records first 2024-08-20T22:13:14.1840230Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-20T22:13:14.1840691Z fi 2024-08-20T22:13:14.1840963Z  2024-08-20T22:13:14.1841396Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-20T22:13:14.1841954Z cat /etc/hosts 2024-08-20T22:13:14.1851468Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:14.1851962Z env: 2024-08-20T22:13:14.1852247Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:14.1852596Z ##[endgroup] 2024-08-20T22:13:14.1885600Z + PT_DOMAIN=download.pytorch.org 2024-08-20T22:13:14.1892175Z ++ dig -4 +short download.pytorch.org 2024-08-20T22:13:14.1893133Z ++ tail -n1 2024-08-20T22:13:14.2229558Z + RESOLVED_IP=18.160.10.36 2024-08-20T22:13:14.2230031Z + '[' -z 18.160.10.36 ']' 2024-08-20T22:13:14.2230462Z + grep -r download.pytorch.org /etc/hosts 2024-08-20T22:13:14.2249481Z + echo '18.160.10.36 download.pytorch.org' 2024-08-20T22:13:14.2250417Z + sudo tee -a /etc/hosts 2024-08-20T22:13:14.4440667Z 18.160.10.36 download.pytorch.org 2024-08-20T22:13:14.4463789Z + cat /etc/hosts 2024-08-20T22:13:14.4473877Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-20T22:13:14.4485778Z ::1 localhost6 localhost6.localdomain6 2024-08-20T22:13:14.4486286Z 18.160.10.36 download.pytorch.org 2024-08-20T22:13:14.4643197Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-20T22:13:14.4643909Z with: 2024-08-20T22:13:14.4645015Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:14.4646636Z docker-build-dir: .ci/docker 2024-08-20T22:13:14.4647016Z working-directory: . 2024-08-20T22:13:14.4647483Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:14.4648005Z force-push: false 2024-08-20T22:13:14.4648314Z env: 2024-08-20T22:13:14.4648629Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:14.4648962Z ##[endgroup] 2024-08-20T22:13:14.4668082Z ##[group]Run set -ex 2024-08-20T22:13:14.4668467Z set -ex 2024-08-20T22:13:14.4668782Z  2024-08-20T22:13:14.4669488Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-20T22:13:14.4670465Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-20T22:13:14.4671266Z # job could then download the pre-built image as usual 2024-08-20T22:13:14.4672012Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-20T22:13:14.4672692Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:14.4673324Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:14.4673884Z  2024-08-20T22:13:14.4674391Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-20T22:13:14.4675022Z  exit 0 2024-08-20T22:13:14.4675321Z else 2024-08-20T22:13:14.4675685Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:14.4676137Z fi 2024-08-20T22:13:14.4676425Z  2024-08-20T22:13:14.4676907Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-20T22:13:14.4677758Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-20T22:13:14.4678549Z  # use it as it is, but first let's extract the tag 2024-08-20T22:13:14.4679258Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-20T22:13:14.4679989Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:14.4680699Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:14.4681255Z else 2024-08-20T22:13:14.4681695Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-20T22:13:14.4682362Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:14.4683259Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:14.4684086Z fi 2024-08-20T22:13:14.4695261Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:14.4695764Z env: 2024-08-20T22:13:14.4696054Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:14.4696419Z REPO_NAME: pytorch 2024-08-20T22:13:14.4697555Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:14.4698781Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:13:14.4699288Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:14.4699811Z ##[endgroup] 2024-08-20T22:13:14.4731037Z + [[ ! -d .ci/docker ]] 2024-08-20T22:13:14.4731678Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-20T22:13:14.4732146Z + echo skip=false 2024-08-20T22:13:14.4734260Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-20T22:13:14.4738660Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:14.4739963Z ++ awk -F '[:,]' '{print $2}' 2024-08-20T22:13:14.4766654Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:14.4767450Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:14.4769491Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:14.4802290Z ##[group]Run set +e 2024-08-20T22:13:14.4802733Z set +e 2024-08-20T22:13:14.4803216Z set -x 2024-08-20T22:13:14.4803693Z  2024-08-20T22:13:14.4804195Z login() { 2024-08-20T22:13:14.4805022Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:13:14.4805890Z } 2024-08-20T22:13:14.4806353Z  2024-08-20T22:13:14.4806680Z retry () { 2024-08-20T22:13:14.4807169Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:13:14.4807776Z } 2024-08-20T22:13:14.4808142Z  2024-08-20T22:13:14.4808508Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:13:14.4809088Z  2024-08-20T22:13:14.4809653Z # Check if image already exists, if it does then skip building it 2024-08-20T22:13:14.4810471Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-20T22:13:14.4811073Z  exit 0 2024-08-20T22:13:14.4811465Z fi 2024-08-20T22:13:14.4811899Z  2024-08-20T22:13:14.4812499Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-20T22:13:14.4813429Z # be empty. The default action would be to continue rebuild the image 2024-08-20T22:13:14.4814287Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-20T22:13:14.4815099Z  # if we're on the base branch then use the parent commit 2024-08-20T22:13:14.4815804Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-20T22:13:14.4816288Z else 2024-08-20T22:13:14.4816901Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-20T22:13:14.4817705Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-20T22:13:14.4818307Z fi 2024-08-20T22:13:14.4818688Z  2024-08-20T22:13:14.4819172Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-20T22:13:14.4819769Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:14.4820330Z  2024-08-20T22:13:14.4821106Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-20T22:13:14.4821979Z  exit 0 2024-08-20T22:13:14.4822444Z fi 2024-08-20T22:13:14.4822777Z  2024-08-20T22:13:14.4823311Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-20T22:13:14.4824453Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-20T22:13:14.4825346Z  exit 1 2024-08-20T22:13:14.4825789Z fi 2024-08-20T22:13:14.4826141Z  2024-08-20T22:13:14.4826781Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-20T22:13:14.4827921Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-20T22:13:14.4828934Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-20T22:13:14.4829954Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-20T22:13:14.4831218Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-20T22:13:14.4832118Z fi 2024-08-20T22:13:14.4832515Z  2024-08-20T22:13:14.4832982Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:13:14.4841389Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:14.4842097Z env: 2024-08-20T22:13:14.4842458Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:14.4842896Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:13:14.4843587Z BASE_REVISION: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:13:14.4844963Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:14.4846725Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:14.4847488Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:14.4848229Z ##[endgroup] 2024-08-20T22:13:14.4877489Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:14.4878252Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:14.4880277Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:13:14.4881790Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:15.0287391Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:13:15.0288283Z Configure a credential helper to remove this warning. See 2024-08-20T22:13:15.0289373Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:13:15.0290091Z 2024-08-20T22:13:15.0290254Z Login Succeeded 2024-08-20T22:13:15.0313862Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:15.2446482Z { 2024-08-20T22:13:15.2446950Z "schemaVersion": 2, 2024-08-20T22:13:15.2447809Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-20T22:13:15.2450183Z "config": { 2024-08-20T22:13:15.2450992Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-20T22:13:15.2451621Z "size": 48711, 2024-08-20T22:13:15.2452557Z "digest": "sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb" 2024-08-20T22:13:15.2453392Z }, 2024-08-20T22:13:15.2454159Z "layers": [ 2024-08-20T22:13:15.2454670Z { 2024-08-20T22:13:15.2455798Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2456638Z "size": 28580681, 2024-08-20T22:13:15.2457874Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-08-20T22:13:15.2458794Z }, 2024-08-20T22:13:15.2459384Z { 2024-08-20T22:13:15.2460011Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2460667Z "size": 7943451, 2024-08-20T22:13:15.2461702Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-08-20T22:13:15.2462881Z }, 2024-08-20T22:13:15.2463344Z { 2024-08-20T22:13:15.2464235Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2465009Z "size": 55728572, 2024-08-20T22:13:15.2466621Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-08-20T22:13:15.2468090Z }, 2024-08-20T22:13:15.2468892Z { 2024-08-20T22:13:15.2469568Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2470929Z "size": 186, 2024-08-20T22:13:15.2472080Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-08-20T22:13:15.2472856Z }, 2024-08-20T22:13:15.2473458Z { 2024-08-20T22:13:15.2474524Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2475813Z "size": 6886, 2024-08-20T22:13:15.2476720Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-08-20T22:13:15.2477443Z }, 2024-08-20T22:13:15.2478467Z { 2024-08-20T22:13:15.2479244Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2480143Z "size": 1329001756, 2024-08-20T22:13:15.2481057Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-08-20T22:13:15.2482009Z }, 2024-08-20T22:13:15.2482604Z { 2024-08-20T22:13:15.2483264Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2484415Z "size": 62501, 2024-08-20T22:13:15.2485325Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-08-20T22:13:15.2486339Z }, 2024-08-20T22:13:15.2486754Z { 2024-08-20T22:13:15.2487571Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2488579Z "size": 1684, 2024-08-20T22:13:15.2489439Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-08-20T22:13:15.2490627Z }, 2024-08-20T22:13:15.2491059Z { 2024-08-20T22:13:15.2491620Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2492347Z "size": 1523, 2024-08-20T22:13:15.2492981Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-08-20T22:13:15.2493709Z }, 2024-08-20T22:13:15.2494100Z { 2024-08-20T22:13:15.2494609Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2495263Z "size": 2528295403, 2024-08-20T22:13:15.2495990Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-08-20T22:13:15.2496721Z }, 2024-08-20T22:13:15.2497249Z + exit 0 2024-08-20T22:13:15.2497626Z { 2024-08-20T22:13:15.2498149Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2498843Z "size": 86016, 2024-08-20T22:13:15.2499471Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-08-20T22:13:15.2500194Z }, 2024-08-20T22:13:15.2500575Z { 2024-08-20T22:13:15.2501091Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2501729Z "size": 1822, 2024-08-20T22:13:15.2502435Z "digest": "sha256:ac23c5fcc1ef310e9cd0d138b180281188e20a3e28c51cd14751622e3c7d92ea" 2024-08-20T22:13:15.2503150Z }, 2024-08-20T22:13:15.2503455Z { 2024-08-20T22:13:15.2504039Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2504686Z "size": 246761563, 2024-08-20T22:13:15.2505352Z "digest": "sha256:ca426ffc9d2bd051ad1ff5c4f02fd5d9be161e70953cc37882c815080c708f4d" 2024-08-20T22:13:15.2506110Z }, 2024-08-20T22:13:15.2506454Z { 2024-08-20T22:13:15.2506972Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2507687Z "size": 544, 2024-08-20T22:13:15.2508264Z "digest": "sha256:4373aa743caf6660c671295b837048c8350a2165f2336dc3b7b6c96462579e50" 2024-08-20T22:13:15.2509022Z }, 2024-08-20T22:13:15.2509405Z { 2024-08-20T22:13:15.2509897Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2510558Z "size": 1258, 2024-08-20T22:13:15.2511242Z "digest": "sha256:75b2abb16c685854c017d0cf75c626526c956734c656d9e78fdd4520317e9b3c" 2024-08-20T22:13:15.2511943Z }, 2024-08-20T22:13:15.2512262Z { 2024-08-20T22:13:15.2512834Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2513476Z "size": 484, 2024-08-20T22:13:15.2514163Z "digest": "sha256:ef1e3ef1e85f0124270b8506721219150e230184181f8207d64463bf5c67e656" 2024-08-20T22:13:15.2514818Z }, 2024-08-20T22:13:15.2515174Z { 2024-08-20T22:13:15.2515787Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2516495Z "size": 91711766, 2024-08-20T22:13:15.2517340Z "digest": "sha256:0bf61114d37b2d09065aa6852b1e78a32580a1ff011dd5d23f03b0f5c4e710af" 2024-08-20T22:13:15.2518142Z }, 2024-08-20T22:13:15.2518494Z { 2024-08-20T22:13:15.2519141Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2519890Z "size": 3390, 2024-08-20T22:13:15.2520558Z "digest": "sha256:2df849f72ba4258df10c9c45ace21a66d93b73196e78e96f12c13098822f60e2" 2024-08-20T22:13:15.2521335Z }, 2024-08-20T22:13:15.2521697Z { 2024-08-20T22:13:15.2522332Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2522962Z "size": 1909, 2024-08-20T22:13:15.2523815Z "digest": "sha256:34c47d63c5de27418739aa048b4dd781c3006d65691e9b903c95304d0509d89c" 2024-08-20T22:13:15.2524573Z }, 2024-08-20T22:13:15.2525022Z { 2024-08-20T22:13:15.2525691Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2526964Z "size": 701, 2024-08-20T22:13:15.2527801Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:13:15.2528803Z }, 2024-08-20T22:13:15.2529264Z { 2024-08-20T22:13:15.2529807Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2530543Z "size": 2864254879, 2024-08-20T22:13:15.2531259Z "digest": "sha256:6efb96fc87ea1bca54876d85331e5585e28f325a1d60ee1aef7811eabb74f933" 2024-08-20T22:13:15.2531944Z }, 2024-08-20T22:13:15.2532363Z { 2024-08-20T22:13:15.2532889Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2533525Z "size": 380, 2024-08-20T22:13:15.2534210Z "digest": "sha256:b119c4fc6c58d9e72744201ff4850b34184a94eb3566031b7d36e87ecce950a6" 2024-08-20T22:13:15.2534915Z }, 2024-08-20T22:13:15.2535345Z { 2024-08-20T22:13:15.2535840Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2536491Z "size": 12900, 2024-08-20T22:13:15.2537223Z "digest": "sha256:33b639226eeefa41d7bb26a8ecc20d271ceef45571dabfd80aaee3634926381b" 2024-08-20T22:13:15.2537987Z }, 2024-08-20T22:13:15.2538358Z { 2024-08-20T22:13:15.2538946Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2539579Z "size": 803, 2024-08-20T22:13:15.2540244Z "digest": "sha256:d3547bd43146930aa830d54562c98aa678e355cf5f84527e9717eb72fffe8ca3" 2024-08-20T22:13:15.2541006Z }, 2024-08-20T22:13:15.2541313Z { 2024-08-20T22:13:15.2541865Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2542590Z "size": 106, 2024-08-20T22:13:15.2543242Z "digest": "sha256:5645520256fa6d7f2270608eaed53849a131d03fbf4c5d4472c9cfde1a439e47" 2024-08-20T22:13:15.2543906Z }, 2024-08-20T22:13:15.2544304Z { 2024-08-20T22:13:15.2544888Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2545889Z "size": 503, 2024-08-20T22:13:15.2546652Z "digest": "sha256:829ed7d91e3818d0e11834c71551117c7aeec0a51d798e93d641faa0c02582cb" 2024-08-20T22:13:15.2547438Z }, 2024-08-20T22:13:15.2547777Z { 2024-08-20T22:13:15.2548365Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2549070Z "size": 121477267, 2024-08-20T22:13:15.2549733Z "digest": "sha256:385d0421d67c533d628985c453a186028a61f6ad4209ae519ec3d4ecea661d3e" 2024-08-20T22:13:15.2550514Z }, 2024-08-20T22:13:15.2550829Z { 2024-08-20T22:13:15.2551366Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2552129Z "size": 109, 2024-08-20T22:13:15.2552777Z "digest": "sha256:7cbfce999d76851e07dd71056a6d852629d789ccfd46be44cc405b451fb8bb58" 2024-08-20T22:13:15.2553462Z }, 2024-08-20T22:13:15.2553882Z { 2024-08-20T22:13:15.2554426Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2555132Z "size": 490, 2024-08-20T22:13:15.2555752Z "digest": "sha256:6658e558ebe496f0d628947534b0ea1e44808f84f2dfd8a9a704e8aa5329d791" 2024-08-20T22:13:15.2556462Z }, 2024-08-20T22:13:15.2556878Z { 2024-08-20T22:13:15.2557371Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2558036Z "size": 297, 2024-08-20T22:13:15.2558745Z "digest": "sha256:33444a5a7e01a3010fd1cc16b32a66892a08600d7f700747d23faf3c36624f3a" 2024-08-20T22:13:15.2559655Z }, 2024-08-20T22:13:15.2559969Z { 2024-08-20T22:13:15.2560578Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2561236Z "size": 103, 2024-08-20T22:13:15.2561896Z "digest": "sha256:7c47122f4e09b47e891b4ae9e499a35a4eb46faacae87d0e946615adda6e0706" 2024-08-20T22:13:15.2562634Z }, 2024-08-20T22:13:15.2562976Z { 2024-08-20T22:13:15.2563709Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2564421Z "size": 1473, 2024-08-20T22:13:15.2565019Z "digest": "sha256:1811ff4d3556ac255604f346d879528aff2aadd7652c2d39baf4fe6eaa60aec3" 2024-08-20T22:13:15.2565752Z }, 2024-08-20T22:13:15.2566186Z { 2024-08-20T22:13:15.2566678Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2567351Z "size": 594729516, 2024-08-20T22:13:15.2568237Z "digest": "sha256:7754ee068b33f760d9c320c26a1a3935247ace0c8c463a976105c1586a43fb1c" 2024-08-20T22:13:15.2569013Z }, 2024-08-20T22:13:15.2569327Z { 2024-08-20T22:13:15.2569914Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2570581Z "size": 164, 2024-08-20T22:13:15.2571213Z "digest": "sha256:0479fbac3361713c075bf788e778117917dcacdd1ff71ed79a789dd2016a1714" 2024-08-20T22:13:15.2571940Z }, 2024-08-20T22:13:15.2572301Z { 2024-08-20T22:13:15.2572948Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2573868Z "size": 564, 2024-08-20T22:13:15.2574661Z "digest": "sha256:f1d6874dee9c34d154278d2b6935d04447f07fa89d25d4a09398234040d8a7e8" 2024-08-20T22:13:15.2575504Z }, 2024-08-20T22:13:15.2575936Z { 2024-08-20T22:13:15.2576425Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2577181Z "size": 35874203, 2024-08-20T22:13:15.2577903Z "digest": "sha256:afc429e87b593a64f3e0a1be51e09e966ea1aa71e6e8a0bec4edc9f2f7bc1c05" 2024-08-20T22:13:15.2578640Z }, 2024-08-20T22:13:15.2578973Z { 2024-08-20T22:13:15.2579556Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2580245Z "size": 104, 2024-08-20T22:13:15.2580963Z "digest": "sha256:019620471c2592cd22437fe8ed1a94c784114cf7445c5c2ab36dd0bae727fc66" 2024-08-20T22:13:15.2581632Z }, 2024-08-20T22:13:15.2581979Z { 2024-08-20T22:13:15.2582571Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2583183Z "size": 425, 2024-08-20T22:13:15.2583861Z "digest": "sha256:27aeb1b2cd5e28d8f0e54fc1090baf88abc29ebf5fce088b493fcae99c9550d1" 2024-08-20T22:13:15.2584646Z }, 2024-08-20T22:13:15.2584992Z { 2024-08-20T22:13:15.2585499Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2586203Z "size": 20262183, 2024-08-20T22:13:15.2586855Z "digest": "sha256:829c0e19f4b7797e9120a86172fabfd710f9b3971ffd714a616931a3fadc7192" 2024-08-20T22:13:15.2587629Z }, 2024-08-20T22:13:15.2587987Z { 2024-08-20T22:13:15.2588521Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2589243Z "size": 438, 2024-08-20T22:13:15.2589935Z "digest": "sha256:560cd6122a3868e7e82e75eb103d182edf6512114d61ccdfae099af00cb16b4c" 2024-08-20T22:13:15.2590639Z }, 2024-08-20T22:13:15.2590985Z { 2024-08-20T22:13:15.2591569Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2592209Z "size": 701, 2024-08-20T22:13:15.2592845Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:13:15.2593635Z }, 2024-08-20T22:13:15.2594042Z { 2024-08-20T22:13:15.2594534Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2595242Z "size": 141, 2024-08-20T22:13:15.2595908Z "digest": "sha256:5a4eb19d72198a01f3babd4711c40fcf7c6e5cb9d5cfedc507b0061408d47097" 2024-08-20T22:13:15.2596647Z }, 2024-08-20T22:13:15.2597024Z { 2024-08-20T22:13:15.2597553Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2598320Z "size": 135, 2024-08-20T22:13:15.2599044Z "digest": "sha256:da6a838a66c72b5de9a5590c233779880ea901c1264940a1c618825d2501ffd1" 2024-08-20T22:13:15.2599750Z }, 2024-08-20T22:13:15.2600114Z { 2024-08-20T22:13:15.2600690Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2601350Z "size": 4838068375, 2024-08-20T22:13:15.2602036Z "digest": "sha256:cbb79a7053537bed2d80b77198ea552b985563b11d3f8616593c6ab575a3a090" 2024-08-20T22:13:15.2602809Z }, 2024-08-20T22:13:15.2603161Z { 2024-08-20T22:13:15.2603792Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2604522Z "size": 194, 2024-08-20T22:13:15.2605160Z "digest": "sha256:4d4fa6802ab3e6b36d8d4adb650046e9370c4a16a19876970559ef8a312ca855" 2024-08-20T22:13:15.2605945Z }, 2024-08-20T22:13:15.2606253Z { 2024-08-20T22:13:15.2606898Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2607608Z "size": 564, 2024-08-20T22:13:15.2608258Z "digest": "sha256:0f085d4fa279366fcd52d303b7ad8abddc5962a8872174b8109f3f5d5ca5f2cf" 2024-08-20T22:13:15.2609056Z }, 2024-08-20T22:13:15.2609447Z { 2024-08-20T22:13:15.2610004Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2610623Z "size": 43162921, 2024-08-20T22:13:15.2611319Z "digest": "sha256:4a0412373305517489d891a4749e6d2b80b4aeb7df3b8adca0ed0abb2f3e5300" 2024-08-20T22:13:15.2612053Z }, 2024-08-20T22:13:15.2612398Z { 2024-08-20T22:13:15.2612958Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2613631Z "size": 106, 2024-08-20T22:13:15.2614261Z "digest": "sha256:302c8c1779444e54919f83e91c6d5e1fdea653fc899be802b3a2a34d34d99d5d" 2024-08-20T22:13:15.2614984Z }, 2024-08-20T22:13:15.2615402Z { 2024-08-20T22:13:15.2615935Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2616662Z "size": 1380, 2024-08-20T22:13:15.2617267Z "digest": "sha256:02970347dff622ee45629bc28788716d00528464f71f88ec7b74ae5f44a486e2" 2024-08-20T22:13:15.2617969Z }, 2024-08-20T22:13:15.2618386Z { 2024-08-20T22:13:15.2618916Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2619525Z "size": 701, 2024-08-20T22:13:15.2620228Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:13:15.2620943Z }, 2024-08-20T22:13:15.2621248Z { 2024-08-20T22:13:15.2621891Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2622553Z "size": 139, 2024-08-20T22:13:15.2623202Z "digest": "sha256:916c5e8580164926a3aec87e88a5d1c2d02756e49fdeed4198e1e6304efa2fd3" 2024-08-20T22:13:15.2623928Z }, 2024-08-20T22:13:15.2624269Z { 2024-08-20T22:13:15.2624821Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2625536Z "size": 118, 2024-08-20T22:13:15.2626143Z "digest": "sha256:c42b3602a1fb4df5111224650f15adffb025979ae9c614807379ea84e189533b" 2024-08-20T22:13:15.2626875Z }, 2024-08-20T22:13:15.2627267Z { 2024-08-20T22:13:15.2627793Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2628442Z "size": 4324672728, 2024-08-20T22:13:15.2629209Z "digest": "sha256:11fad972ef6cb9d2dc236504b0ee5b6e5ea996e4576ece0ed3262fbb9ee2de60" 2024-08-20T22:13:15.2629964Z }, 2024-08-20T22:13:15.2630271Z { 2024-08-20T22:13:15.2630852Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2631537Z "size": 174, 2024-08-20T22:13:15.2632224Z "digest": "sha256:f957c16244819132b478272aaf24231699c4fcbd495c229a574f94daab8d6b86" 2024-08-20T22:13:15.2632894Z }, 2024-08-20T22:13:15.2633265Z { 2024-08-20T22:13:15.2633847Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2634516Z "size": 907, 2024-08-20T22:13:15.2635145Z "digest": "sha256:768c31dca016b80bbdc973c8ee818034bdbd17b7ea9bdadbf0ec45ea51005720" 2024-08-20T22:13:15.2636084Z }, 2024-08-20T22:13:15.2636468Z { 2024-08-20T22:13:15.2636968Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2637700Z "size": 701, 2024-08-20T22:13:15.2638372Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:13:15.2639151Z }, 2024-08-20T22:13:15.2639513Z { 2024-08-20T22:13:15.2640075Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2640737Z "size": 134, 2024-08-20T22:13:15.2641457Z "digest": "sha256:8c32d0ecebbe2054c96a3a5c1f3d6c8b87c2e826686b369e5242cd55d0f01c2d" 2024-08-20T22:13:15.2642155Z }, 2024-08-20T22:13:15.2642557Z { 2024-08-20T22:13:15.2643168Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2643936Z "size": 32, 2024-08-20T22:13:15.2644635Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:13:15.2645447Z }, 2024-08-20T22:13:15.2646122Z { 2024-08-20T22:13:15.2646662Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2647375Z "size": 158, 2024-08-20T22:13:15.2648003Z "digest": "sha256:78ee77eddef23f20afc78e2924f4467041cadb754634ee3df431c68211cf676d" 2024-08-20T22:13:15.2648763Z }, 2024-08-20T22:13:15.2649141Z { 2024-08-20T22:13:15.2649709Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2650372Z "size": 1841, 2024-08-20T22:13:15.2651070Z "digest": "sha256:37513e9b43d7123e7ddee09100f62a72b57af7b3833eaecd515e247e2f63e13f" 2024-08-20T22:13:15.2651743Z }, 2024-08-20T22:13:15.2652103Z { 2024-08-20T22:13:15.2652677Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2653353Z "size": 7529776, 2024-08-20T22:13:15.2653968Z "digest": "sha256:a7a5392e2bf6b73d9be292cc22a99ad4637636c15733beb7afbe6403b5ba21a8" 2024-08-20T22:13:15.2654742Z }, 2024-08-20T22:13:15.2655109Z { 2024-08-20T22:13:15.2655598Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2656316Z "size": 163, 2024-08-20T22:13:15.2657013Z "digest": "sha256:b10de02255c021b2366b85bf12ddf593aac89f88964201c455b71a145bc5ca2e" 2024-08-20T22:13:15.2657783Z }, 2024-08-20T22:13:15.2658086Z { 2024-08-20T22:13:15.2658631Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2659383Z "size": 7943, 2024-08-20T22:13:15.2660055Z "digest": "sha256:e56eeff2db93ffe5002a2bb764795b7333321c6802cc28bef0801a619fe6a07e" 2024-08-20T22:13:15.2660730Z }, 2024-08-20T22:13:15.2661119Z { 2024-08-20T22:13:15.2661666Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2662277Z "size": 8068, 2024-08-20T22:13:15.2662957Z "digest": "sha256:70ac3e38d0fd1f0f46ad710c3fc485fd01406606054e8cf84a63af1570bc9f3a" 2024-08-20T22:13:15.2663760Z }, 2024-08-20T22:13:15.2664097Z { 2024-08-20T22:13:15.2664631Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2665304Z "size": 302, 2024-08-20T22:13:15.2665930Z "digest": "sha256:eb09cbf3e4168749cc8605e3188ed35070f6037d13930e443c38d8941734488f" 2024-08-20T22:13:15.2666701Z }, 2024-08-20T22:13:15.2667002Z { 2024-08-20T22:13:15.2667540Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2668279Z "size": 7629638, 2024-08-20T22:13:15.2668934Z "digest": "sha256:ee1273153675e6543b58f5be50321abee5e91f7351701d5d501a6837eaf676ef" 2024-08-20T22:13:15.2669607Z }, 2024-08-20T22:13:15.2670009Z { 2024-08-20T22:13:15.2670588Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2671206Z "size": 108, 2024-08-20T22:13:15.2671915Z "digest": "sha256:e22979f9e0eedfa16f98eb4fa26f30a50c237f5ffd2f6e5b898f2b908058496d" 2024-08-20T22:13:15.2672640Z }, 2024-08-20T22:13:15.2673006Z { 2024-08-20T22:13:15.2673548Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2674352Z "size": 54145661, 2024-08-20T22:13:15.2675049Z "digest": "sha256:6c1f0aabce7c84f9b0b9ab3b96325778dba1c1465b7463c5b4c6be1cf52b74db" 2024-08-20T22:13:15.2675833Z }, 2024-08-20T22:13:15.2676152Z { 2024-08-20T22:13:15.2676685Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2677441Z "size": 473, 2024-08-20T22:13:15.2678070Z "digest": "sha256:7ebe49c3a23e71b289ce012b48dad35791a2f2aa7fcee2326a64d763b0d14df8" 2024-08-20T22:13:15.2678806Z }, 2024-08-20T22:13:15.2679198Z { 2024-08-20T22:13:15.2679748Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2680368Z "size": 1374858766, 2024-08-20T22:13:15.2681092Z "digest": "sha256:5d115b29b5d53e5418762bc41ae975dd245bcffc3d058b8902c58146d505cb37" 2024-08-20T22:13:15.2681814Z }, 2024-08-20T22:13:15.2682208Z { 2024-08-20T22:13:15.2682859Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2683614Z "size": 106, 2024-08-20T22:13:15.2684392Z "digest": "sha256:7f2d7a04fbe65bbe1e895138e0cfeeef53ad926c890bab8e7e132ad1e655e48f" 2024-08-20T22:13:15.2685139Z }, 2024-08-20T22:13:15.2685444Z { 2024-08-20T22:13:15.2686042Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2686690Z "size": 559, 2024-08-20T22:13:15.2687281Z "digest": "sha256:d79fd7d526989e554455eda7d459f2c88444030f61710101e52ec8c7a49b7b79" 2024-08-20T22:13:15.2688059Z }, 2024-08-20T22:13:15.2688408Z { 2024-08-20T22:13:15.2688941Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2689636Z "size": 46248488, 2024-08-20T22:13:15.2690281Z "digest": "sha256:9752d6fbbcab4e7403f26777c5b65911320de6389c81b833a47ba45c4f3354b4" 2024-08-20T22:13:15.2690988Z }, 2024-08-20T22:13:15.2691450Z { 2024-08-20T22:13:15.2691934Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2692605Z "size": 111, 2024-08-20T22:13:15.2693279Z "digest": "sha256:83808b2eff4d71ed46db7c29f66fe918f328405c25b314225dfad01b4868d1f4" 2024-08-20T22:13:15.2693946Z }, 2024-08-20T22:13:15.2694372Z { 2024-08-20T22:13:15.2694954Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2695631Z "size": 32, 2024-08-20T22:13:15.2696234Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:13:15.2697002Z }, 2024-08-20T22:13:15.2697362Z { 2024-08-20T22:13:15.2698002Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2698623Z "size": 32, 2024-08-20T22:13:15.2699282Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:13:15.2700061Z }, 2024-08-20T22:13:15.2700415Z { 2024-08-20T22:13:15.2700935Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2701650Z "size": 32, 2024-08-20T22:13:15.2702320Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:13:15.2703005Z }, 2024-08-20T22:13:15.2703410Z { 2024-08-20T22:13:15.2703968Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:13:15.2704675Z "size": 32, 2024-08-20T22:13:15.2705330Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:13:15.2706071Z } 2024-08-20T22:13:15.2706424Z ] 2024-08-20T22:13:15.2706820Z } 2024-08-20T22:13:15.3012912Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:13:15.3013424Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:13:15.3013969Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-20T22:13:15.3023334Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:15.3023831Z env: 2024-08-20T22:13:15.3024110Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:15.3025289Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:15.3026484Z ##[endgroup] 2024-08-20T22:13:15.3061140Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:15.3114359Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-20T22:13:15.3115114Z with: 2024-08-20T22:13:15.3116200Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:15.3117535Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:15.3118051Z env: 2024-08-20T22:13:15.3118332Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:15.3118681Z ##[endgroup] 2024-08-20T22:13:15.3135986Z ##[group]Run set -x 2024-08-20T22:13:15.3136478Z set -x 2024-08-20T22:13:15.3136795Z set +e 2024-08-20T22:13:15.3137101Z  2024-08-20T22:13:15.3137385Z login() { 2024-08-20T22:13:15.3138061Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:13:15.3138838Z } 2024-08-20T22:13:15.3139133Z  2024-08-20T22:13:15.3139441Z retry () { 2024-08-20T22:13:15.3139840Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:13:15.3140297Z } 2024-08-20T22:13:15.3140583Z  2024-08-20T22:13:15.3140907Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:13:15.3141327Z  2024-08-20T22:13:15.3141609Z set -e 2024-08-20T22:13:15.3142093Z # ignore output since only exit code is used for conditional 2024-08-20T22:13:15.3142804Z # only pull docker image if it's not available locally 2024-08-20T22:13:15.3143578Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-20T22:13:15.3144295Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-20T22:13:15.3144746Z fi 2024-08-20T22:13:15.3153311Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:15.3153804Z env: 2024-08-20T22:13:15.3154080Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:15.3155231Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:15.3156539Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:15.3157054Z ##[endgroup] 2024-08-20T22:13:15.3183989Z + set +e 2024-08-20T22:13:15.3184519Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:15.3185160Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:15.3187955Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:13:15.3189179Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:13:15.8499310Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:13:15.8500120Z Configure a credential helper to remove this warning. See 2024-08-20T22:13:15.8501009Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:13:15.8501658Z 2024-08-20T22:13:15.8501790Z Login Succeeded 2024-08-20T22:13:15.8524171Z + set -e 2024-08-20T22:13:15.8525501Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:15.8685500Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:15.8687666Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:16.1120839Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-08-20T22:13:16.1122101Z 7a2c55901189: Pulling fs layer 2024-08-20T22:13:16.1122597Z 224fe954d725: Pulling fs layer 2024-08-20T22:13:16.1123603Z 75722010b82e: Pulling fs layer 2024-08-20T22:13:16.1124146Z d527cbbb87e3: Pulling fs layer 2024-08-20T22:13:16.1124629Z b57676e46aee: Pulling fs layer 2024-08-20T22:13:16.1125125Z a8c1e85b5e14: Pulling fs layer 2024-08-20T22:13:16.1125630Z a41a8d1c11c8: Pulling fs layer 2024-08-20T22:13:16.1126127Z 0c1227890755: Pulling fs layer 2024-08-20T22:13:16.1126607Z d8d1234baab3: Pulling fs layer 2024-08-20T22:13:16.1127103Z 7ed32bc8e469: Pulling fs layer 2024-08-20T22:13:16.1127589Z ec1e7978c1fe: Pulling fs layer 2024-08-20T22:13:16.1128091Z ac23c5fcc1ef: Pulling fs layer 2024-08-20T22:13:16.1128585Z ca426ffc9d2b: Pulling fs layer 2024-08-20T22:13:16.1129351Z 4373aa743caf: Pulling fs layer 2024-08-20T22:13:16.1129840Z 75b2abb16c68: Pulling fs layer 2024-08-20T22:13:16.1130326Z ef1e3ef1e85f: Pulling fs layer 2024-08-20T22:13:16.1130886Z 0bf61114d37b: Pulling fs layer 2024-08-20T22:13:16.1131377Z 2df849f72ba4: Pulling fs layer 2024-08-20T22:13:16.1131791Z 34c47d63c5de: Pulling fs layer 2024-08-20T22:13:16.1132175Z 1fef4bd19792: Pulling fs layer 2024-08-20T22:13:16.1132550Z 6efb96fc87ea: Pulling fs layer 2024-08-20T22:13:16.1132928Z b119c4fc6c58: Pulling fs layer 2024-08-20T22:13:16.1133298Z 33b639226eee: Pulling fs layer 2024-08-20T22:13:16.1133667Z d3547bd43146: Pulling fs layer 2024-08-20T22:13:16.1134038Z 5645520256fa: Pulling fs layer 2024-08-20T22:13:16.1134406Z 829ed7d91e38: Pulling fs layer 2024-08-20T22:13:16.1134777Z 385d0421d67c: Pulling fs layer 2024-08-20T22:13:16.1135147Z 7cbfce999d76: Pulling fs layer 2024-08-20T22:13:16.1135517Z 6658e558ebe4: Pulling fs layer 2024-08-20T22:13:16.1135880Z d527cbbb87e3: Waiting 2024-08-20T22:13:16.1136225Z 33444a5a7e01: Pulling fs layer 2024-08-20T22:13:16.1136583Z b57676e46aee: Waiting 2024-08-20T22:13:16.1136921Z 7c47122f4e09: Pulling fs layer 2024-08-20T22:13:16.1137293Z 1811ff4d3556: Pulling fs layer 2024-08-20T22:13:16.1137660Z 7754ee068b33: Pulling fs layer 2024-08-20T22:13:16.1138019Z a8c1e85b5e14: Waiting 2024-08-20T22:13:16.1138358Z 0479fbac3361: Pulling fs layer 2024-08-20T22:13:16.1138731Z f1d6874dee9c: Pulling fs layer 2024-08-20T22:13:16.1139101Z afc429e87b59: Pulling fs layer 2024-08-20T22:13:16.1139459Z a41a8d1c11c8: Waiting 2024-08-20T22:13:16.1139792Z 019620471c25: Pulling fs layer 2024-08-20T22:13:16.1140146Z 0c1227890755: Waiting 2024-08-20T22:13:16.1140477Z 27aeb1b2cd5e: Pulling fs layer 2024-08-20T22:13:16.1140840Z d8d1234baab3: Waiting 2024-08-20T22:13:16.1141174Z 829c0e19f4b7: Pulling fs layer 2024-08-20T22:13:16.1141536Z 560cd6122a38: Pulling fs layer 2024-08-20T22:13:16.1141892Z 7ed32bc8e469: Waiting 2024-08-20T22:13:16.1142218Z ca426ffc9d2b: Waiting 2024-08-20T22:13:16.1142551Z 5a4eb19d7219: Pulling fs layer 2024-08-20T22:13:16.1142917Z da6a838a66c7: Pulling fs layer 2024-08-20T22:13:16.1143295Z cbb79a705353: Pulling fs layer 2024-08-20T22:13:16.1143667Z 4d4fa6802ab3: Pulling fs layer 2024-08-20T22:13:16.1144013Z 5645520256fa: Waiting 2024-08-20T22:13:16.1144341Z 0f085d4fa279: Pulling fs layer 2024-08-20T22:13:16.1144696Z 75b2abb16c68: Waiting 2024-08-20T22:13:16.1145027Z 4a0412373305: Pulling fs layer 2024-08-20T22:13:16.1145376Z 019620471c25: Waiting 2024-08-20T22:13:16.1145930Z 302c8c177944: Pulling fs layer 2024-08-20T22:13:16.1146291Z ef1e3ef1e85f: Waiting 2024-08-20T22:13:16.1146633Z 02970347dff6: Pulling fs layer 2024-08-20T22:13:16.1146995Z 34c47d63c5de: Waiting 2024-08-20T22:13:16.1147338Z 916c5e858016: Pulling fs layer 2024-08-20T22:13:16.1147707Z 829ed7d91e38: Waiting 2024-08-20T22:13:16.1148025Z 385d0421d67c: Waiting 2024-08-20T22:13:16.1148357Z 7cbfce999d76: Waiting 2024-08-20T22:13:16.1148680Z 0bf61114d37b: Waiting 2024-08-20T22:13:16.1149029Z 7754ee068b33: Waiting 2024-08-20T22:13:16.1149373Z 4373aa743caf: Waiting 2024-08-20T22:13:16.1149694Z 0479fbac3361: Waiting 2024-08-20T22:13:16.1150013Z 1fef4bd19792: Waiting 2024-08-20T22:13:16.1150350Z c42b3602a1fb: Pulling fs layer 2024-08-20T22:13:16.1150734Z 11fad972ef6c: Pulling fs layer 2024-08-20T22:13:16.1151106Z f1d6874dee9c: Waiting 2024-08-20T22:13:16.1151622Z f957c1624481: Pulling fs layer 2024-08-20T22:13:16.1151993Z afc429e87b59: Waiting 2024-08-20T22:13:16.1152334Z 768c31dca016: Pulling fs layer 2024-08-20T22:13:16.1152698Z 829c0e19f4b7: Waiting 2024-08-20T22:13:16.1153014Z 560cd6122a38: Waiting 2024-08-20T22:13:16.1153344Z 8c32d0ecebbe: Pulling fs layer 2024-08-20T22:13:16.1153734Z 4f4fb700ef54: Pulling fs layer 2024-08-20T22:13:16.1154103Z da6a838a66c7: Waiting 2024-08-20T22:13:16.1154425Z cbb79a705353: Waiting 2024-08-20T22:13:16.1154747Z 5a4eb19d7219: Waiting 2024-08-20T22:13:16.1155088Z 78ee77eddef2: Pulling fs layer 2024-08-20T22:13:16.1155470Z 37513e9b43d7: Pulling fs layer 2024-08-20T22:13:16.1155973Z a7a5392e2bf6: Pulling fs layer 2024-08-20T22:13:16.1156345Z ec1e7978c1fe: Waiting 2024-08-20T22:13:16.1156672Z ac23c5fcc1ef: Waiting 2024-08-20T22:13:16.1157010Z b10de02255c0: Pulling fs layer 2024-08-20T22:13:16.1157374Z 4d4fa6802ab3: Waiting 2024-08-20T22:13:16.1157696Z 0f085d4fa279: Waiting 2024-08-20T22:13:16.1158038Z e56eeff2db93: Pulling fs layer 2024-08-20T22:13:16.1158416Z 11fad972ef6c: Waiting 2024-08-20T22:13:16.1158763Z 70ac3e38d0fd: Pulling fs layer 2024-08-20T22:13:16.1159137Z 02970347dff6: Waiting 2024-08-20T22:13:16.1159481Z eb09cbf3e416: Pulling fs layer 2024-08-20T22:13:16.1159859Z ee1273153675: Pulling fs layer 2024-08-20T22:13:16.1160237Z e22979f9e0ee: Pulling fs layer 2024-08-20T22:13:16.1160610Z c42b3602a1fb: Waiting 2024-08-20T22:13:16.1160960Z 6c1f0aabce7c: Pulling fs layer 2024-08-20T22:13:16.1161341Z 7ebe49c3a23e: Pulling fs layer 2024-08-20T22:13:16.1161724Z 5d115b29b5d5: Pulling fs layer 2024-08-20T22:13:16.1162093Z 916c5e858016: Waiting 2024-08-20T22:13:16.1162424Z 7f2d7a04fbe6: Pulling fs layer 2024-08-20T22:13:16.1162806Z 768c31dca016: Waiting 2024-08-20T22:13:16.1163142Z d79fd7d52698: Pulling fs layer 2024-08-20T22:13:16.1163613Z 9752d6fbbcab: Pulling fs layer 2024-08-20T22:13:16.1163973Z 302c8c177944: Waiting 2024-08-20T22:13:16.1164309Z 83808b2eff4d: Pulling fs layer 2024-08-20T22:13:16.1164679Z 33444a5a7e01: Waiting 2024-08-20T22:13:16.1165005Z 6efb96fc87ea: Waiting 2024-08-20T22:13:16.1165369Z 6658e558ebe4: Waiting 2024-08-20T22:13:16.1165703Z 8c32d0ecebbe: Waiting 2024-08-20T22:13:16.1166028Z d3547bd43146: Waiting 2024-08-20T22:13:16.1166345Z 7ebe49c3a23e: Waiting 2024-08-20T22:13:16.1166669Z 4f4fb700ef54: Waiting 2024-08-20T22:13:16.1166993Z 5d115b29b5d5: Waiting 2024-08-20T22:13:16.1167319Z a7a5392e2bf6: Waiting 2024-08-20T22:13:16.1167639Z 78ee77eddef2: Waiting 2024-08-20T22:13:16.1167960Z 7c47122f4e09: Waiting 2024-08-20T22:13:16.1168279Z b10de02255c0: Waiting 2024-08-20T22:13:16.1168607Z e56eeff2db93: Waiting 2024-08-20T22:13:16.1168932Z 37513e9b43d7: Waiting 2024-08-20T22:13:16.1169260Z f957c1624481: Waiting 2024-08-20T22:13:16.1169578Z 83808b2eff4d: Waiting 2024-08-20T22:13:16.1169900Z 6c1f0aabce7c: Waiting 2024-08-20T22:13:16.1170225Z 1811ff4d3556: Waiting 2024-08-20T22:13:16.1170540Z 4a0412373305: Waiting 2024-08-20T22:13:16.1170853Z e22979f9e0ee: Waiting 2024-08-20T22:13:16.1171179Z eb09cbf3e416: Waiting 2024-08-20T22:13:16.1171514Z 2df849f72ba4: Waiting 2024-08-20T22:13:16.1171841Z 70ac3e38d0fd: Waiting 2024-08-20T22:13:16.1172161Z 9752d6fbbcab: Waiting 2024-08-20T22:13:16.2758496Z 224fe954d725: Download complete 2024-08-20T22:13:16.3380395Z d527cbbb87e3: Download complete 2024-08-20T22:13:16.4153943Z b57676e46aee: Verifying Checksum 2024-08-20T22:13:16.4154361Z b57676e46aee: Download complete 2024-08-20T22:13:16.4588565Z 7a2c55901189: Download complete 2024-08-20T22:13:16.5367317Z a41a8d1c11c8: Download complete 2024-08-20T22:13:16.6073462Z 0c1227890755: Verifying Checksum 2024-08-20T22:13:16.6073939Z 0c1227890755: Download complete 2024-08-20T22:13:16.6721510Z d8d1234baab3: Verifying Checksum 2024-08-20T22:13:16.6722108Z d8d1234baab3: Download complete 2024-08-20T22:13:16.7166050Z 75722010b82e: Verifying Checksum 2024-08-20T22:13:16.7166522Z 75722010b82e: Download complete 2024-08-20T22:13:16.7882629Z ec1e7978c1fe: Download complete 2024-08-20T22:13:16.8672301Z ac23c5fcc1ef: Download complete 2024-08-20T22:13:17.6106006Z 7a2c55901189: Pull complete 2024-08-20T22:13:17.9373424Z 224fe954d725: Pull complete 2024-08-20T22:13:18.8776617Z 75722010b82e: Pull complete 2024-08-20T22:13:18.8990925Z d527cbbb87e3: Pull complete 2024-08-20T22:13:18.9218426Z b57676e46aee: Pull complete 2024-08-20T22:13:19.3951566Z ca426ffc9d2b: Download complete 2024-08-20T22:13:19.4651684Z 4373aa743caf: Download complete 2024-08-20T22:13:19.5406683Z 75b2abb16c68: Verifying Checksum 2024-08-20T22:13:19.6248175Z ef1e3ef1e85f: Download complete 2024-08-20T22:13:20.5940301Z 0bf61114d37b: Verifying Checksum 2024-08-20T22:13:20.5940934Z 0bf61114d37b: Download complete 2024-08-20T22:13:20.6793946Z 2df849f72ba4: Download complete 2024-08-20T22:13:20.7518132Z 34c47d63c5de: Verifying Checksum 2024-08-20T22:13:20.7518769Z 34c47d63c5de: Download complete 2024-08-20T22:13:20.8246466Z 1fef4bd19792: Download complete 2024-08-20T22:13:29.7501420Z a8c1e85b5e14: Verifying Checksum 2024-08-20T22:13:29.7501947Z a8c1e85b5e14: Download complete 2024-08-20T22:13:29.8135492Z b119c4fc6c58: Verifying Checksum 2024-08-20T22:13:29.8135986Z b119c4fc6c58: Download complete 2024-08-20T22:13:29.9012607Z 33b639226eee: Verifying Checksum 2024-08-20T22:13:29.9013185Z 33b639226eee: Download complete 2024-08-20T22:13:29.9748458Z d3547bd43146: Verifying Checksum 2024-08-20T22:13:29.9749038Z d3547bd43146: Download complete 2024-08-20T22:13:30.0569613Z 5645520256fa: Verifying Checksum 2024-08-20T22:13:30.0570170Z 5645520256fa: Download complete 2024-08-20T22:13:30.1246616Z 829ed7d91e38: Verifying Checksum 2024-08-20T22:13:30.1247105Z 829ed7d91e38: Download complete 2024-08-20T22:13:31.3917109Z 385d0421d67c: Verifying Checksum 2024-08-20T22:13:31.3917786Z 385d0421d67c: Download complete 2024-08-20T22:13:31.4520067Z 7cbfce999d76: Download complete 2024-08-20T22:13:31.5152341Z 6658e558ebe4: Verifying Checksum 2024-08-20T22:13:31.5152857Z 6658e558ebe4: Download complete 2024-08-20T22:13:31.5794153Z 33444a5a7e01: Verifying Checksum 2024-08-20T22:13:31.5794578Z 33444a5a7e01: Download complete 2024-08-20T22:13:31.6481454Z 7c47122f4e09: Verifying Checksum 2024-08-20T22:13:31.6481875Z 7c47122f4e09: Download complete 2024-08-20T22:13:31.7150220Z 1811ff4d3556: Download complete 2024-08-20T22:13:37.7101449Z 7754ee068b33: Verifying Checksum 2024-08-20T22:13:37.7102336Z 7754ee068b33: Download complete 2024-08-20T22:13:37.7713514Z 0479fbac3361: Download complete 2024-08-20T22:13:37.9123463Z f1d6874dee9c: Download complete 2024-08-20T22:13:38.3723040Z afc429e87b59: Verifying Checksum 2024-08-20T22:13:38.3723719Z afc429e87b59: Download complete 2024-08-20T22:13:38.4475741Z 019620471c25: Download complete 2024-08-20T22:13:38.5158204Z 27aeb1b2cd5e: Verifying Checksum 2024-08-20T22:13:38.5158801Z 27aeb1b2cd5e: Download complete 2024-08-20T22:13:38.7700897Z 829c0e19f4b7: Verifying Checksum 2024-08-20T22:13:38.7701430Z 829c0e19f4b7: Download complete 2024-08-20T22:13:38.8394098Z 560cd6122a38: Verifying Checksum 2024-08-20T22:13:38.8394723Z 560cd6122a38: Download complete 2024-08-20T22:13:38.9069847Z 5a4eb19d7219: Verifying Checksum 2024-08-20T22:13:38.9070388Z 5a4eb19d7219: Download complete 2024-08-20T22:13:38.9812164Z da6a838a66c7: Verifying Checksum 2024-08-20T22:13:38.9812761Z da6a838a66c7: Download complete 2024-08-20T22:13:42.0097363Z 7ed32bc8e469: Verifying Checksum 2024-08-20T22:13:42.0097969Z 7ed32bc8e469: Download complete 2024-08-20T22:13:42.0717882Z 4d4fa6802ab3: Verifying Checksum 2024-08-20T22:13:42.0718505Z 4d4fa6802ab3: Download complete 2024-08-20T22:13:42.1487665Z 0f085d4fa279: Verifying Checksum 2024-08-20T22:13:42.1488266Z 0f085d4fa279: Download complete 2024-08-20T22:13:42.6411273Z 4a0412373305: Verifying Checksum 2024-08-20T22:13:42.6411749Z 4a0412373305: Download complete 2024-08-20T22:13:42.7147481Z 302c8c177944: Verifying Checksum 2024-08-20T22:13:42.7148089Z 302c8c177944: Download complete 2024-08-20T22:13:42.7885328Z 02970347dff6: Verifying Checksum 2024-08-20T22:13:42.7885908Z 02970347dff6: Download complete 2024-08-20T22:13:42.8500746Z a8c1e85b5e14: Pull complete 2024-08-20T22:13:42.8634145Z 916c5e858016: Download complete 2024-08-20T22:13:42.9361741Z c42b3602a1fb: Download complete 2024-08-20T22:13:42.9934193Z a41a8d1c11c8: Pull complete 2024-08-20T22:13:43.1350334Z 0c1227890755: Pull complete 2024-08-20T22:13:43.2967037Z d8d1234baab3: Pull complete 2024-08-20T22:13:49.5395579Z 6efb96fc87ea: Verifying Checksum 2024-08-20T22:13:49.5396183Z 6efb96fc87ea: Download complete 2024-08-20T22:13:49.6125579Z f957c1624481: Download complete 2024-08-20T22:13:49.6848048Z 768c31dca016: Verifying Checksum 2024-08-20T22:13:49.6848697Z 768c31dca016: Download complete 2024-08-20T22:13:49.7669210Z 8c32d0ecebbe: Verifying Checksum 2024-08-20T22:13:49.7669955Z 8c32d0ecebbe: Download complete 2024-08-20T22:13:49.7749115Z 4f4fb700ef54: Download complete 2024-08-20T22:13:49.8623799Z 78ee77eddef2: Download complete 2024-08-20T22:13:49.9347754Z 37513e9b43d7: Download complete 2024-08-20T22:13:50.0622600Z a7a5392e2bf6: Verifying Checksum 2024-08-20T22:13:50.0623076Z a7a5392e2bf6: Download complete 2024-08-20T22:13:50.1323894Z b10de02255c0: Download complete 2024-08-20T22:13:50.2059710Z e56eeff2db93: Verifying Checksum 2024-08-20T22:13:50.2060202Z e56eeff2db93: Download complete 2024-08-20T22:13:50.2764392Z 70ac3e38d0fd: Verifying Checksum 2024-08-20T22:13:50.2764926Z 70ac3e38d0fd: Download complete 2024-08-20T22:13:50.3743209Z eb09cbf3e416: Download complete 2024-08-20T22:13:50.5010079Z ee1273153675: Download complete 2024-08-20T22:13:50.5694803Z e22979f9e0ee: Verifying Checksum 2024-08-20T22:13:50.5695596Z e22979f9e0ee: Download complete 2024-08-20T22:13:51.1542557Z 6c1f0aabce7c: Verifying Checksum 2024-08-20T22:13:51.1543425Z 6c1f0aabce7c: Download complete 2024-08-20T22:13:51.2272461Z 7ebe49c3a23e: Verifying Checksum 2024-08-20T22:13:51.2273085Z 7ebe49c3a23e: Download complete 2024-08-20T22:14:16.9951396Z 5d115b29b5d5: Verifying Checksum 2024-08-20T22:14:16.9951907Z 5d115b29b5d5: Download complete 2024-08-20T22:14:17.0649571Z 7f2d7a04fbe6: Download complete 2024-08-20T22:14:17.1320955Z d79fd7d52698: Download complete 2024-08-20T22:14:18.5272640Z 9752d6fbbcab: Verifying Checksum 2024-08-20T22:14:18.5273153Z 9752d6fbbcab: Download complete 2024-08-20T22:14:18.5832068Z 83808b2eff4d: Verifying Checksum 2024-08-20T22:14:18.5832595Z 83808b2eff4d: Download complete 2024-08-20T22:14:56.2866912Z 11fad972ef6c: Verifying Checksum 2024-08-20T22:14:56.2867679Z 11fad972ef6c: Download complete 2024-08-20T22:14:58.2782682Z cbb79a705353: Verifying Checksum 2024-08-20T22:14:58.2783316Z cbb79a705353: Download complete 2024-08-20T22:15:12.9211172Z 7ed32bc8e469: Pull complete 2024-08-20T22:15:13.1318293Z ec1e7978c1fe: Pull complete 2024-08-20T22:15:13.3418836Z ac23c5fcc1ef: Pull complete 2024-08-20T22:15:22.1358900Z ca426ffc9d2b: Pull complete 2024-08-20T22:15:22.2603989Z 4373aa743caf: Pull complete 2024-08-20T22:15:22.4046819Z 75b2abb16c68: Pull complete 2024-08-20T22:15:22.5534959Z ef1e3ef1e85f: Pull complete 2024-08-20T22:15:25.0843687Z 0bf61114d37b: Pull complete 2024-08-20T22:15:25.1577821Z 2df849f72ba4: Pull complete 2024-08-20T22:15:25.2085614Z 34c47d63c5de: Pull complete 2024-08-20T22:15:25.3213300Z 1fef4bd19792: Pull complete 2024-08-20T22:16:23.6761890Z 6efb96fc87ea: Pull complete 2024-08-20T22:16:23.8157558Z b119c4fc6c58: Pull complete 2024-08-20T22:16:23.9783206Z 33b639226eee: Pull complete 2024-08-20T22:16:24.1313640Z d3547bd43146: Pull complete 2024-08-20T22:16:24.2796826Z 5645520256fa: Pull complete 2024-08-20T22:16:24.4189786Z 829ed7d91e38: Pull complete 2024-08-20T22:16:27.8568271Z 385d0421d67c: Pull complete 2024-08-20T22:16:28.0294467Z 7cbfce999d76: Pull complete 2024-08-20T22:16:28.1698128Z 6658e558ebe4: Pull complete 2024-08-20T22:16:28.3134624Z 33444a5a7e01: Pull complete 2024-08-20T22:16:28.4864045Z 7c47122f4e09: Pull complete 2024-08-20T22:16:28.6996559Z 1811ff4d3556: Pull complete 2024-08-20T22:16:38.9339463Z 7754ee068b33: Pull complete 2024-08-20T22:16:39.1190531Z 0479fbac3361: Pull complete 2024-08-20T22:16:39.3201639Z f1d6874dee9c: Pull complete 2024-08-20T22:16:40.2950047Z afc429e87b59: Pull complete 2024-08-20T22:16:40.5008760Z 019620471c25: Pull complete 2024-08-20T22:16:40.6854437Z 27aeb1b2cd5e: Pull complete 2024-08-20T22:16:41.1615182Z 829c0e19f4b7: Pull complete 2024-08-20T22:16:41.3763806Z 560cd6122a38: Pull complete 2024-08-20T22:16:41.7553758Z 5a4eb19d7219: Pull complete 2024-08-20T22:16:41.9614185Z da6a838a66c7: Pull complete 2024-08-20T22:17:20.7565125Z cbb79a705353: Pull complete 2024-08-20T22:17:20.9763763Z 4d4fa6802ab3: Pull complete 2024-08-20T22:17:21.1826770Z 0f085d4fa279: Pull complete 2024-08-20T22:17:23.5783274Z 4a0412373305: Pull complete 2024-08-20T22:17:23.5981280Z 302c8c177944: Pull complete 2024-08-20T22:17:23.6167744Z 02970347dff6: Pull complete 2024-08-20T22:17:23.6548473Z 916c5e858016: Pull complete 2024-08-20T22:17:23.6731101Z c42b3602a1fb: Pull complete 2024-08-20T22:18:45.5043943Z 11fad972ef6c: Pull complete 2024-08-20T22:18:45.7189970Z f957c1624481: Pull complete 2024-08-20T22:18:45.9173788Z 768c31dca016: Pull complete 2024-08-20T22:18:46.3192518Z 8c32d0ecebbe: Pull complete 2024-08-20T22:18:46.5333293Z 4f4fb700ef54: Pull complete 2024-08-20T22:18:46.7605030Z 78ee77eddef2: Pull complete 2024-08-20T22:18:46.9720906Z 37513e9b43d7: Pull complete 2024-08-20T22:18:47.3103919Z a7a5392e2bf6: Pull complete 2024-08-20T22:18:47.5224099Z b10de02255c0: Pull complete 2024-08-20T22:18:47.7426273Z e56eeff2db93: Pull complete 2024-08-20T22:18:47.9576037Z 70ac3e38d0fd: Pull complete 2024-08-20T22:18:48.1726384Z eb09cbf3e416: Pull complete 2024-08-20T22:18:49.5837293Z ee1273153675: Pull complete 2024-08-20T22:18:49.7998582Z e22979f9e0ee: Pull complete 2024-08-20T22:18:51.9790610Z 6c1f0aabce7c: Pull complete 2024-08-20T22:18:52.1943551Z 7ebe49c3a23e: Pull complete 2024-08-20T22:19:05.6682310Z 5d115b29b5d5: Pull complete 2024-08-20T22:19:05.8571829Z 7f2d7a04fbe6: Pull complete 2024-08-20T22:19:06.0498066Z d79fd7d52698: Pull complete 2024-08-20T22:19:06.9038532Z 9752d6fbbcab: Pull complete 2024-08-20T22:19:07.1061939Z 83808b2eff4d: Pull complete 2024-08-20T22:19:08.0293873Z Digest: sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-20T22:19:08.0725370Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:19:08.1023599Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:19:08.1149736Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:19:08.1150673Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:19:08.1162266Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:19:08.1162851Z env: 2024-08-20T22:19:08.1163127Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:08.1163467Z ##[endgroup] 2024-08-20T22:19:08.1373790Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-20T22:19:08.1374354Z with: 2024-08-20T22:19:08.1374639Z driver-version: 550.54.15 2024-08-20T22:19:08.1374974Z env: 2024-08-20T22:19:08.1375251Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:08.1375598Z ##[endgroup] 2024-08-20T22:19:08.1508185Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:19:08.1508758Z with: 2024-08-20T22:19:08.1509045Z timeout_minutes: 10 2024-08-20T22:19:08.1509369Z max_attempts: 3 2024-08-20T22:19:08.1543973Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-20T22:19:08.1577622Z retry_wait_seconds: 10 2024-08-20T22:19:08.1577998Z polling_interval_seconds: 1 2024-08-20T22:19:08.1578369Z warning_on_retry: true 2024-08-20T22:19:08.1578724Z continue_on_error: false 2024-08-20T22:19:08.1579061Z env: 2024-08-20T22:19:08.1579334Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:19:08.1579688Z DRIVER_VERSION: 550.54.15 2024-08-20T22:19:08.1580039Z ##[endgroup] 2024-08-20T22:19:08.2386584Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-20T22:19:08.2387413Z + pre_install_nvidia_driver_amzn2 2024-08-20T22:19:08.2389763Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-20T22:19:08.6933095Z No match for argument: nvidia-driver-latest-dkms 2024-08-20T22:19:08.6933616Z No packages marked for removal. 2024-08-20T22:19:08.6997455Z Dependencies resolved. 2024-08-20T22:19:08.7007439Z Nothing to do. 2024-08-20T22:19:08.7007748Z Complete! 2024-08-20T22:19:08.7822014Z + install_nvidia_driver_common 2024-08-20T22:19:08.7825554Z + echo 'Before installing NVIDIA driver' 2024-08-20T22:19:08.7826017Z Before installing NVIDIA driver 2024-08-20T22:19:08.7826795Z + lspci 2024-08-20T22:19:08.8818125Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:19:08.8818890Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:19:08.8819822Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:19:08.8820621Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:19:08.8824233Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:19:08.8825092Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:19:08.8825850Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:19:08.8826705Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:19:08.8827355Z + lsmod 2024-08-20T22:19:08.8867058Z Module Size Used by 2024-08-20T22:19:08.8867518Z xt_conntrack 16384 1 2024-08-20T22:19:08.8867898Z nft_chain_nat 16384 3 2024-08-20T22:19:08.8868277Z xt_MASQUERADE 20480 1 2024-08-20T22:19:08.8868731Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-20T22:19:08.8869234Z nf_conntrack_netlink 57344 0 2024-08-20T22:19:08.8869812Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:19:08.8874024Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:19:08.8874625Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:19:08.8875059Z xfrm_user 57344 1 2024-08-20T22:19:08.8875467Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:19:08.8876065Z xt_addrtype 16384 2 2024-08-20T22:19:08.8876469Z nft_compat 20480 4 2024-08-20T22:19:08.8876924Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:19:08.8877577Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:19:08.8878152Z br_netfilter 36864 0 2024-08-20T22:19:08.8878651Z bridge 307200 1 br_netfilter 2024-08-20T22:19:08.8879108Z stp 16384 1 bridge 2024-08-20T22:19:08.8879540Z llc 16384 2 bridge,stp 2024-08-20T22:19:08.8879961Z overlay 167936 0 2024-08-20T22:19:08.8880327Z tls 114688 0 2024-08-20T22:19:08.8880686Z sunrpc 692224 1 2024-08-20T22:19:08.8881055Z nls_ascii 16384 1 2024-08-20T22:19:08.8881490Z nls_cp437 20480 1 2024-08-20T22:19:08.8881854Z vfat 24576 1 2024-08-20T22:19:08.8882225Z fat 86016 1 vfat 2024-08-20T22:19:08.8882925Z ghash_clmulni_intel 16384 0 2024-08-20T22:19:08.8883310Z ena 167936 0 2024-08-20T22:19:08.8883694Z ptp 36864 1 ena 2024-08-20T22:19:08.8884088Z pps_core 24576 1 ptp 2024-08-20T22:19:08.8884602Z aesni_intel 393216 0 2024-08-20T22:19:08.8884985Z i8042 45056 0 2024-08-20T22:19:08.8885360Z serio 28672 3 i8042 2024-08-20T22:19:08.8885797Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:19:08.8886349Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:19:08.8886864Z button 24576 0 2024-08-20T22:19:08.8887245Z sch_fq_codel 20480 17 2024-08-20T22:19:08.8887631Z dm_mod 188416 0 2024-08-20T22:19:08.8888003Z fuse 163840 1 2024-08-20T22:19:08.8888383Z loop 36864 0 2024-08-20T22:19:08.8888755Z configfs 57344 1 2024-08-20T22:19:08.8889148Z dax 45056 1 dm_mod 2024-08-20T22:19:08.8889570Z dmi_sysfs 20480 0 2024-08-20T22:19:08.8889950Z crc32_pclmul 16384 0 2024-08-20T22:19:08.8890321Z crc32c_intel 24576 0 2024-08-20T22:19:08.8890706Z efivarfs 24576 1 2024-08-20T22:19:08.8891086Z + modinfo nvidia 2024-08-20T22:19:08.8891888Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:19:08.8892703Z alias: char-major-195-* 2024-08-20T22:19:08.8893083Z version: 550.54.15 2024-08-20T22:19:08.8893438Z supported: external 2024-08-20T22:19:08.8893776Z license: NVIDIA 2024-08-20T22:19:08.8894146Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:19:08.8894635Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:19:08.8895099Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:19:08.8895556Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:19:08.8896064Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:19:08.8896567Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:19:08.8897153Z depends: i2c-core,drm 2024-08-20T22:19:08.8897613Z retpoline: Y 2024-08-20T22:19:08.8898006Z name: nvidia 2024-08-20T22:19:08.8898794Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:19:08.8899505Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:19:08.8900136Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:19:08.8900731Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:19:08.8901168Z parm: NVreg_RmLogonRC:int 2024-08-20T22:19:08.8901598Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:19:08.8902045Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:19:08.8902474Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:19:08.8902904Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:19:08.8903425Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:19:08.8903984Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:19:08.8904459Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:19:08.8904884Z parm: NVreg_EnableMSI:int 2024-08-20T22:19:08.8905347Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:19:08.8905908Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:19:08.8906439Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:19:08.8907015Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:19:08.8907565Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:19:08.8908171Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:19:08.8908771Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:19:08.8909378Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:19:08.8909970Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:19:08.8910463Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:19:08.8911100Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:19:08.8911655Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:19:08.8912145Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:19:08.8912600Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:19:08.8913083Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:19:08.8913549Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:19:08.8913981Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:19:08.8914485Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:19:08.8915009Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:19:08.8915482Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:19:08.8915954Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:19:08.8916450Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:19:08.8916927Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:19:08.8917423Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:19:08.8917908Z parm: NVreg_RmMsg:charp 2024-08-20T22:19:08.8918313Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:19:08.8918781Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:19:08.8919252Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:19:08.8919707Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:19:08.8920179Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:19:08.8920645Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:19:08.8921099Z parm: rm_firmware_active:charp 2024-08-20T22:19:08.8921508Z + HAS_NVIDIA_DRIVER=0 2024-08-20T22:19:08.8921995Z ++ command -v nvidia-smi 2024-08-20T22:19:08.8922402Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-20T22:19:08.8922755Z + set +e 2024-08-20T22:19:08.8923259Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-20T22:19:11.1364346Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-20T22:19:11.1364961Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:19:11.1365579Z + '[' 0 -ne 0 ']' 2024-08-20T22:19:11.1365946Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-20T22:19:11.1366333Z + HAS_NVIDIA_DRIVER=1 2024-08-20T22:19:11.1367375Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-20T22:19:11.1368100Z + set -e 2024-08-20T22:19:11.1368404Z + '[' 1 -eq 0 ']' 2024-08-20T22:19:11.1368982Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-20T22:19:11.1369677Z + post_install_nvidia_driver_common 2024-08-20T22:19:11.1370703Z + sudo modprobe nvidia 2024-08-20T22:19:11.2740340Z + echo 'After installing NVIDIA driver' 2024-08-20T22:19:11.2740894Z + lspci 2024-08-20T22:19:11.2741333Z After installing NVIDIA driver 2024-08-20T22:19:11.2847123Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:19:11.2847962Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:19:11.2848895Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:19:11.2849717Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:19:11.2850552Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:19:11.2851348Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:19:11.2852086Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:19:11.2852902Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:19:11.2853504Z + lsmod 2024-08-20T22:19:11.2881193Z Module Size Used by 2024-08-20T22:19:11.2881644Z nvidia_uvm 4706304 0 2024-08-20T22:19:11.2882176Z nvidia 54071296 1 nvidia_uvm 2024-08-20T22:19:11.2882744Z drm 602112 1 nvidia 2024-08-20T22:19:11.2883332Z drm_panel_orientation_quirks 28672 1 drm 2024-08-20T22:19:11.2883937Z backlight 24576 1 drm 2024-08-20T22:19:11.2884831Z i2c_core 106496 2 nvidia,drm 2024-08-20T22:19:11.2885269Z xt_conntrack 16384 1 2024-08-20T22:19:11.2885663Z nft_chain_nat 16384 3 2024-08-20T22:19:11.2886039Z xt_MASQUERADE 20480 1 2024-08-20T22:19:11.2886602Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-20T22:19:11.2887262Z nf_conntrack_netlink 57344 0 2024-08-20T22:19:11.2888041Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:19:11.2888856Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:19:11.2889323Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:19:11.2889759Z xfrm_user 57344 1 2024-08-20T22:19:11.2890157Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:19:11.2890568Z xt_addrtype 16384 2 2024-08-20T22:19:11.2890944Z nft_compat 20480 4 2024-08-20T22:19:11.2891398Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:19:11.2892019Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:19:11.2892563Z br_netfilter 36864 0 2024-08-20T22:19:11.2892975Z bridge 307200 1 br_netfilter 2024-08-20T22:19:11.2893419Z stp 16384 1 bridge 2024-08-20T22:19:11.2893837Z llc 16384 2 bridge,stp 2024-08-20T22:19:11.2894258Z overlay 167936 0 2024-08-20T22:19:11.2894623Z tls 114688 0 2024-08-20T22:19:11.2894986Z sunrpc 692224 1 2024-08-20T22:19:11.2895349Z nls_ascii 16384 1 2024-08-20T22:19:11.2895722Z nls_cp437 20480 1 2024-08-20T22:19:11.2896110Z vfat 24576 1 2024-08-20T22:19:11.2896476Z fat 86016 1 vfat 2024-08-20T22:19:11.2896867Z ghash_clmulni_intel 16384 0 2024-08-20T22:19:11.2897237Z ena 167936 0 2024-08-20T22:19:11.2897606Z ptp 36864 1 ena 2024-08-20T22:19:11.2898004Z pps_core 24576 1 ptp 2024-08-20T22:19:11.2898395Z aesni_intel 393216 0 2024-08-20T22:19:11.2898763Z i8042 45056 0 2024-08-20T22:19:11.2899134Z serio 28672 3 i8042 2024-08-20T22:19:11.2899755Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:19:11.2900282Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:19:11.2900923Z button 24576 0 2024-08-20T22:19:11.2901423Z sch_fq_codel 20480 17 2024-08-20T22:19:11.2901912Z dm_mod 188416 0 2024-08-20T22:19:11.2902387Z fuse 163840 1 2024-08-20T22:19:11.2902811Z loop 36864 0 2024-08-20T22:19:11.2903222Z configfs 57344 1 2024-08-20T22:19:11.2903596Z dax 45056 1 dm_mod 2024-08-20T22:19:11.2903993Z dmi_sysfs 20480 0 2024-08-20T22:19:11.2904365Z crc32_pclmul 16384 0 2024-08-20T22:19:11.2904736Z crc32c_intel 24576 0 2024-08-20T22:19:11.2905103Z efivarfs 24576 1 2024-08-20T22:19:11.2905462Z + modinfo nvidia 2024-08-20T22:19:11.2906267Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:19:11.2907225Z alias: char-major-195-* 2024-08-20T22:19:11.2907641Z version: 550.54.15 2024-08-20T22:19:11.2907984Z supported: external 2024-08-20T22:19:11.2908321Z license: NVIDIA 2024-08-20T22:19:11.2908700Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:19:11.2909178Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:19:11.2909638Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:19:11.2910103Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:19:11.2910600Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:19:11.2911088Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:19:11.2911589Z depends: i2c-core,drm 2024-08-20T22:19:11.2911946Z retpoline: Y 2024-08-20T22:19:11.2912246Z name: nvidia 2024-08-20T22:19:11.2913010Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:19:11.2913716Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:19:11.2914365Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:19:11.2914950Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:19:11.2915395Z parm: NVreg_RmLogonRC:int 2024-08-20T22:19:11.2915827Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:19:11.2916285Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:19:11.2916711Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:19:11.2917149Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:19:11.2917674Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:19:11.2918229Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:19:11.2918701Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:19:11.2919128Z parm: NVreg_EnableMSI:int 2024-08-20T22:19:11.2919551Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:19:11.2920002Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:19:11.2920529Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:19:11.2921113Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:19:11.2921672Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:19:11.2922272Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:19:11.2922866Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:19:11.2923546Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:19:11.2924138Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:19:11.2924703Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:19:11.2925235Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:19:11.2925779Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:19:11.2926276Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:19:11.2926737Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:19:11.2927219Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:19:11.2927691Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:19:11.2928250Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:19:11.2928755Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:19:11.2929280Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:19:11.2929756Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:19:11.2930239Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:19:11.2930733Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:19:11.2931220Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:19:11.2931780Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:19:11.2932263Z parm: NVreg_RmMsg:charp 2024-08-20T22:19:11.2932667Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:19:11.2933134Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:19:11.2933612Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:19:11.2934061Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:19:11.2934548Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:19:11.2935021Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:19:11.2935475Z parm: rm_firmware_active:charp 2024-08-20T22:19:11.2935876Z + set +e 2024-08-20T22:19:11.2936196Z + nvidia-smi 2024-08-20T22:19:12.8546508Z Tue Aug 20 22:19:12 2024 2024-08-20T22:19:12.8547372Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:19:12.8548523Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:19:12.8549333Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:19:12.8550134Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:19:12.8551299Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:19:12.8552003Z | | | MIG M. | 2024-08-20T22:19:12.8552559Z |=========================================+========================+======================| 2024-08-20T22:19:12.8649220Z | 0 NVIDIA A10G Off | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:19:12.8650198Z | 0% 30C P0 58W / 300W | 0MiB / 23028MiB | 5% Default | 2024-08-20T22:19:12.8650969Z | | | N/A | 2024-08-20T22:19:12.8651912Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:19:12.8652613Z 2024-08-20T22:19:12.8653277Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:19:12.8653917Z | Processes: | 2024-08-20T22:19:12.8654798Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:19:12.8655699Z | ID ID Usage | 2024-08-20T22:19:12.8656260Z |=========================================================================================| 2024-08-20T22:19:12.8656920Z | No running processes found | 2024-08-20T22:19:12.8657699Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:19:13.4720398Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-20T22:19:15.0351794Z NVIDIA A10G 2024-08-20T22:19:15.4889724Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:19:15.4890318Z + '[' 0 -eq 0 ']' 2024-08-20T22:19:15.4890745Z + echo 'INFO: Ignoring allowed status 0' 2024-08-20T22:19:15.4891179Z + set -e 2024-08-20T22:19:15.4891479Z INFO: Ignoring allowed status 0 2024-08-20T22:19:15.4899776Z == Installing nvidia container toolkit for amzn2023 == 2024-08-20T22:19:15.4903478Z + sudo yum install -y yum-utils 2024-08-20T22:19:15.9258663Z Last metadata expiration check: 0:09:35 ago on Tue Aug 20 22:09:40 2024. 2024-08-20T22:19:15.9483469Z Package dnf-utils-4.3.0-13.amzn2023.0.4.noarch is already installed. 2024-08-20T22:19:15.9791477Z Dependencies resolved. 2024-08-20T22:19:15.9916886Z Nothing to do. 2024-08-20T22:19:15.9917194Z Complete! 2024-08-20T22:19:16.0708943Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2024-08-20T22:19:16.0710345Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:19:16.0712163Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:19:16.3546580Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:19:16.4215075Z + sudo yum install -y nvidia-docker2 2024-08-20T22:19:16.9780754Z nvidia-container-toolkit 10 kB/s | 833 B 00:00 2024-08-20T22:19:17.0003697Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2024-08-20T22:19:17.0304474Z Dependencies resolved. 2024-08-20T22:19:17.0433046Z Nothing to do. 2024-08-20T22:19:17.0433362Z Complete! 2024-08-20T22:19:17.1314838Z + sudo systemctl restart docker 2024-08-20T22:20:11.6672494Z Tue Aug 20 22:20:11 2024 2024-08-20T22:20:11.6673328Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:20:11.6674173Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:20:11.6674978Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:20:11.6675789Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:20:11.6677831Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:20:11.6678606Z | | | MIG M. | 2024-08-20T22:20:11.6679151Z |=========================================+========================+======================| 2024-08-20T22:20:11.6801839Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:20:11.6803783Z | 0% 30C P0 59W / 300W | 0MiB / 23028MiB | 5% Default | 2024-08-20T22:20:11.6805230Z | | | N/A | 2024-08-20T22:20:11.6807102Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:20:11.6808476Z 2024-08-20T22:20:11.6809151Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:20:11.6809786Z | Processes: | 2024-08-20T22:20:11.6810498Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:20:11.6811203Z | ID ID Usage | 2024-08-20T22:20:11.6811764Z |=========================================================================================| 2024-08-20T22:20:11.6812415Z | No running processes found | 2024-08-20T22:20:11.6813181Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:20:12.2778584Z Command completed after 1 attempt(s). 2024-08-20T22:20:12.2860690Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:20:12.2861483Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:20:12.2862172Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-20T22:20:12.2862830Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:20:12.2877296Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:12.2877818Z env: 2024-08-20T22:20:12.2878100Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:12.2878629Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:12.2879102Z ##[endgroup] 2024-08-20T22:20:12.5775844Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:20:12.9960183Z Collecting psutil==5.9.1 2024-08-20T22:20:13.0321241Z Downloading psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB) 2024-08-20T22:20:13.0830687Z Collecting nvidia-ml-py==11.525.84 2024-08-20T22:20:13.0870049Z Downloading nvidia_ml_py-11.525.84-py3-none-any.whl (34 kB) 2024-08-20T22:20:13.1701815Z Installing collected packages: psutil, nvidia-ml-py 2024-08-20T22:20:13.3299915Z Successfully installed nvidia-ml-py-11.525.84 psutil-5.9.1 2024-08-20T22:20:13.5262007Z Prepare all required actions 2024-08-20T22:20:13.5262521Z Getting action download info 2024-08-20T22:20:13.6360420Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-20T22:20:13.8274130Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-20T22:20:13.9663517Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-20T22:20:13.9664006Z with: 2024-08-20T22:20:13.9664337Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:20:13.9664793Z s3-bucket: gha-artifacts 2024-08-20T22:20:13.9665127Z env: 2024-08-20T22:20:13.9665395Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:13.9665834Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:13.9666491Z ##[endgroup] 2024-08-20T22:20:13.9700426Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:20:13.9700867Z with: 2024-08-20T22:20:13.9701241Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:20:13.9701706Z s3-bucket: gha-artifacts 2024-08-20T22:20:13.9702052Z region: us-east-1 2024-08-20T22:20:13.9702354Z env: 2024-08-20T22:20:13.9702627Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:13.9703067Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:13.9703558Z ##[endgroup] 2024-08-20T22:20:14.4517699Z (node:52492) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:20:14.4518413Z 2024-08-20T22:20:14.4518673Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:20:14.4519438Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:20:14.4520350Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:20:14.5182976Z Found 1 objects with prefix pytorch/pytorch/10479310725/linux-focal-cuda12.1-py3.10-gcc9-sm86/ 2024-08-20T22:20:14.5184102Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:20:25.0844044Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:20:25.0851502Z Artifact download has finished successfully 2024-08-20T22:20:25.1214516Z ##[group]Run unzip -o artifacts.zip 2024-08-20T22:20:25.1214969Z unzip -o artifacts.zip 2024-08-20T22:20:25.1223908Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:25.1224406Z env: 2024-08-20T22:20:25.1224686Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:25.1225147Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:25.1225641Z ##[endgroup] 2024-08-20T22:20:25.1301286Z Archive: artifacts.zip 2024-08-20T22:20:25.1302358Z creating: dist/ 2024-08-20T22:20:27.2663734Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:20:27.2665379Z creating: build/custom_test_artifacts/ 2024-08-20T22:20:27.2666578Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-20T22:20:27.2668025Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-20T22:20:27.2669693Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:20:27.2673658Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:20:27.2674776Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-20T22:20:27.2675767Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:20:27.2676794Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:20:27.2677942Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:20:27.2679280Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:20:27.2680893Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:20:27.2681948Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:20:27.2683183Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:20:27.2684703Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:20:27.2686027Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:20:27.2687522Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:20:27.2688789Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:20:27.2690535Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:20:27.2691951Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:20:27.2693025Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:20:27.2694061Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:20:27.2732728Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:20:27.2772913Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:20:27.2774454Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:20:27.2819724Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:20:27.2821467Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:20:27.2823183Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:20:27.2824883Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:20:27.2826601Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:20:27.2828206Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:20:27.2829852Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:20:27.2831493Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:20:27.2833026Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:20:27.2834486Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:20:27.2835733Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:20:27.2836966Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:20:27.2838201Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:20:27.2839407Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:20:27.2840639Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:20:27.2894824Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:20:27.2954963Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:20:27.2956361Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:20:27.2957574Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:20:27.2958631Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:20:27.2959753Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-20T22:20:27.2960916Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-20T22:20:27.2962347Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-20T22:20:27.2963716Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-20T22:20:27.2965137Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-20T22:20:27.2966265Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-20T22:20:27.2967365Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-20T22:20:27.2968456Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-20T22:20:27.2969557Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-20T22:20:27.2970643Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-20T22:20:27.2971730Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-20T22:20:27.2987048Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-20T22:20:27.3131494Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-20T22:20:27.3134043Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-20T22:20:27.3135349Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-20T22:20:27.3136802Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-20T22:20:27.3138199Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-20T22:20:27.3139316Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-20T22:20:27.3140485Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-20T22:20:27.3141643Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-20T22:20:27.3142801Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-20T22:20:27.3143958Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-20T22:20:27.3145106Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-20T22:20:27.3159768Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-20T22:20:27.3242622Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-20T22:20:27.3244292Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:20:27.3245536Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:20:27.3247049Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-20T22:20:27.3248232Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-20T22:20:27.3249149Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-20T22:20:27.3250053Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-20T22:20:27.3251376Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-20T22:20:27.3252372Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-20T22:20:27.3253327Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-20T22:20:27.3373219Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-20T22:20:27.3437713Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-20T22:20:27.3439215Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-20T22:20:27.3440605Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-20T22:20:27.3442230Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:20:27.3445456Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:20:27.3446673Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-20T22:20:27.3447632Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:20:27.3448622Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:20:27.3449606Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:20:27.3450718Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:20:27.3452335Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:20:27.3453419Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:20:27.3454422Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:20:27.3456076Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:20:27.3457748Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:20:27.3459656Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:20:27.3460985Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:20:27.3462490Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:20:27.3463946Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:20:27.3465112Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:20:27.3466128Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:20:27.3505927Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:20:27.3545500Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:20:27.3547187Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:20:27.3592558Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:20:27.3594312Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:20:27.3596263Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:20:27.3598025Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:20:27.3599693Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:20:27.3601345Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:20:27.3602945Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:20:27.3604978Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:20:27.3606495Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:20:27.3607821Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:20:27.3609062Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:20:27.3610288Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:20:27.3611516Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:20:27.3612711Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:20:27.3613927Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:20:27.3668316Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:20:27.3728943Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:20:27.3730303Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:20:27.3731535Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:20:27.3732585Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:20:27.3733691Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-20T22:20:27.3734815Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-20T22:20:27.3736063Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-20T22:20:27.3737513Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-20T22:20:27.3738860Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-20T22:20:27.3740112Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-20T22:20:27.3741245Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-20T22:20:27.3742379Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-20T22:20:27.3743520Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-20T22:20:27.3744651Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-20T22:20:27.3746016Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-20T22:20:27.3761469Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-20T22:20:27.3825855Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-20T22:20:27.3827351Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:20:27.3828602Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:20:27.3829726Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-20T22:20:27.3830815Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-20T22:20:27.3831734Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-20T22:20:27.3832647Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-20T22:20:27.3833857Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-20T22:20:27.3834880Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-20T22:20:27.3835852Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-20T22:20:27.3886140Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-20T22:20:27.3887021Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-20T22:20:27.3887803Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-20T22:20:27.3888690Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:20:27.3895197Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:20:27.3896352Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-20T22:20:27.3897431Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:20:27.3898512Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:20:27.3899726Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:20:27.3901066Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:20:27.3902371Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:20:27.3903451Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:20:27.3904721Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:20:27.3905995Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:20:27.3907403Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:20:27.3908850Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:20:27.3910185Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:20:27.3911609Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:20:27.3913100Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:20:27.3914284Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:20:27.3915356Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:20:27.3956439Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:20:27.3995906Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:20:27.3997615Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:20:27.4042646Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:20:27.4045340Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:20:27.4049397Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:20:27.4053293Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:20:27.4055220Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:20:27.4056847Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:20:27.4058383Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:20:27.4059832Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:20:27.4061241Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:20:27.4062589Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:20:27.4063965Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:20:27.4065280Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:20:27.4066583Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:20:27.4067839Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:20:27.4069134Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:20:27.4119134Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:20:27.4178333Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:20:27.4179804Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:20:27.4181012Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:20:27.4182155Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:20:27.4183376Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-20T22:20:27.4184585Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-20T22:20:27.4185925Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-20T22:20:27.4187392Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-20T22:20:27.4188898Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-20T22:20:27.4190127Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-20T22:20:27.4191340Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-20T22:20:27.4192718Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-20T22:20:27.4194158Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-20T22:20:27.4195348Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-20T22:20:27.4196532Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-20T22:20:27.4197790Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-20T22:20:27.4322036Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-20T22:20:27.4324132Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-20T22:20:27.4325799Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-20T22:20:27.4327378Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-20T22:20:27.4328749Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-20T22:20:27.4329956Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-20T22:20:27.4331205Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-20T22:20:27.4332460Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-20T22:20:27.4333715Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-20T22:20:27.4334983Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-20T22:20:27.4336224Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-20T22:20:27.4350076Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-20T22:20:27.4407086Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-20T22:20:27.4410313Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:20:27.4412667Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:20:27.4414568Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-20T22:20:27.4415686Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-20T22:20:27.4416660Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-20T22:20:27.4417612Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-20T22:20:27.4418522Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-20T22:20:27.4419358Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-20T22:20:27.4420224Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-20T22:20:27.4520854Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-20T22:20:27.4563041Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-20T22:20:27.4563687Z creating: build/lib/ 2024-08-20T22:20:27.4653300Z inflating: build/lib/libprotobuf-lite.a 2024-08-20T22:20:27.5109472Z inflating: build/lib/libprotobuf.a 2024-08-20T22:20:27.5118502Z inflating: build/lib/libpthreadpool.a 2024-08-20T22:20:27.5127130Z inflating: build/lib/libcpuinfo.a 2024-08-20T22:20:27.5135300Z inflating: build/lib/libcpuinfo_internals.a 2024-08-20T22:20:27.5136129Z inflating: build/lib/libclog.a 2024-08-20T22:20:27.5154382Z inflating: build/lib/libnnpack.a 2024-08-20T22:20:27.5157042Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-20T22:20:27.5221840Z inflating: build/lib/libgtest.a 2024-08-20T22:20:27.5296525Z inflating: build/lib/libbenchmark.a 2024-08-20T22:20:27.5360364Z inflating: build/lib/libasmjit.a 2024-08-20T22:20:27.5368204Z inflating: build/lib/libittnotify.a 2024-08-20T22:20:27.5395993Z inflating: build/lib/libtensorpipe_uv.a 2024-08-20T22:20:27.5524616Z inflating: build/lib/libgloo.a 2024-08-20T22:20:27.5545396Z inflating: build/lib/libfmt.a 2024-08-20T22:20:27.5640278Z inflating: build/lib/libc10.so 2024-08-20T22:20:27.5642140Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-20T22:20:27.5643328Z inflating: build/lib/libtorch_global_deps.so 2024-08-20T22:20:27.5663629Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-20T22:20:27.6166173Z inflating: build/lib/libprotoc.a 2024-08-20T22:20:27.6184378Z inflating: build/lib/libgmock.a 2024-08-20T22:20:27.6184963Z inflating: build/lib/libgtest_main.a 2024-08-20T22:20:27.6185812Z inflating: build/lib/libbenchmark_main.a 2024-08-20T22:20:27.6577094Z inflating: build/lib/libgloo_cuda.a 2024-08-20T22:20:28.6752708Z inflating: build/lib/libdnnl.a 2024-08-20T22:20:28.7325256Z inflating: build/lib/libtensorpipe.a 2024-08-20T22:20:28.7382985Z inflating: build/lib/libc10_cuda.so 2024-08-20T22:20:28.7383544Z inflating: build/lib/libgmock_main.a 2024-08-20T22:20:28.8643544Z inflating: build/lib/libfbgemm.a 2024-08-20T22:20:28.8899440Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-20T22:20:28.9404833Z inflating: build/lib/libkineto.a 2024-08-20T22:20:28.9588732Z inflating: build/lib/libXNNPACK.a 2024-08-20T22:20:28.9631140Z inflating: build/lib/libonnx_proto.a 2024-08-20T22:20:29.0325683Z inflating: build/lib/libonnx.a 2024-08-20T22:20:31.4819673Z inflating: build/lib/libtorch_cpu.so 2024-08-20T22:20:31.4824577Z inflating: build/lib/libunbox_lib.a 2024-08-20T22:20:31.4828221Z inflating: build/lib/libshm.so 2024-08-20T22:20:33.5451287Z inflating: build/lib/libtorch_cuda.so 2024-08-20T22:20:33.5452371Z inflating: build/lib/libtorch.so 2024-08-20T22:20:33.5455783Z inflating: build/lib/libc10d_cuda_test.so 2024-08-20T22:20:34.3633670Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-20T22:20:34.5627197Z inflating: build/lib/libtorch_python.so 2024-08-20T22:20:34.5699991Z inflating: build/lib/libtorchbind_test.so 2024-08-20T22:20:34.5722547Z inflating: build/lib/libjitbackend_test.so 2024-08-20T22:20:34.5749063Z inflating: build/lib/libaoti_custom_ops.so 2024-08-20T22:20:34.5775251Z inflating: build/lib/libbackend_with_compiler.so 2024-08-20T22:20:34.5810184Z inflating: build/lib/libnnapi_backend.so 2024-08-20T22:20:34.5810732Z creating: build/bin/ 2024-08-20T22:20:34.5861426Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-20T22:20:34.5913059Z inflating: build/bin/c10_DeviceGuard_test 2024-08-20T22:20:34.5964619Z inflating: build/bin/c10_Device_test 2024-08-20T22:20:34.6024014Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-20T22:20:34.6078740Z inflating: build/bin/c10_Scalar_test 2024-08-20T22:20:34.6127428Z inflating: build/bin/c10_StreamGuard_test 2024-08-20T22:20:34.6179278Z inflating: build/bin/c10_SymInt_test 2024-08-20T22:20:34.6234213Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-20T22:20:34.6290549Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-20T22:20:34.6347241Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-20T22:20:34.6418821Z inflating: build/bin/c10_cow_test 2024-08-20T22:20:34.6472471Z inflating: build/bin/c10_Bitset_test 2024-08-20T22:20:34.6521751Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-20T22:20:34.6572975Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-20T22:20:34.6623088Z inflating: build/bin/c10_Half_test 2024-08-20T22:20:34.6680385Z inflating: build/bin/c10_LeftRight_test 2024-08-20T22:20:34.6736701Z inflating: build/bin/c10_Metaprogramming_test 2024-08-20T22:20:34.6787430Z inflating: build/bin/c10_Synchronized_test 2024-08-20T22:20:34.6843196Z inflating: build/bin/c10_ThreadLocal_test 2024-08-20T22:20:34.6895521Z inflating: build/bin/c10_TypeIndex_test 2024-08-20T22:20:34.6947160Z inflating: build/bin/c10_TypeList_test 2024-08-20T22:20:34.6996711Z inflating: build/bin/c10_TypeTraits_test 2024-08-20T22:20:34.7050868Z inflating: build/bin/c10_accumulate_test 2024-08-20T22:20:34.7108885Z inflating: build/bin/c10_bfloat16_test 2024-08-20T22:20:34.7159379Z inflating: build/bin/c10_bit_cast_test 2024-08-20T22:20:34.7216167Z inflating: build/bin/c10_complex_math_test 2024-08-20T22:20:34.7272617Z inflating: build/bin/c10_complex_test 2024-08-20T22:20:34.7326547Z inflating: build/bin/c10_exception_test 2024-08-20T22:20:34.7377679Z inflating: build/bin/c10_flags_test 2024-08-20T22:20:34.7427938Z inflating: build/bin/c10_generic_math_test 2024-08-20T22:20:34.7593820Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-20T22:20:34.7645182Z inflating: build/bin/c10_irange_test 2024-08-20T22:20:34.7699323Z inflating: build/bin/c10_lazy_test 2024-08-20T22:20:34.7757335Z inflating: build/bin/c10_logging_test 2024-08-20T22:20:34.7820278Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-20T22:20:34.7896470Z inflating: build/bin/c10_optional_test 2024-08-20T22:20:34.8049065Z inflating: build/bin/c10_small_vector_test 2024-08-20T22:20:34.8104961Z inflating: build/bin/c10_registry_test 2024-08-20T22:20:34.8157340Z inflating: build/bin/c10_ssize_test 2024-08-20T22:20:34.8210001Z inflating: build/bin/c10_string_util_test 2024-08-20T22:20:34.8260971Z inflating: build/bin/c10_tempfile_test 2024-08-20T22:20:34.8320022Z inflating: build/bin/c10_string_view_test 2024-08-20T22:20:34.8369749Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-20T22:20:34.8425655Z inflating: build/bin/c10_typeid_test 2024-08-20T22:20:34.8876048Z inflating: build/bin/protoc-3.13.0.0 2024-08-20T22:20:34.9323171Z inflating: build/bin/protoc 2024-08-20T22:20:34.9376604Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-20T22:20:34.9430651Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-20T22:20:34.9483389Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-20T22:20:34.9536534Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-20T22:20:34.9590579Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-20T22:20:34.9644978Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-20T22:20:34.9694573Z inflating: build/bin/c10_cuda_CUDATest 2024-08-20T22:20:34.9748488Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-20T22:20:35.0081627Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-20T22:20:35.0428143Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-20T22:20:35.0789058Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-20T22:20:35.0842042Z inflating: build/bin/BackoffTest 2024-08-20T22:20:35.0895511Z inflating: build/bin/FileStoreTest 2024-08-20T22:20:35.0951322Z inflating: build/bin/TCPStoreTest 2024-08-20T22:20:35.1005023Z inflating: build/bin/HashStoreTest 2024-08-20T22:20:35.1018674Z inflating: build/bin/ProcessGroupMPITest 2024-08-20T22:20:35.1072881Z inflating: build/bin/test_edge_op_registration 2024-08-20T22:20:35.1077224Z inflating: build/bin/torch_shm_manager 2024-08-20T22:20:35.1080132Z inflating: build/bin/example_allreduce 2024-08-20T22:20:35.1135554Z inflating: build/bin/test_dist_autograd 2024-08-20T22:20:35.1205401Z inflating: build/bin/test_cpp_rpc 2024-08-20T22:20:35.1207554Z inflating: build/bin/parallel_benchmark 2024-08-20T22:20:35.1275407Z inflating: build/bin/test_mobile_nnc 2024-08-20T22:20:35.1284120Z inflating: build/bin/aot_model_compiler_test 2024-08-20T22:20:35.1632995Z inflating: build/bin/test_lazy 2024-08-20T22:20:35.2817803Z inflating: build/bin/test_api 2024-08-20T22:20:35.2891970Z inflating: build/bin/Dict_test 2024-08-20T22:20:35.2944378Z inflating: build/bin/Dimname_test 2024-08-20T22:20:35.3010142Z inflating: build/bin/MaybeOwned_test 2024-08-20T22:20:35.3068011Z inflating: build/bin/NamedTensor_test 2024-08-20T22:20:35.3127470Z inflating: build/bin/apply_utils_test 2024-08-20T22:20:35.3186814Z inflating: build/bin/atest 2024-08-20T22:20:35.3250839Z inflating: build/bin/basic 2024-08-20T22:20:35.3305265Z inflating: build/bin/broadcast_test 2024-08-20T22:20:35.3356399Z inflating: build/bin/cpu_allocator_test 2024-08-20T22:20:35.3414857Z inflating: build/bin/cpu_generator_test 2024-08-20T22:20:35.3469132Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-20T22:20:35.3562163Z inflating: build/bin/cpu_rng_test 2024-08-20T22:20:35.3613095Z inflating: build/bin/dispatch_key_set_test 2024-08-20T22:20:35.3663228Z inflating: build/bin/dlconvertor_test 2024-08-20T22:20:35.3721745Z inflating: build/bin/extension_backend_test 2024-08-20T22:20:35.3777503Z inflating: build/bin/half_test 2024-08-20T22:20:35.3874267Z inflating: build/bin/ivalue_test 2024-08-20T22:20:35.3923840Z inflating: build/bin/lazy_tensor_test 2024-08-20T22:20:35.3978823Z inflating: build/bin/math_kernel_test 2024-08-20T22:20:35.4034585Z inflating: build/bin/memory_format_test 2024-08-20T22:20:35.4090101Z inflating: build/bin/memory_overlapping_test 2024-08-20T22:20:35.4142667Z inflating: build/bin/mobile_memory_cleanup 2024-08-20T22:20:35.4199229Z inflating: build/bin/native_test 2024-08-20T22:20:35.4250335Z inflating: build/bin/operator_name_test 2024-08-20T22:20:35.4302375Z inflating: build/bin/operators_test 2024-08-20T22:20:35.4354385Z inflating: build/bin/packedtensoraccessor_test 2024-08-20T22:20:35.4421823Z inflating: build/bin/pow_test 2024-08-20T22:20:35.4479405Z inflating: build/bin/quantized_test 2024-08-20T22:20:35.4529667Z inflating: build/bin/reduce_ops_test 2024-08-20T22:20:35.4581302Z inflating: build/bin/reportMemoryUsage_test 2024-08-20T22:20:35.4637974Z inflating: build/bin/scalar_tensor_test 2024-08-20T22:20:35.4697090Z inflating: build/bin/scalar_test 2024-08-20T22:20:35.4749032Z inflating: build/bin/StorageUtils_test 2024-08-20T22:20:35.4801939Z inflating: build/bin/stride_properties_test 2024-08-20T22:20:35.4881159Z inflating: build/bin/tensor_iterator_test 2024-08-20T22:20:35.4935465Z inflating: build/bin/test_parallel 2024-08-20T22:20:35.4938482Z inflating: build/bin/thread_init_test 2024-08-20T22:20:35.4994770Z inflating: build/bin/type_ptr_test 2024-08-20T22:20:35.5055045Z inflating: build/bin/type_test 2024-08-20T22:20:35.5110101Z inflating: build/bin/undefined_tensor_test 2024-08-20T22:20:35.5111809Z inflating: build/bin/verify_api_visibility 2024-08-20T22:20:35.5181614Z inflating: build/bin/legacy_vmap_test 2024-08-20T22:20:35.5233509Z inflating: build/bin/weakref_test 2024-08-20T22:20:35.5285950Z inflating: build/bin/wrapdim_test 2024-08-20T22:20:35.5337359Z inflating: build/bin/xla_tensor_test 2024-08-20T22:20:35.5398074Z inflating: build/bin/IListRef_test 2024-08-20T22:20:35.5505131Z inflating: build/bin/List_test 2024-08-20T22:20:35.5572851Z inflating: build/bin/KernelFunction_test 2024-08-20T22:20:35.5695207Z inflating: build/bin/kernel_function_legacy_test 2024-08-20T22:20:35.5792376Z inflating: build/bin/kernel_function_test 2024-08-20T22:20:35.5922295Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-20T22:20:35.6025553Z inflating: build/bin/kernel_lambda_test 2024-08-20T22:20:35.6088210Z inflating: build/bin/kernel_stackbased_test 2024-08-20T22:20:35.6185953Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-20T22:20:35.6236786Z inflating: build/bin/CppSignature_test 2024-08-20T22:20:35.6292872Z inflating: build/bin/backend_fallback_test 2024-08-20T22:20:35.6342052Z inflating: build/bin/op_allowlist_test 2024-08-20T22:20:35.6647279Z inflating: build/bin/op_registration_test 2024-08-20T22:20:35.6711846Z inflating: build/bin/inline_container_test 2024-08-20T22:20:35.6764733Z inflating: build/bin/cuda_apply_test 2024-08-20T22:20:35.6816807Z inflating: build/bin/cuda_allocator_test 2024-08-20T22:20:35.6876198Z inflating: build/bin/cuda_atomic_ops_test 2024-08-20T22:20:35.6931179Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-20T22:20:35.7002303Z inflating: build/bin/cuda_complex_math_test 2024-08-20T22:20:35.7061418Z inflating: build/bin/cuda_complex_test 2024-08-20T22:20:35.7110931Z inflating: build/bin/cuda_device_test 2024-08-20T22:20:35.7169255Z inflating: build/bin/cuda_cub_test 2024-08-20T22:20:35.7219811Z inflating: build/bin/cuda_dlconvertor_test 2024-08-20T22:20:35.7285697Z inflating: build/bin/cuda_distributions_test 2024-08-20T22:20:35.7342075Z inflating: build/bin/cuda_generator_test 2024-08-20T22:20:35.7392657Z inflating: build/bin/cuda_half_test 2024-08-20T22:20:35.7445444Z inflating: build/bin/cuda_integer_divider_test 2024-08-20T22:20:35.7496137Z inflating: build/bin/cuda_optional_test 2024-08-20T22:20:35.7548469Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-20T22:20:35.7600720Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-20T22:20:35.7651745Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-20T22:20:35.7712268Z inflating: build/bin/cuda_stream_test 2024-08-20T22:20:35.7762220Z inflating: build/bin/cuda_cudnn_test 2024-08-20T22:20:35.7814337Z inflating: build/bin/cuda_vectorized_test 2024-08-20T22:20:35.7828906Z inflating: build/bin/tutorial_tensorexpr 2024-08-20T22:20:35.7894950Z inflating: build/bin/ProcessGroupGlooTest 2024-08-20T22:20:35.7952776Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-20T22:20:35.8016823Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-20T22:20:35.8080099Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-20T22:20:35.8911002Z inflating: build/bin/test_tensorexpr 2024-08-20T22:20:35.9493922Z inflating: build/bin/test_jit 2024-08-20T22:20:35.9494854Z creating: .additional_ci_files/ 2024-08-20T22:20:35.9554724Z inflating: .additional_ci_files/test-times.json 2024-08-20T22:20:35.9795170Z inflating: .additional_ci_files/test-class-times.json 2024-08-20T22:20:35.9830283Z ##[group]Run rm artifacts.zip 2024-08-20T22:20:35.9830674Z rm artifacts.zip 2024-08-20T22:20:35.9841677Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:35.9842183Z env: 2024-08-20T22:20:35.9842453Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:35.9842908Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:35.9843396Z ##[endgroup] 2024-08-20T22:20:36.0967868Z ##[group]Run df -H 2024-08-20T22:20:36.0968187Z df -H 2024-08-20T22:20:36.0977035Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:36.0977532Z env: 2024-08-20T22:20:36.0977798Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.0978258Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.0978752Z ##[endgroup] 2024-08-20T22:20:36.1030584Z Filesystem Size Used Avail Use% Mounted on 2024-08-20T22:20:36.1031239Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-20T22:20:36.1033022Z tmpfs 34G 0 34G 0% /dev/shm 2024-08-20T22:20:36.1033647Z tmpfs 14G 553k 14G 1% /run 2024-08-20T22:20:36.1034293Z /dev/nvme0n1p1 161G 53G 109G 33% / 2024-08-20T22:20:36.1035142Z tmpfs 34G 4.1k 34G 1% /tmp 2024-08-20T22:20:36.1035827Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-20T22:20:36.1036531Z tmpfs 6.7G 0 6.7G 0% /run/user/0 2024-08-20T22:20:36.1076934Z Prepare all required actions 2024-08-20T22:20:36.1077421Z Getting action download info 2024-08-20T22:20:36.2295733Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-20T22:20:36.2296199Z with: 2024-08-20T22:20:36.2296458Z env: 2024-08-20T22:20:36.2296741Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.2297195Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.2297689Z ##[endgroup] 2024-08-20T22:20:36.2330783Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:20:36.2331224Z with: 2024-08-20T22:20:36.2331663Z name: td_results 2024-08-20T22:20:36.2331979Z s3-bucket: gha-artifacts 2024-08-20T22:20:36.2332324Z region: us-east-1 2024-08-20T22:20:36.2332618Z env: 2024-08-20T22:20:36.2332880Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.2333328Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.2333803Z ##[endgroup] 2024-08-20T22:20:36.7072654Z (node:52513) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:20:36.7073356Z 2024-08-20T22:20:36.7073616Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:20:36.7074357Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:20:36.7075218Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:20:36.7655606Z Found 0 objects with prefix pytorch/pytorch/10479310725/td_results/ 2024-08-20T22:20:36.7661808Z Artifact download has finished successfully 2024-08-20T22:20:36.7984287Z ##[group]Run mkdir -p .additional_ci_files 2024-08-20T22:20:36.7984776Z mkdir -p .additional_ci_files 2024-08-20T22:20:36.7985334Z mv td_results.json .additional_ci_files/td_results.json 2024-08-20T22:20:36.7994613Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:36.7995128Z env: 2024-08-20T22:20:36.7995414Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.7995901Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.7996416Z ##[endgroup] 2024-08-20T22:20:36.8055029Z mv: cannot stat 'td_results.json': No such file or directory 2024-08-20T22:20:36.8068167Z ##[error]Process completed with exit code 1. 2024-08-20T22:20:36.8102615Z ##[group]Run .github/scripts/parse_ref.py 2024-08-20T22:20:36.8103099Z .github/scripts/parse_ref.py 2024-08-20T22:20:36.8111874Z shell: /usr/bin/bash -e {0} 2024-08-20T22:20:36.8112216Z env: 2024-08-20T22:20:36.8112493Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.8112947Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.8113454Z ##[endgroup] 2024-08-20T22:20:36.8397918Z Prepare all required actions 2024-08-20T22:20:36.8468057Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-20T22:20:36.8468514Z with: 2024-08-20T22:20:36.8469068Z github-token: *** 2024-08-20T22:20:36.8469375Z env: 2024-08-20T22:20:36.8469639Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.8470087Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.8470572Z ##[endgroup] 2024-08-20T22:20:36.8487507Z ##[group]Run set -eux 2024-08-20T22:20:36.8487847Z set -eux 2024-08-20T22:20:36.8488444Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-20T22:20:36.8497213Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:36.8497719Z env: 2024-08-20T22:20:36.8498020Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:36.8498491Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:36.8499165Z GITHUB_TOKEN: *** 2024-08-20T22:20:36.8499479Z ##[endgroup] 2024-08-20T22:20:36.8528537Z + python3 .github/scripts/get_workflow_job_id.py 10479310725 i-0a41c091cbed476e8 2024-08-20T22:20:38.5696810Z setting job-id=29026530815 2024-08-20T22:20:38.5698632Z setting job-name=cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:20:38.5918667Z Prepare all required actions 2024-08-20T22:20:38.5919123Z Getting action download info 2024-08-20T22:20:38.6974740Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-20T22:20:38.6975201Z with: 2024-08-20T22:20:38.6975656Z github-token: *** 2024-08-20T22:20:38.6984695Z test-matrix: {"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:20:38.6994782Z job-name: cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:20:38.6995558Z env: 2024-08-20T22:20:38.6995843Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:38.6996306Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:38.6996785Z ##[endgroup] 2024-08-20T22:20:38.7039441Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:20:38.7039993Z with: 2024-08-20T22:20:38.7040262Z shell: bash 2024-08-20T22:20:38.7040564Z timeout_minutes: 10 2024-08-20T22:20:38.7040885Z max_attempts: 5 2024-08-20T22:20:38.7041202Z retry_wait_seconds: 30 2024-08-20T22:20:38.7042332Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:20:38.7043514Z polling_interval_seconds: 1 2024-08-20T22:20:38.7043891Z warning_on_retry: true 2024-08-20T22:20:38.7044309Z continue_on_error: false 2024-08-20T22:20:38.7044648Z env: 2024-08-20T22:20:38.7044923Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:38.7045368Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:38.7046263Z GITHUB_TOKEN: *** 2024-08-20T22:20:38.7046581Z ##[endgroup] 2024-08-20T22:20:38.7799852Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:20:39.0187794Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:20:39.1643130Z Collecting requests==2.27.1 2024-08-20T22:20:39.2074156Z Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB) 2024-08-20T22:20:39.4017524Z Collecting pyyaml==6.0.1 2024-08-20T22:20:39.4077212Z Downloading PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (738 kB) 2024-08-20T22:20:39.4651990Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-20T22:20:39.4661585Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-20T22:20:39.7646340Z Collecting charset-normalizer~=2.0.0 2024-08-20T22:20:39.7705138Z Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 2024-08-20T22:20:39.8214759Z Collecting certifi>=2017.4.17 2024-08-20T22:20:39.8272121Z Downloading certifi-2024.7.4-py3-none-any.whl (162 kB) 2024-08-20T22:20:39.9119089Z Installing collected packages: charset-normalizer, certifi, requests, pyyaml 2024-08-20T22:20:40.0320258Z Successfully installed certifi-2024.7.4 charset-normalizer-2.0.12 pyyaml-6.0.1 requests-2.27.1 2024-08-20T22:20:40.7596302Z Command completed after 1 attempt(s). 2024-08-20T22:20:40.7659139Z ##[group]Run set -x 2024-08-20T22:20:40.7659473Z set -x 2024-08-20T22:20:40.7659769Z  2024-08-20T22:20:40.7660321Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:20:40.7660999Z # in runner workspace 2024-08-20T22:20:40.7661536Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-20T22:20:40.7670670Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:40.7671188Z env: 2024-08-20T22:20:40.7671472Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:40.7671939Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:40.7672437Z ##[endgroup] 2024-08-20T22:20:40.7702588Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-20T22:20:40.7939100Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:20:40.7939666Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:20:40.7940135Z echo "Job name: ${JOB_NAME}" 2024-08-20T22:20:40.7940548Z  2024-08-20T22:20:40.7941112Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:20:40.7941798Z # in runner workspace 2024-08-20T22:20:40.7942373Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-20T22:20:40.7943025Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-20T22:20:40.7943508Z  --job-name "${JOB_NAME}" \ 2024-08-20T22:20:40.7953151Z  --test-matrix "{"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-08-20T22:20:40.7962760Z  --selected-test-configs "" \ 2024-08-20T22:20:40.7963243Z  --pr-number "${PR_NUMBER}" \ 2024-08-20T22:20:40.7963673Z  --tag "${TAG}" \ 2024-08-20T22:20:40.7964140Z  --event-name "${EVENT_NAME}" \ 2024-08-20T22:20:40.7964587Z  --schedule "${SCHEDULE}" \ 2024-08-20T22:20:40.7965016Z  --branch "${HEAD_BRANCH}" 2024-08-20T22:20:40.7974007Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:40.7974506Z env: 2024-08-20T22:20:40.7974787Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:40.7975244Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:40.7975922Z GITHUB_TOKEN: *** 2024-08-20T22:20:40.7976625Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:20:40.7977396Z PR_NUMBER: 2024-08-20T22:20:40.7977713Z TAG: ciflow/inductor/133712 2024-08-20T22:20:40.7978087Z EVENT_NAME: push 2024-08-20T22:20:40.7978400Z SCHEDULE: 2024-08-20T22:20:40.7978682Z HEAD_BRANCH: 2024-08-20T22:20:40.7978995Z ##[endgroup] 2024-08-20T22:20:40.8008270Z Workflow: inductor 2024-08-20T22:20:40.8009281Z Job name: cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:20:41.0374800Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-20T22:20:41.1785329Z ##[group]Run echo "Filtered matrix:" 2024-08-20T22:20:41.1785786Z echo "Filtered matrix:" 2024-08-20T22:20:41.1795121Z echo "{"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]}" 2024-08-20T22:20:41.1804784Z  2024-08-20T22:20:41.1805053Z echo 2024-08-20T22:20:41.1805431Z echo "Is the current job unstable? False" 2024-08-20T22:20:41.1805881Z  2024-08-20T22:20:41.1806151Z echo 2024-08-20T22:20:41.1806509Z echo "Is keep-going label set? False" 2024-08-20T22:20:41.1806946Z  2024-08-20T22:20:41.1807216Z echo 2024-08-20T22:20:41.1807528Z echo "Renabled issues? " 2024-08-20T22:20:41.1816523Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:41.1817016Z env: 2024-08-20T22:20:41.1817299Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:41.1817770Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:41.1818298Z ##[endgroup] 2024-08-20T22:20:41.1849596Z Filtered matrix: 2024-08-20T22:20:41.1860222Z {include: [{config: inductor, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_distributed, shard: 1, num_shards: 1, runner: amz2023.linux.g5.12xlarge.nvidia.gpu}, {config: inductor_huggingface, shard: 1, num_shards: 1, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_torchbench, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_torchbench, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_huggingface, shard: 1, num_shards: 1, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_timm, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_timm, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_torchbench, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_torchbench, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_huggingface, shard: 1, num_shards: 1, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_timm, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_timm, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_torchbench, shard: 1, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_torchbench, shard: 2, num_shards: 2, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: inductor_cpp_wrapper_abi_compatible, shard: 1, num_shards: 1, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}]} 2024-08-20T22:20:41.1869299Z 2024-08-20T22:20:41.1869450Z Is the current job unstable? False 2024-08-20T22:20:41.1869741Z 2024-08-20T22:20:41.1870102Z Is keep-going label set? False 2024-08-20T22:20:41.1870358Z 2024-08-20T22:20:41.1870487Z Renabled issues? 2024-08-20T22:20:41.1912926Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:20:41.1913642Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:20:41.1921871Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:20:41.1922371Z env: 2024-08-20T22:20:41.1922657Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:41.1923124Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:41.1923620Z JOB_TIMEOUT: 240 2024-08-20T22:20:41.1924020Z ##[endgroup] 2024-08-20T22:20:41.2041247Z ##[group]Run set -x 2024-08-20T22:20:41.2041645Z set -x 2024-08-20T22:20:41.2041938Z  2024-08-20T22:20:41.2042291Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-20T22:20:41.2042831Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-20T22:20:41.2043390Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-20T22:20:41.2043897Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-20T22:20:41.2044563Z else 2024-08-20T22:20:41.2044895Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:20:41.2045319Z fi 2024-08-20T22:20:41.2045920Z  2024-08-20T22:20:41.2046460Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-20T22:20:41.2047213Z # TODO: Stop building test binaries as part of the build phase 2024-08-20T22:20:41.2047898Z # Used for GPU_FLAG since that doesn't play nice 2024-08-20T22:20:41.2048485Z # shellcheck disable=SC2086,SC2090 2024-08-20T22:20:41.2048943Z container_name=$(docker run \ 2024-08-20T22:20:41.2049361Z  ${GPU_FLAG:-} \ 2024-08-20T22:20:41.2049732Z  -e BUILD_ENVIRONMENT \ 2024-08-20T22:20:41.2050140Z  -e PR_NUMBER \ 2024-08-20T22:20:41.2050497Z  -e GITHUB_ACTIONS \ 2024-08-20T22:20:41.2050891Z  -e GITHUB_REPOSITORY \ 2024-08-20T22:20:41.2051296Z  -e GITHUB_WORKFLOW \ 2024-08-20T22:20:41.2051686Z  -e GITHUB_JOB \ 2024-08-20T22:20:41.2052053Z  -e GITHUB_RUN_ID \ 2024-08-20T22:20:41.2052434Z  -e GITHUB_RUN_NUMBER \ 2024-08-20T22:20:41.2052837Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-20T22:20:41.2053242Z  -e JOB_ID \ 2024-08-20T22:20:41.2053580Z  -e JOB_NAME \ 2024-08-20T22:20:41.2053924Z  -e BASE_SHA \ 2024-08-20T22:20:41.2054317Z  -e BRANCH \ 2024-08-20T22:20:41.2054648Z  -e SHA1 \ 2024-08-20T22:20:41.2054995Z  -e AWS_DEFAULT_REGION \ 2024-08-20T22:20:41.2055399Z  -e IN_WHEEL_TEST \ 2024-08-20T22:20:41.2055777Z  -e SHARD_NUMBER \ 2024-08-20T22:20:41.2056143Z  -e TEST_CONFIG \ 2024-08-20T22:20:41.2056520Z  -e NUM_TEST_SHARDS \ 2024-08-20T22:20:41.2056919Z  -e REENABLED_ISSUES \ 2024-08-20T22:20:41.2057334Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-20T22:20:41.2057753Z  -e VERBOSE_TEST_LOGS \ 2024-08-20T22:20:41.2058153Z  -e TEST_SHOWLOCALS \ 2024-08-20T22:20:41.2058545Z  -e NO_TEST_TIMEOUT \ 2024-08-20T22:20:41.2058915Z  -e NO_TD \ 2024-08-20T22:20:41.2059251Z  -e TD_DISTRIBUTED \ 2024-08-20T22:20:41.2059633Z  -e PR_LABELS \ 2024-08-20T22:20:41.2060079Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-20T22:20:41.2060559Z  -e SCCACHE_BUCKET \ 2024-08-20T22:20:41.2060955Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-20T22:20:41.2061368Z  -e XLA_CUDA \ 2024-08-20T22:20:41.2061774Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-20T22:20:41.2062277Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-20T22:20:41.2062801Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-20T22:20:41.2063311Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-20T22:20:41.2063779Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-20T22:20:41.2064220Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-20T22:20:41.2064664Z  -e DASHBOARD_TAG \ 2024-08-20T22:20:41.2065143Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-20T22:20:41.2065699Z  --security-opt seccomp=unconfined \ 2024-08-20T22:20:41.2066156Z  --cap-add=SYS_PTRACE \ 2024-08-20T22:20:41.2066552Z  --ipc=host \ 2024-08-20T22:20:41.2066916Z  --shm-size="${SHM_SIZE}" \ 2024-08-20T22:20:41.2067312Z  --tty \ 2024-08-20T22:20:41.2067639Z  --detach \ 2024-08-20T22:20:41.2068193Z  --name="${container_name}" \ 2024-08-20T22:20:41.2068623Z  --user jenkins \ 2024-08-20T22:20:41.2069130Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-20T22:20:41.2069697Z  -w /var/lib/jenkins/workspace \ 2024-08-20T22:20:41.2070141Z  "${DOCKER_IMAGE}" 2024-08-20T22:20:41.2070501Z ) 2024-08-20T22:20:41.2070927Z # Propagate download.pytorch.org IP to container 2024-08-20T22:20:41.2071849Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-20T22:20:41.2072960Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-20T22:20:41.2073882Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-20T22:20:41.2083326Z shell: /usr/bin/bash -e {0} 2024-08-20T22:20:41.2083677Z env: 2024-08-20T22:20:41.2084049Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:20:41.2084523Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:20:41.2085125Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:20:41.2085614Z PR_NUMBER: 2024-08-20T22:20:41.2085933Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-20T22:20:41.2086339Z GITHUB_WORKFLOW: inductor 2024-08-20T22:20:41.2086686Z GITHUB_JOB: test 2024-08-20T22:20:41.2086987Z GITHUB_RUN_ID: 10479310725 2024-08-20T22:20:41.2087339Z GITHUB_RUN_NUMBER: 88082 2024-08-20T22:20:41.2087688Z GITHUB_RUN_ATTEMPT: 1 2024-08-20T22:20:41.2088014Z JOB_ID: 29026530815 2024-08-20T22:20:41.2088706Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:20:41.2089464Z BRANCH: 2024-08-20T22:20:41.2089798Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:20:41.2090310Z BASE_SHA: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:20:41.2090787Z TEST_CONFIG: aot_inductor_torchbench 2024-08-20T22:20:41.2091195Z SHARD_NUMBER: 2 2024-08-20T22:20:41.2091501Z NUM_TEST_SHARDS: 2 2024-08-20T22:20:41.2091816Z REENABLED_ISSUES: 2024-08-20T22:20:41.2092142Z CONTINUE_THROUGH_ERROR: False 2024-08-20T22:20:41.2092525Z VERBOSE_TEST_LOGS: False 2024-08-20T22:20:41.2092877Z TEST_SHOWLOCALS: False 2024-08-20T22:20:41.2093214Z NO_TEST_TIMEOUT: False 2024-08-20T22:20:41.2093543Z NO_TD: False 2024-08-20T22:20:41.2093834Z TD_DISTRIBUTED: False 2024-08-20T22:20:41.2094247Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-20T22:20:41.2094732Z SCCACHE_S3_KEY_PREFIX: inductor 2024-08-20T22:20:41.2095104Z SHM_SIZE: 2g 2024-08-20T22:20:41.2096196Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:20:41.2097352Z XLA_CUDA: 2024-08-20T22:20:41.2097821Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:20:41.2098431Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-20T22:20:41.2098866Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-20T22:20:41.2099276Z DASHBOARD_TAG: 2024-08-20T22:20:41.2099782Z HUGGING_FACE_HUB_TOKEN: *** 2024-08-20T22:20:41.2100154Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-20T22:20:41.2100529Z ##[endgroup] 2024-08-20T22:20:41.2134550Z + [[ aot_inductor_torchbench == \m\u\l\t\i\g\p\u ]] 2024-08-20T22:20:41.2135368Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]] 2024-08-20T22:20:41.2135884Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:20:41.2143422Z +++ nproc --ignore=2 2024-08-20T22:20:41.2177205Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479310725 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:20:52.6249004Z + container_name=d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T22:20:52.6250498Z + grep download.pytorch.org /etc/hosts 2024-08-20T22:20:52.6253161Z + docker exec -i d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-20T22:20:52.7612895Z + echo DOCKER_CONTAINER_ID=d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T22:20:52.7616566Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:20:52.7619509Z + docker exec -t d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-20T22:20:53.1580064Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-20T22:20:53.4839242Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-20T22:20:53.4842841Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-20T22:20:53.4845067Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-20T22:20:53.4847060Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-20T22:20:53.4849441Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-20T22:20:53.4854695Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.13.1) 2024-08-20T22:20:53.4870773Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-20T22:20:53.4884333Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-20T22:20:53.4901871Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.22.4) 2024-08-20T22:20:53.5288464Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-20T22:20:54.0956192Z Installing collected packages: torch 2024-08-20T22:21:03.8830540Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T22:21:03.8831971Z timm 1.0.9.dev0 requires torchvision, which is not installed. 2024-08-20T22:21:03.8835483Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-20T22:21:03.9634488Z + export TERM=vt100 2024-08-20T22:21:03.9634876Z + TERM=vt100 2024-08-20T22:21:03.9637743Z ++ dirname .ci/pytorch/test.sh 2024-08-20T22:21:03.9648290Z + source .ci/pytorch/common.sh 2024-08-20T22:21:03.9652758Z +++ dirname .ci/pytorch/common.sh 2024-08-20T22:21:03.9663531Z ++ source .ci/pytorch/common_utils.sh 2024-08-20T22:21:03.9665809Z +++ declare -f -t trap_add 2024-08-20T22:21:03.9671073Z ++ set -ex 2024-08-20T22:21:03.9671657Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:21:03.9672172Z ++ BUILD_TEST_LIBTORCH=0 2024-08-20T22:21:03.9672848Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]] 2024-08-20T22:21:03.9676339Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-20T22:21:03.9697088Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-20T22:21:03.9697657Z + trap_add cleanup_workspace EXIT 2024-08-20T22:21:03.9698090Z + trap_add_cmd=cleanup_workspace 2024-08-20T22:21:03.9698473Z + shift 2024-08-20T22:21:03.9698760Z + for trap_add_name in "$@" 2024-08-20T22:21:03.9705282Z +++ trap -p EXIT 2024-08-20T22:21:03.9708628Z ++ eval 'extract_trap_cmd ' 2024-08-20T22:21:03.9709026Z +++ extract_trap_cmd 2024-08-20T22:21:03.9709398Z +++ printf '%s\n' '' 2024-08-20T22:21:03.9709803Z ++ printf '%s\n' cleanup_workspace 2024-08-20T22:21:03.9712712Z + trap -- ' 2024-08-20T22:21:03.9713135Z cleanup_workspace' EXIT 2024-08-20T22:21:03.9713631Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-20T22:21:04.6703130Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-20T22:21:04.6726169Z + echo 'Environment variables:' 2024-08-20T22:21:04.6726710Z Environment variables: 2024-08-20T22:21:04.6727136Z + env 2024-08-20T22:21:04.6736521Z INSTALLED_DB=yes 2024-08-20T22:21:04.6737066Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-20T22:21:04.6737569Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:21:04.6738061Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-20T22:21:04.6738871Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:21:04.6739664Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:21:04.6740282Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-20T22:21:04.6740944Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:21:04.6741729Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:21:04.6742310Z HOSTNAME=d9bd82139fb7 2024-08-20T22:21:04.6743516Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_f396f70c-bce0-448d-94f9-8bc0844c827c 2024-08-20T22:21:04.6744365Z GITHUB_ACTION=__self 2024-08-20T22:21:04.6744738Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:21:04.6749493Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-20T22:21:04.6754227Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-20T22:21:04.6754749Z NV_NVTX_VERSION=12.1.105-1 2024-08-20T22:21:04.6755104Z GITHUB_RUN_NUMBER=88082 2024-08-20T22:21:04.6755463Z TEST_CONFIG=aot_inductor_torchbench 2024-08-20T22:21:04.6755878Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:21:04.6756416Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:21:04.6756882Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-20T22:21:04.6757326Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-20T22:21:04.6758020Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:21:04.6758433Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-20T22:21:04.6758951Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:21:04.6759444Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:21:04.6759908Z GITHUB_REF_TYPE=tag 2024-08-20T22:21:04.6760227Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:21:04.6760607Z NCCL_VERSION=2.17.1-1 2024-08-20T22:21:04.6760999Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.6761442Z XLA_CUDA= 2024-08-20T22:21:04.6762147Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:21:04.6764561Z *** 2024-08-20T22:21:04.6764859Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:21:04.6765262Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:21:04.6765622Z GITHUB_ACTIONS=true 2024-08-20T22:21:04.6765957Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:21:04.6766484Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-20T22:21:04.6767020Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-20T22:21:04.6767501Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.6768019Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:21:04.6768501Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.6769289Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:21:04.6770208Z UCC_HOME=/usr 2024-08-20T22:21:04.6770577Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-20T22:21:04.6770977Z VERBOSE_TEST_LOGS=False 2024-08-20T22:21:04.6771317Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:21:04.6771789Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-20T22:21:04.6772294Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:21:04.6772817Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-20T22:21:04.6773191Z SHARD_NUMBER=2 2024-08-20T22:21:04.6773501Z GITHUB_REF_PROTECTED=false 2024-08-20T22:21:04.6773851Z HOME=/var/lib/jenkins 2024-08-20T22:21:04.6774211Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:21:04.6774652Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:21:04.6775133Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:21:04.6775623Z SCCACHE_S3_KEY_PREFIX=inductor 2024-08-20T22:21:04.6775982Z CUDA_VERSION=12.1.1 2024-08-20T22:21:04.6776408Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-20T22:21:04.6776851Z NUM_TEST_SHARDS=2 2024-08-20T22:21:04.6777153Z UCX_HOME=/usr 2024-08-20T22:21:04.6777661Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-20T22:21:04.6778750Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_f396f70c-bce0-448d-94f9-8bc0844c827c 2024-08-20T22:21:04.6780065Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:21:04.6781345Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_f396f70c-bce0-448d-94f9-8bc0844c827c 2024-08-20T22:21:04.6782461Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:21:04.6783122Z GITHUB_EVENT_NAME=push 2024-08-20T22:21:04.6783453Z DASHBOARD_TAG= 2024-08-20T22:21:04.6783756Z GITHUB_RUN_ID=10479310725 2024-08-20T22:21:04.6784214Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-20T22:21:04.6784753Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-20T22:21:04.6785799Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_f396f70c-bce0-448d-94f9-8bc0844c827c 2024-08-20T22:21:04.6786723Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:21:04.6787132Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-20T22:21:04.6787509Z PR_NUMBER= 2024-08-20T22:21:04.6787795Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:21:04.6788127Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:21:04.6788567Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:21:04.6789026Z TERM=vt100 2024-08-20T22:21:04.6789361Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-20T22:21:04.6789771Z INSTALLED_VISION=yes 2024-08-20T22:21:04.6790082Z BRANCH= 2024-08-20T22:21:04.6790492Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:21:04.6790900Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:21:04.6791318Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:21:04.6792082Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:21:04.6792836Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:21:04.6793315Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:21:04.6793785Z REENABLED_ISSUES= 2024-08-20T22:21:04.6794170Z SHLVL=1 2024-08-20T22:21:04.6794434Z MAX_JOBS=14 2024-08-20T22:21:04.6794744Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-20T22:21:04.6795186Z NVARCH=x86_64 2024-08-20T22:21:04.6795596Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:21:04.6796059Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.6796594Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:21:04.6797093Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-20T22:21:04.6797758Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:21:04.6798340Z GITHUB_JOB=test 2024-08-20T22:21:04.6798743Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-20T22:21:04.6799301Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:21:04.6799813Z NO_TEST_TIMEOUT=False 2024-08-20T22:21:04.6800135Z TD_DISTRIBUTED=False 2024-08-20T22:21:04.6800515Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-20T22:21:04.6800950Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:21:04.6801368Z NV_NVPROF_VERSION=12.1.105-1 2024-08-20T22:21:04.6801733Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:21:04.6802085Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:21:04.6802434Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:21:04.6803484Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:21:04.6804665Z GITHUB_BASE_REF= 2024-08-20T22:21:04.6804989Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:21:04.6805366Z CI=true 2024-08-20T22:21:04.6805684Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:21:04.6806098Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:21:04.6806469Z JOB_ID=29026530815 2024-08-20T22:21:04.6806780Z INSTALLED_PROTOBUF=yes 2024-08-20T22:21:04.6807103Z GITHUB_HEAD_REF= 2024-08-20T22:21:04.6807405Z GITHUB_ACTION_REF= 2024-08-20T22:21:04.6807847Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:21:04.6808311Z TEST_SHOWLOCALS=False 2024-08-20T22:21:04.6808637Z GITHUB_WORKFLOW=inductor 2024-08-20T22:21:04.6809000Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:21:04.6809912Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_f396f70c-bce0-448d-94f9-8bc0844c827c 2024-08-20T22:21:04.6810726Z NO_TD=False 2024-08-20T22:21:04.6811021Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:21:04.6811382Z _=/usr/bin/env 2024-08-20T22:21:04.6811863Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-20T22:21:04.6970616Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-20T22:21:04.6971876Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-20T22:21:04.6973008Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-20T22:21:04.6973897Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-20T22:21:04.6974510Z + BUILD_DIR=build 2024-08-20T22:21:04.6974830Z + BUILD_RENAMED_DIR=build_renamed 2024-08-20T22:21:04.6975222Z + BUILD_BIN_DIR=build/bin 2024-08-20T22:21:04.6975570Z + SHARD_NUMBER=2 2024-08-20T22:21:04.6975871Z + NUM_TEST_SHARDS=2 2024-08-20T22:21:04.6976190Z + export VALGRIND=ON 2024-08-20T22:21:04.6976511Z + VALGRIND=ON 2024-08-20T22:21:04.6977082Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]] 2024-08-20T22:21:04.6977721Z + [[ 0 == \1 ]] 2024-08-20T22:21:04.6978057Z + [[ False == \1 ]] 2024-08-20T22:21:04.6978540Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]] 2024-08-20T22:21:04.6979256Z ++ realpath build/custom_test_artifacts 2024-08-20T22:21:04.6988583Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-20T22:21:04.6989530Z + [[ -n '' ]] 2024-08-20T22:21:04.6989973Z + echo 'Environment variables' 2024-08-20T22:21:04.6990362Z Environment variables 2024-08-20T22:21:04.6990676Z + env 2024-08-20T22:21:04.6999205Z INSTALLED_DB=yes 2024-08-20T22:21:04.6999768Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-20T22:21:04.7000220Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:21:04.7000955Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-20T22:21:04.7001748Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:21:04.7002506Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:21:04.7003217Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-20T22:21:04.7004053Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:21:04.7004672Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:21:04.7005338Z HOSTNAME=d9bd82139fb7 2024-08-20T22:21:04.7006406Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_f396f70c-bce0-448d-94f9-8bc0844c827c 2024-08-20T22:21:04.7007483Z GITHUB_ACTION=__self 2024-08-20T22:21:04.7007842Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:21:04.7011908Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-20T22:21:04.7016132Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-20T22:21:04.7016656Z NV_NVTX_VERSION=12.1.105-1 2024-08-20T22:21:04.7017012Z GITHUB_RUN_NUMBER=88082 2024-08-20T22:21:04.7017367Z TEST_CONFIG=aot_inductor_torchbench 2024-08-20T22:21:04.7017786Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:21:04.7018262Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:21:04.7018735Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-20T22:21:04.7019179Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-20T22:21:04.7019593Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:21:04.7019990Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-20T22:21:04.7020494Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:21:04.7020994Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:21:04.7021462Z GITHUB_REF_TYPE=tag 2024-08-20T22:21:04.7021787Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:21:04.7022173Z NCCL_VERSION=2.17.1-1 2024-08-20T22:21:04.7022566Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.7023015Z XLA_CUDA= 2024-08-20T22:21:04.7023464Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:21:04.7023851Z *** 2024-08-20T22:21:04.7024136Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:21:04.7024537Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:21:04.7024906Z GITHUB_ACTIONS=true 2024-08-20T22:21:04.7025240Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:21:04.7025733Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-20T22:21:04.7026270Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-20T22:21:04.7026753Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.7027265Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:21:04.7027752Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.7028546Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:21:04.7029265Z UCC_HOME=/usr 2024-08-20T22:21:04.7029601Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-20T22:21:04.7030165Z VERBOSE_TEST_LOGS=False 2024-08-20T22:21:04.7030515Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:21:04.7030982Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-20T22:21:04.7031490Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:21:04.7031961Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-20T22:21:04.7032342Z SHARD_NUMBER=2 2024-08-20T22:21:04.7032660Z GITHUB_REF_PROTECTED=false 2024-08-20T22:21:04.7033007Z HOME=/var/lib/jenkins 2024-08-20T22:21:04.7033373Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:21:04.7033909Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:21:04.7034384Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:21:04.7034870Z SCCACHE_S3_KEY_PREFIX=inductor 2024-08-20T22:21:04.7035241Z CUDA_VERSION=12.1.1 2024-08-20T22:21:04.7035668Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-20T22:21:04.7036105Z NUM_TEST_SHARDS=2 2024-08-20T22:21:04.7036413Z UCX_HOME=/usr 2024-08-20T22:21:04.7036931Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-20T22:21:04.7038021Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_f396f70c-bce0-448d-94f9-8bc0844c827c 2024-08-20T22:21:04.7039318Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 2, 2, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:21:04.7040592Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_f396f70c-bce0-448d-94f9-8bc0844c827c 2024-08-20T22:21:04.7041717Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:21:04.7042384Z GITHUB_EVENT_NAME=push 2024-08-20T22:21:04.7042710Z DASHBOARD_TAG= 2024-08-20T22:21:04.7043025Z GITHUB_RUN_ID=10479310725 2024-08-20T22:21:04.7043489Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-20T22:21:04.7044157Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-20T22:21:04.7045151Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_f396f70c-bce0-448d-94f9-8bc0844c827c 2024-08-20T22:21:04.7046429Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:21:04.7046848Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-20T22:21:04.7047227Z PR_NUMBER= 2024-08-20T22:21:04.7047508Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:21:04.7047828Z VALGRIND=ON 2024-08-20T22:21:04.7048127Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:21:04.7048561Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:21:04.7049019Z TERM=vt100 2024-08-20T22:21:04.7049356Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-20T22:21:04.7049775Z INSTALLED_VISION=yes 2024-08-20T22:21:04.7050083Z BRANCH= 2024-08-20T22:21:04.7050369Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:21:04.7050771Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:21:04.7051197Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:21:04.7051954Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:21:04.7052712Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:21:04.7053203Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:21:04.7053668Z REENABLED_ISSUES= 2024-08-20T22:21:04.7053969Z SHLVL=1 2024-08-20T22:21:04.7054234Z MAX_JOBS=14 2024-08-20T22:21:04.7054550Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-20T22:21:04.7054897Z NVARCH=x86_64 2024-08-20T22:21:04.7055200Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:21:04.7055659Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:21:04.7056199Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:21:04.7056684Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-20T22:21:04.7057351Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:21:04.7057926Z GITHUB_JOB=test 2024-08-20T22:21:04.7058324Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-20T22:21:04.7058881Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:21:04.7059396Z NO_TEST_TIMEOUT=False 2024-08-20T22:21:04.7059726Z TD_DISTRIBUTED=False 2024-08-20T22:21:04.7060101Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-20T22:21:04.7060697Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:21:04.7061128Z NV_NVPROF_VERSION=12.1.105-1 2024-08-20T22:21:04.7061490Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:21:04.7061832Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:21:04.7062182Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:21:04.7063226Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:21:04.7064291Z GITHUB_BASE_REF= 2024-08-20T22:21:04.7064737Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:21:04.7065107Z CI=true 2024-08-20T22:21:04.7065423Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:21:04.7065830Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:21:04.7066203Z JOB_ID=29026530815 2024-08-20T22:21:04.7066515Z INSTALLED_PROTOBUF=yes 2024-08-20T22:21:04.7066843Z GITHUB_HEAD_REF= 2024-08-20T22:21:04.7067139Z GITHUB_ACTION_REF= 2024-08-20T22:21:04.7067573Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:21:04.7068042Z TEST_SHOWLOCALS=False 2024-08-20T22:21:04.7068379Z GITHUB_WORKFLOW=inductor 2024-08-20T22:21:04.7068731Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:21:04.7069646Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_f396f70c-bce0-448d-94f9-8bc0844c827c 2024-08-20T22:21:04.7070460Z NO_TD=False 2024-08-20T22:21:04.7070781Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:21:04.7071137Z _=/usr/bin/env 2024-08-20T22:21:04.7071480Z + echo 'Testing pytorch' 2024-08-20T22:21:04.7071830Z Testing pytorch 2024-08-20T22:21:04.7072151Z + export LANG=C.UTF-8 2024-08-20T22:21:04.7072491Z + LANG=C.UTF-8 2024-08-20T22:21:04.7072783Z + PR_NUMBER= 2024-08-20T22:21:04.7073132Z + [[ aot_inductor_torchbench == \d\e\f\a\u\l\t ]] 2024-08-20T22:21:04.7073665Z + [[ aot_inductor_torchbench == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-20T22:21:04.7074190Z + [[ aot_inductor_torchbench == \s\l\o\w ]] 2024-08-20T22:21:04.7074840Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-08-20T22:21:04.7075545Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]] 2024-08-20T22:21:04.7076113Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:21:04.7076609Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:21:04.7077065Z + [[ aot_inductor_torchbench == *crossref* ]] 2024-08-20T22:21:04.7077653Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:21:04.7078288Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]] 2024-08-20T22:21:04.7078941Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:21:04.7079506Z + pip_install --user ninja==1.10.2 2024-08-20T22:21:04.7080040Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-20T22:21:05.6953154Z Collecting ninja==1.10.2 2024-08-20T22:21:05.7137544Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-20T22:21:05.7600979Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-20T22:21:06.3615299Z Installing collected packages: ninja 2024-08-20T22:21:06.3693784Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-20T22:21:06.3695077Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-20T22:21:06.4157386Z Successfully installed ninja-1.10.2 2024-08-20T22:21:06.4956736Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:21:06.4958928Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:21:06.4960552Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:21:06.4961066Z + install_tlparse 2024-08-20T22:21:06.4961823Z + pip_install --user tlparse==0.3.25 2024-08-20T22:21:06.4962554Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-20T22:21:06.9155014Z Collecting tlparse==0.3.25 2024-08-20T22:21:06.9325152Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-20T22:21:06.9775701Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-20T22:21:07.6225327Z Installing collected packages: tlparse 2024-08-20T22:21:07.6960516Z Successfully installed tlparse-0.3.25 2024-08-20T22:21:07.7746210Z ++ python -m site --user-base 2024-08-20T22:21:07.7992316Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:21:07.7993943Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]] 2024-08-20T22:21:07.7994650Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]] 2024-08-20T22:21:07.7995323Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:21:07.7996280Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-08-20T22:21:07.7997444Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass 2024-08-20T22:21:07.7998161Z + cd test 2024-08-20T22:21:07.7998701Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-20T22:21:09.4308735Z + [[ aot_inductor_torchbench == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-20T22:21:09.4309382Z + [[ aot_inductor_torchbench == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-20T22:21:09.4313436Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-20T22:21:09.4314519Z + [[ aot_inductor_torchbench == *pr_time_benchmarks* ]] 2024-08-20T22:21:09.4315051Z + [[ aot_inductor_torchbench == *dynamo_eager* ]] 2024-08-20T22:21:09.4315561Z + [[ aot_inductor_torchbench == *aot_eager* ]] 2024-08-20T22:21:09.4316074Z + [[ aot_inductor_torchbench == *aot_inductor* ]] 2024-08-20T22:21:09.4316770Z + DYNAMO_BENCHMARK_FLAGS+=(--export-aot-inductor) 2024-08-20T22:21:09.4317292Z + [[ aot_inductor_torchbench == *dynamic* ]] 2024-08-20T22:21:09.4317782Z + [[ aot_inductor_torchbench == *cpu* ]] 2024-08-20T22:21:09.4318288Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-20T22:21:09.4347120Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:21:09.4347813Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-08-20T22:21:09.4350152Z + cd test 2024-08-20T22:21:09.4350621Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-20T22:21:10.9077926Z PyTorch built with: 2024-08-20T22:21:10.9078464Z - GCC 9.4 2024-08-20T22:21:10.9078802Z - C++ Version: 201703 2024-08-20T22:21:10.9079844Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:21:10.9081123Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:21:10.9081838Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:21:10.9082357Z - LAPACK is enabled (usually provided by MKL) 2024-08-20T22:21:10.9082846Z - NNPACK is enabled 2024-08-20T22:21:10.9083234Z - CPU capability usage: AVX2 2024-08-20T22:21:10.9083759Z - CUDA Runtime 12.1 2024-08-20T22:21:10.9084294Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-08-20T22:21:10.9084907Z - CuDNN 90.1 (built against CUDA 12.4) 2024-08-20T22:21:10.9085376Z - Magma 2.6.1 2024-08-20T22:21:10.9092838Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-20T22:21:10.9099525Z 2024-08-20T22:21:11.2341476Z + cd test 2024-08-20T22:21:11.2342244Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-20T22:21:12.5735510Z ATen/Parallel: 2024-08-20T22:21:12.5735991Z at::get_num_threads() : 8 2024-08-20T22:21:12.5736499Z at::get_num_interop_threads() : 16 2024-08-20T22:21:12.5737056Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:21:12.5737567Z omp_get_max_threads() : 8 2024-08-20T22:21:12.5738753Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:21:12.5739559Z mkl_get_max_threads() : 8 2024-08-20T22:21:12.5740196Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:21:12.5740847Z std::thread::hardware_concurrency() : 16 2024-08-20T22:21:12.5741275Z Environment variables: 2024-08-20T22:21:12.5741619Z OMP_NUM_THREADS : [not set] 2024-08-20T22:21:12.5741998Z MKL_NUM_THREADS : [not set] 2024-08-20T22:21:12.5742375Z ATen parallel backend: OpenMP 2024-08-20T22:21:12.5742629Z 2024-08-20T22:21:12.8498178Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:21:12.8498882Z + [[ aot_inductor_torchbench == *backward* ]] 2024-08-20T22:21:12.8499471Z + [[ aot_inductor_torchbench == *xla* ]] 2024-08-20T22:21:12.8499956Z + [[ aot_inductor_torchbench == *executorch* ]] 2024-08-20T22:21:12.8500476Z + [[ aot_inductor_torchbench == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-20T22:21:12.8501140Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:21:12.8501696Z + [[ aot_inductor_torchbench == distributed ]] 2024-08-20T22:21:12.8502227Z + [[ aot_inductor_torchbench == *inductor_distributed* ]] 2024-08-20T22:21:12.8502835Z + [[ aot_inductor_torchbench == *inductor-halide* ]] 2024-08-20T22:21:12.8503479Z + [[ aot_inductor_torchbench == *inductor-micro-benchmark* ]] 2024-08-20T22:21:12.8504044Z + [[ aot_inductor_torchbench == *huggingface* ]] 2024-08-20T22:21:12.8504516Z + [[ aot_inductor_torchbench == *timm* ]] 2024-08-20T22:21:12.8504983Z + [[ aot_inductor_torchbench == *torchbench* ]] 2024-08-20T22:21:12.8505453Z + [[ aot_inductor_torchbench == *cpu* ]] 2024-08-20T22:21:12.8505874Z + install_torchaudio cuda 2024-08-20T22:21:12.8506209Z + local commit 2024-08-20T22:21:12.8506526Z ++ get_pinned_commit audio 2024-08-20T22:21:12.8506901Z ++ cat .github/ci_commit_pins/audio.txt 2024-08-20T22:21:12.8523034Z + commit=b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:21:12.8523489Z + [[ cuda == \c\u\d\a ]] 2024-08-20T22:21:12.8523989Z + TORCH_CUDA_ARCH_LIST='8.0;8.6' 2024-08-20T22:21:12.8524912Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:21:12.8526363Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:21:13.1871248Z Collecting git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:21:13.1875648Z Cloning https://github.com/pytorch/audio.git (to revision b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862) to /tmp/pip-req-build-b4wpw7pn 2024-08-20T22:21:13.1905226Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/audio.git /tmp/pip-req-build-b4wpw7pn 2024-08-20T22:21:13.9805148Z Running command git rev-parse -q --verify 'sha^b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862' 2024-08-20T22:21:13.9834696Z Running command git fetch -q https://github.com/pytorch/audio.git b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:21:14.3851776Z Resolved https://github.com/pytorch/audio.git to commit b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:21:14.3852908Z Running command git submodule update --init --recursive -q 2024-08-20T22:21:16.8029073Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:21:16.8060545Z [?25hRequirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchaudio==2.4.0a0+b3f6f51) (2.5.0a0+git40ec5f6) 2024-08-20T22:21:16.8091831Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (3.13.1) 2024-08-20T22:21:16.8096218Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (4.12.2) 2024-08-20T22:21:16.8099422Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (2.8.8) 2024-08-20T22:21:16.8102927Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (3.1.4) 2024-08-20T22:21:16.8106376Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (2024.6.1) 2024-08-20T22:21:16.8111730Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (1.13.1) 2024-08-20T22:21:16.8126689Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchaudio==2.4.0a0+b3f6f51) (1.3.0) 2024-08-20T22:21:16.8597980Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchaudio==2.4.0a0+b3f6f51) (2.1.5) 2024-08-20T22:21:16.8659700Z Building wheels for collected packages: torchaudio 2024-08-20T22:23:50.3870888Z Building wheel for torchaudio (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2024-08-20T22:23:50.3908251Z [?25h Created wheel for torchaudio: filename=torchaudio-2.4.0a0+b3f6f51-cp310-cp310-linux_x86_64.whl size=2310399 sha256=b14892822f011ab886446bc18ae6401ee89c71f1d7d5be66b7bde2946eaaedd6 2024-08-20T22:23:50.3910711Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/b5/02/94/08dddd0295a3ebec4d4a4d9f226adfd865d48fb8ae4af2cc95 2024-08-20T22:23:50.3937564Z Successfully built torchaudio 2024-08-20T22:23:50.8686154Z Installing collected packages: torchaudio 2024-08-20T22:23:51.0815661Z Successfully installed torchaudio-2.4.0a0+b3f6f51 2024-08-20T22:23:51.2597939Z + install_torchtext 2024-08-20T22:23:51.2598325Z + local data_commit 2024-08-20T22:23:51.2598637Z + local text_commit 2024-08-20T22:23:51.2601881Z ++ get_pinned_commit data 2024-08-20T22:23:51.2602372Z ++ cat .github/ci_commit_pins/data.txt 2024-08-20T22:23:51.2635406Z + data_commit=11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:23:51.2638916Z ++ get_pinned_commit text 2024-08-20T22:23:51.2639381Z ++ cat .github/ci_commit_pins/text.txt 2024-08-20T22:23:51.2658137Z + text_commit=b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:23:51.2659432Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:23:51.2661332Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:23:51.6129006Z Collecting git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:23:51.6133983Z Cloning https://github.com/pytorch/data.git (to revision 11bb5b847ea8b9e0d9bb82db3304daf383008d3f) to /tmp/pip-req-build-p51q6963 2024-08-20T22:23:51.6174248Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/data.git /tmp/pip-req-build-p51q6963 2024-08-20T22:23:52.0404972Z Running command git rev-parse -q --verify 'sha^11bb5b847ea8b9e0d9bb82db3304daf383008d3f' 2024-08-20T22:23:52.0432830Z Running command git fetch -q https://github.com/pytorch/data.git 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:23:52.3238796Z Running command git checkout -q 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:23:52.4746202Z Resolved https://github.com/pytorch/data.git to commit 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:23:52.4747412Z Running command git submodule update --init --recursive -q 2024-08-20T22:25:14.6384728Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:25:14.6426759Z [?25hRequirement already satisfied: urllib3>=1.25 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (1.26.19) 2024-08-20T22:25:14.6428968Z Requirement already satisfied: requests in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.32.3) 2024-08-20T22:25:14.6434000Z Requirement already satisfied: torch>2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.5.0a0+git40ec5f6) 2024-08-20T22:25:14.6464707Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.13.1) 2024-08-20T22:25:14.6469089Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (4.12.2) 2024-08-20T22:25:14.6472818Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.8.8) 2024-08-20T22:25:14.6475480Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.1.4) 2024-08-20T22:25:14.6479053Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2024.6.1) 2024-08-20T22:25:14.6484276Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.13.1) 2024-08-20T22:25:14.6499012Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.3.0) 2024-08-20T22:25:14.6558491Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.3.2) 2024-08-20T22:25:14.6562019Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.7) 2024-08-20T22:25:14.6567212Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (2024.7.4) 2024-08-20T22:25:14.7061512Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.1.5) 2024-08-20T22:25:14.7125528Z Building wheels for collected packages: torchdata 2024-08-20T22:25:16.6730729Z Building wheel for torchdata (setup.py) ... [?25l- \ | done 2024-08-20T22:25:16.6739373Z [?25h Created wheel for torchdata: filename=torchdata-0.7.0a0+11bb5b8-py3-none-any.whl size=182970 sha256=4168ceaa601680d93f0efccbf57f4392ed6f1b5b265b633c3fb68ee8c679fcab 2024-08-20T22:25:16.6741358Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/ca/59/a3/c8250bfc8d3d4d639498d4beb2e0f0e70b9a508ac61fde85ce 2024-08-20T22:25:16.6767555Z Successfully built torchdata 2024-08-20T22:25:17.1475274Z Installing collected packages: torchdata 2024-08-20T22:25:17.2469012Z Successfully installed torchdata-0.7.0a0+11bb5b8 2024-08-20T22:25:22.2339979Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:25:22.2341831Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:25:22.5713727Z Collecting git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:25:22.5716923Z Cloning https://github.com/pytorch/text.git (to revision b0ebddc648d279826089db91775375221777a2db) to /tmp/pip-req-build-nweui0sp 2024-08-20T22:25:22.5748482Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/text.git /tmp/pip-req-build-nweui0sp 2024-08-20T22:25:23.4461515Z Running command git rev-parse -q --verify 'sha^b0ebddc648d279826089db91775375221777a2db' 2024-08-20T22:25:23.4487776Z Running command git fetch -q https://github.com/pytorch/text.git b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:25:23.9441476Z Running command git checkout -q b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:25:24.1473419Z Resolved https://github.com/pytorch/text.git to commit b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:25:24.1475888Z Running command git submodule update --init --recursive -q 2024-08-20T22:25:31.6148881Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:25:31.6182928Z [?25hRequirement already satisfied: tqdm in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (4.66.5) 2024-08-20T22:25:31.6186490Z Requirement already satisfied: requests in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.32.3) 2024-08-20T22:25:31.6189036Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.5.0a0+git40ec5f6) 2024-08-20T22:25:31.6192463Z Requirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (1.22.4) 2024-08-20T22:25:31.6195643Z Requirement already satisfied: torchdata in /var/lib/jenkins/.local/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (0.7.0a0+11bb5b8) 2024-08-20T22:25:31.6221801Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.3.2) 2024-08-20T22:25:31.6225650Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.7) 2024-08-20T22:25:31.6229866Z Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (1.26.19) 2024-08-20T22:25:31.6233698Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (2024.7.4) 2024-08-20T22:25:31.6261747Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.13.1) 2024-08-20T22:25:31.6265674Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (4.12.2) 2024-08-20T22:25:31.6268708Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2.8.8) 2024-08-20T22:25:31.6272492Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.1.4) 2024-08-20T22:25:31.6275417Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2024.6.1) 2024-08-20T22:25:31.6280346Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (1.13.1) 2024-08-20T22:25:31.6295200Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchtext==0.16.0a0+b0ebddc) (1.3.0) 2024-08-20T22:25:31.6873086Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchtext==0.16.0a0+b0ebddc) (2.1.5) 2024-08-20T22:25:31.6939063Z Building wheels for collected packages: torchtext 2024-08-20T22:26:09.9233160Z Building wheel for torchtext (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / done 2024-08-20T22:26:09.9265932Z [?25h Created wheel for torchtext: filename=torchtext-0.16.0a0+b0ebddc-cp310-cp310-linux_x86_64.whl size=2045456 sha256=c54eea9ad9ac4a6884faf498c59162ba88fa26c0eb3ea95abac6d4ad029b8ffe 2024-08-20T22:26:09.9268826Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/e5/3d/d7/faafad098ec9437ecdf4495c0f5e72b817fffbb06063e32cbf 2024-08-20T22:26:09.9294355Z Successfully built torchtext 2024-08-20T22:26:10.4002195Z Installing collected packages: torchtext 2024-08-20T22:26:10.5257345Z Successfully installed torchtext-0.16.0a0+b0ebddc 2024-08-20T22:26:10.6885960Z + install_torchvision 2024-08-20T22:26:10.6886366Z + local orig_preload 2024-08-20T22:26:10.6886687Z + local commit 2024-08-20T22:26:10.6889711Z ++ get_pinned_commit vision 2024-08-20T22:26:10.6890160Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-20T22:26:10.6907692Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:26:10.6908158Z + orig_preload= 2024-08-20T22:26:10.6908670Z + '[' -n '' ']' 2024-08-20T22:26:10.6909528Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:26:10.6910985Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:26:11.0264581Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:26:11.0269252Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-kesf7nti 2024-08-20T22:26:11.0297191Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-kesf7nti 2024-08-20T22:26:12.5189437Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-20T22:26:12.5216806Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:26:13.8556476Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:26:14.1466754Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:26:16.6336554Z Preparing metadata (setup.py) ... [?25l- \ done 2024-08-20T22:26:16.6369860Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.22.4) 2024-08-20T22:26:16.6374339Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git40ec5f6) 2024-08-20T22:26:16.6378897Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-20T22:26:16.6452028Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-20T22:26:16.6456651Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-20T22:26:16.6460410Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-20T22:26:16.6462942Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-20T22:26:16.6466892Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-20T22:26:16.6471880Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.1) 2024-08-20T22:26:16.6486714Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-20T22:26:16.6971825Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-20T22:26:16.7038248Z Building wheels for collected packages: torchvision 2024-08-20T22:27:33.5346898Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / done 2024-08-20T22:27:33.5380925Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2116242 sha256=b2cde9898c8dcf3e8d0c342b485c26bab5da20487efd8651b9c25cee82f12d7e 2024-08-20T22:27:33.5382642Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-08-20T22:27:33.5415622Z Successfully built torchvision 2024-08-20T22:27:34.0216396Z Installing collected packages: torchvision 2024-08-20T22:27:34.4387898Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-20T22:27:34.5817817Z + '[' -n '' ']' 2024-08-20T22:27:34.5818230Z + TORCH_CUDA_ARCH_LIST='8.0;8.6' 2024-08-20T22:27:34.5818756Z + pip_install git+https://github.com/pytorch/ao.git 2024-08-20T22:27:34.5819499Z + pip install --progress-bar off git+https://github.com/pytorch/ao.git 2024-08-20T22:27:34.9183522Z Collecting git+https://github.com/pytorch/ao.git 2024-08-20T22:27:34.9187324Z Cloning https://github.com/pytorch/ao.git to /tmp/pip-req-build-ahwpvv20 2024-08-20T22:27:34.9217055Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/ao.git /tmp/pip-req-build-ahwpvv20 2024-08-20T22:27:35.4711817Z Resolved https://github.com/pytorch/ao.git to commit 2c8e3f368613f896fde14cc24b9268c9e125575e 2024-08-20T22:27:37.9006052Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:27:37.9042992Z [?25hBuilding wheels for collected packages: torchao 2024-08-20T22:28:56.5180056Z Building wheel for torchao (setup.py) ... [?25l- \ | / - \ | / done 2024-08-20T22:28:56.5197324Z [?25h Created wheel for torchao: filename=torchao-0.4.0+git2c8e3f3-cp310-cp310-linux_x86_64.whl size=928018 sha256=68baf252bc30bdb6a29fcec68e5c974eadc248f996d510cc1692ed257737db51 2024-08-20T22:28:56.5200445Z Stored in directory: /tmp/pip-ephem-wheel-cache-n1s5kiqd/wheels/c0/df/d8/b1e2f5f1ea3bf141268a83a667137647b234f04c636e1e7a5e 2024-08-20T22:28:56.5228533Z Successfully built torchao 2024-08-20T22:28:57.0574949Z Installing collected packages: torchao 2024-08-20T22:28:57.2999808Z Successfully installed torchao-0.4.0+git2c8e3f3 2024-08-20T22:28:57.4067230Z + id=1 2024-08-20T22:28:57.4067989Z + pip_install opencv-python==4.8.0.74 2024-08-20T22:28:57.4074008Z + pip install --progress-bar off opencv-python==4.8.0.74 2024-08-20T22:28:57.8727536Z Collecting opencv-python==4.8.0.74 2024-08-20T22:28:57.8899084Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB) 2024-08-20T22:28:57.8993272Z Requirement already satisfied: numpy>=1.21.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opencv-python==4.8.0.74) (1.22.4) 2024-08-20T22:28:57.9062443Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.7 MB) 2024-08-20T22:28:59.0919276Z Installing collected packages: opencv-python 2024-08-20T22:28:59.9543999Z Successfully installed opencv-python-4.8.0.74 2024-08-20T22:29:00.0540252Z + [[ aot_inductor_torchbench == *inductor_torchbench_smoketest_perf* ]] 2024-08-20T22:29:00.0541106Z + [[ aot_inductor_torchbench == *inductor_torchbench_cpu_smoketest_perf* ]] 2024-08-20T22:29:00.0541867Z + [[ aot_inductor_torchbench == *torchbench_gcp_smoketest* ]] 2024-08-20T22:29:00.0542476Z + checkout_install_torchbench 2024-08-20T22:29:00.0542960Z + local commit 2024-08-20T22:29:00.0545199Z ++ get_pinned_commit torchbench 2024-08-20T22:29:00.0545958Z ++ cat .github/ci_commit_pins/torchbench.txt 2024-08-20T22:29:00.0561851Z + commit=23512dbebd44a11eb84afbf53c3c071dd105297e 2024-08-20T22:29:00.0562482Z + git clone https://github.com/pytorch/benchmark torchbench 2024-08-20T22:29:00.0578301Z Cloning into 'torchbench'... 2024-08-20T22:29:00.1980382Z remote: Enumerating objects: 31006, done. 2024-08-20T22:29:00.1982970Z remote: Counting objects: 0% (1/1746) 2024-08-20T22:29:00.1984388Z remote: Counting objects: 1% (18/1746) 2024-08-20T22:29:00.1988260Z remote: Counting objects: 2% (35/1746) 2024-08-20T22:29:00.1989008Z remote: Counting objects: 3% (53/1746) 2024-08-20T22:29:00.1989724Z remote: Counting objects: 4% (70/1746) 2024-08-20T22:29:00.1990382Z remote: Counting objects: 5% (88/1746) 2024-08-20T22:29:00.1991002Z remote: Counting objects: 6% (105/1746) 2024-08-20T22:29:00.1991563Z remote: Counting objects: 7% (123/1746) 2024-08-20T22:29:00.1992189Z remote: Counting objects: 8% (140/1746) 2024-08-20T22:29:00.1992903Z remote: Counting objects: 9% (158/1746) 2024-08-20T22:29:00.1993710Z remote: Counting objects: 10% (175/1746) 2024-08-20T22:29:00.1994346Z remote: Counting objects: 11% (193/1746) 2024-08-20T22:29:00.1995048Z remote: Counting objects: 12% (210/1746) 2024-08-20T22:29:00.1995679Z remote: Counting objects: 13% (227/1746) 2024-08-20T22:29:00.1996314Z remote: Counting objects: 14% (245/1746) 2024-08-20T22:29:00.1996958Z remote: Counting objects: 15% (262/1746) 2024-08-20T22:29:00.1997565Z remote: Counting objects: 16% (280/1746) 2024-08-20T22:29:00.1998187Z remote: Counting objects: 17% (297/1746) 2024-08-20T22:29:00.1998910Z remote: Counting objects: 18% (315/1746) 2024-08-20T22:29:00.1999458Z remote: Counting objects: 19% (332/1746) 2024-08-20T22:29:00.1999985Z remote: Counting objects: 20% (350/1746) 2024-08-20T22:29:00.2000609Z remote: Counting objects: 21% (367/1746) 2024-08-20T22:29:00.2001298Z remote: Counting objects: 22% (385/1746) 2024-08-20T22:29:00.2001940Z remote: Counting objects: 23% (402/1746) 2024-08-20T22:29:00.2002651Z remote: Counting objects: 24% (420/1746) 2024-08-20T22:29:00.2003381Z remote: Counting objects: 25% (437/1746) 2024-08-20T22:29:00.2004204Z remote: Counting objects: 26% (454/1746) 2024-08-20T22:29:00.2004889Z remote: Counting objects: 27% (472/1746) 2024-08-20T22:29:00.2005407Z remote: Counting objects: 28% (489/1746) 2024-08-20T22:29:00.2005922Z remote: Counting objects: 29% (507/1746) 2024-08-20T22:29:00.2006443Z remote: Counting objects: 30% (524/1746) 2024-08-20T22:29:00.2006974Z remote: Counting objects: 31% (542/1746) 2024-08-20T22:29:00.2007497Z remote: Counting objects: 32% (559/1746) 2024-08-20T22:29:00.2008388Z remote: Counting objects: 33% (577/1746) 2024-08-20T22:29:00.2008926Z remote: Counting objects: 34% (594/1746) 2024-08-20T22:29:00.2009454Z remote: Counting objects: 35% (612/1746) 2024-08-20T22:29:00.2009972Z remote: Counting objects: 36% (629/1746) 2024-08-20T22:29:00.2010493Z remote: Counting objects: 37% (647/1746) 2024-08-20T22:29:00.2011014Z remote: Counting objects: 38% (664/1746) 2024-08-20T22:29:00.2011529Z remote: Counting objects: 39% (681/1746) 2024-08-20T22:29:00.2012258Z remote: Counting objects: 40% (699/1746) 2024-08-20T22:29:00.2012787Z remote: Counting objects: 41% (716/1746) 2024-08-20T22:29:00.2013313Z remote: Counting objects: 42% (734/1746) 2024-08-20T22:29:00.2013841Z remote: Counting objects: 43% (751/1746) 2024-08-20T22:29:00.2014363Z remote: Counting objects: 44% (769/1746) 2024-08-20T22:29:00.2014884Z remote: Counting objects: 45% (786/1746) 2024-08-20T22:29:00.2015412Z remote: Counting objects: 46% (804/1746) 2024-08-20T22:29:00.2015942Z remote: Counting objects: 47% (821/1746) 2024-08-20T22:29:00.2016458Z remote: Counting objects: 48% (839/1746) 2024-08-20T22:29:00.2016986Z remote: Counting objects: 49% (856/1746) 2024-08-20T22:29:00.2017508Z remote: Counting objects: 50% (873/1746) 2024-08-20T22:29:00.2018026Z remote: Counting objects: 51% (891/1746) 2024-08-20T22:29:00.2018553Z remote: Counting objects: 52% (908/1746) 2024-08-20T22:29:00.2019084Z remote: Counting objects: 53% (926/1746) 2024-08-20T22:29:00.2019617Z remote: Counting objects: 54% (943/1746) 2024-08-20T22:29:00.2020132Z remote: Counting objects: 55% (961/1746) 2024-08-20T22:29:00.2020656Z remote: Counting objects: 56% (978/1746) 2024-08-20T22:29:00.2021181Z remote: Counting objects: 57% (996/1746) 2024-08-20T22:29:00.2021712Z remote: Counting objects: 58% (1013/1746) 2024-08-20T22:29:00.2022251Z remote: Counting objects: 59% (1031/1746) 2024-08-20T22:29:00.2022800Z remote: Counting objects: 60% (1048/1746) 2024-08-20T22:29:00.2023348Z remote: Counting objects: 61% (1066/1746) 2024-08-20T22:29:00.2023886Z remote: Counting objects: 62% (1083/1746) 2024-08-20T22:29:00.2024436Z remote: Counting objects: 63% (1100/1746) 2024-08-20T22:29:00.2024976Z remote: Counting objects: 64% (1118/1746) 2024-08-20T22:29:00.2025519Z remote: Counting objects: 65% (1135/1746) 2024-08-20T22:29:00.2026115Z remote: Counting objects: 66% (1153/1746) 2024-08-20T22:29:00.2026649Z remote: Counting objects: 67% (1170/1746) 2024-08-20T22:29:00.2027196Z remote: Counting objects: 68% (1188/1746) 2024-08-20T22:29:00.2027727Z remote: Counting objects: 69% (1205/1746) 2024-08-20T22:29:00.2028268Z remote: Counting objects: 70% (1223/1746) 2024-08-20T22:29:00.2028795Z remote: Counting objects: 71% (1240/1746) 2024-08-20T22:29:00.2029328Z remote: Counting objects: 72% (1258/1746) 2024-08-20T22:29:00.2029860Z remote: Counting objects: 73% (1275/1746) 2024-08-20T22:29:00.2030390Z remote: Counting objects: 74% (1293/1746) 2024-08-20T22:29:00.2030959Z remote: Counting objects: 75% (1310/1746) 2024-08-20T22:29:00.2031548Z remote: Counting objects: 76% (1327/1746) 2024-08-20T22:29:00.2032072Z remote: Counting objects: 77% (1345/1746) 2024-08-20T22:29:00.2032585Z remote: Counting objects: 78% (1362/1746) 2024-08-20T22:29:00.2033104Z remote: Counting objects: 79% (1380/1746) 2024-08-20T22:29:00.2033626Z remote: Counting objects: 80% (1397/1746) 2024-08-20T22:29:00.2034147Z remote: Counting objects: 81% (1415/1746) 2024-08-20T22:29:00.2034670Z remote: Counting objects: 82% (1432/1746) 2024-08-20T22:29:00.2035198Z remote: Counting objects: 83% (1450/1746) 2024-08-20T22:29:00.2035718Z remote: Counting objects: 84% (1467/1746) 2024-08-20T22:29:00.2036239Z remote: Counting objects: 85% (1485/1746) 2024-08-20T22:29:00.2036752Z remote: Counting objects: 86% (1502/1746) 2024-08-20T22:29:00.2037281Z remote: Counting objects: 87% (1520/1746) 2024-08-20T22:29:00.2037897Z remote: Counting objects: 88% (1537/1746) 2024-08-20T22:29:00.2038428Z remote: Counting objects: 89% (1554/1746) 2024-08-20T22:29:00.2038946Z remote: Counting objects: 90% (1572/1746) 2024-08-20T22:29:00.2039478Z remote: Counting objects: 91% (1589/1746) 2024-08-20T22:29:00.2040006Z remote: Counting objects: 92% (1607/1746) 2024-08-20T22:29:00.2040528Z remote: Counting objects: 93% (1624/1746) 2024-08-20T22:29:00.2041052Z remote: Counting objects: 94% (1642/1746) 2024-08-20T22:29:00.2041672Z remote: Counting objects: 95% (1659/1746) 2024-08-20T22:29:00.2042194Z remote: Counting objects: 96% (1677/1746) 2024-08-20T22:29:00.2042708Z remote: Counting objects: 97% (1694/1746) 2024-08-20T22:29:00.2043228Z remote: Counting objects: 98% (1712/1746) 2024-08-20T22:29:00.2043748Z remote: Counting objects: 99% (1729/1746) 2024-08-20T22:29:00.2044413Z remote: Counting objects: 100% (1746/1746) 2024-08-20T22:29:00.2044969Z remote: Counting objects: 100% (1746/1746), done. 2024-08-20T22:29:00.2045874Z remote: Compressing objects: 0% (1/904) 2024-08-20T22:29:00.2089201Z remote: Compressing objects: 1% (10/904) 2024-08-20T22:29:00.2203503Z remote: Compressing objects: 2% (19/904) 2024-08-20T22:29:00.2287660Z remote: Compressing objects: 3% (28/904) 2024-08-20T22:29:00.2337618Z remote: Compressing objects: 4% (37/904) 2024-08-20T22:29:00.2387733Z remote: Compressing objects: 5% (46/904) 2024-08-20T22:29:00.2422638Z remote: Compressing objects: 6% (55/904) 2024-08-20T22:29:00.2451122Z remote: Compressing objects: 7% (64/904) 2024-08-20T22:29:00.2485464Z remote: Compressing objects: 8% (73/904) 2024-08-20T22:29:00.2520219Z remote: Compressing objects: 9% (82/904) 2024-08-20T22:29:00.2548703Z remote: Compressing objects: 10% (91/904) 2024-08-20T22:29:00.2562827Z remote: Compressing objects: 11% (100/904) 2024-08-20T22:29:00.2574883Z remote: Compressing objects: 12% (109/904) 2024-08-20T22:29:00.2589726Z remote: Compressing objects: 13% (118/904) 2024-08-20T22:29:00.2605495Z remote: Compressing objects: 14% (127/904) 2024-08-20T22:29:00.2614994Z remote: Compressing objects: 15% (136/904) 2024-08-20T22:29:00.2622677Z remote: Compressing objects: 16% (145/904) 2024-08-20T22:29:00.2629412Z remote: Compressing objects: 17% (154/904) 2024-08-20T22:29:00.2632439Z remote: Compressing objects: 18% (163/904) 2024-08-20T22:29:00.2635728Z remote: Compressing objects: 19% (172/904) 2024-08-20T22:29:00.2637441Z remote: Compressing objects: 20% (181/904) 2024-08-20T22:29:00.2638643Z remote: Compressing objects: 21% (190/904) 2024-08-20T22:29:00.2642127Z remote: Compressing objects: 22% (199/904) 2024-08-20T22:29:00.2643200Z remote: Compressing objects: 23% (208/904) 2024-08-20T22:29:00.2643911Z remote: Compressing objects: 24% (217/904) 2024-08-20T22:29:00.2644621Z remote: Compressing objects: 25% (226/904) 2024-08-20T22:29:00.2646114Z remote: Compressing objects: 26% (236/904) 2024-08-20T22:29:00.2654202Z remote: Compressing objects: 27% (245/904) 2024-08-20T22:29:00.2656723Z remote: Compressing objects: 28% (254/904) 2024-08-20T22:29:00.2660089Z remote: Compressing objects: 29% (263/904) 2024-08-20T22:29:00.2662566Z remote: Compressing objects: 30% (272/904) 2024-08-20T22:29:00.2665338Z remote: Compressing objects: 31% (281/904) 2024-08-20T22:29:00.2668338Z remote: Compressing objects: 32% (290/904) 2024-08-20T22:29:00.2671232Z remote: Compressing objects: 33% (299/904) 2024-08-20T22:29:00.2674064Z remote: Compressing objects: 34% (308/904) 2024-08-20T22:29:00.2676443Z remote: Compressing objects: 35% (317/904) 2024-08-20T22:29:00.2679468Z remote: Compressing objects: 36% (326/904) 2024-08-20T22:29:00.2681340Z remote: Compressing objects: 37% (335/904) 2024-08-20T22:29:00.2683490Z remote: Compressing objects: 38% (344/904) 2024-08-20T22:29:00.2684774Z remote: Compressing objects: 39% (353/904) 2024-08-20T22:29:00.2686729Z remote: Compressing objects: 40% (362/904) 2024-08-20T22:29:00.2687881Z remote: Compressing objects: 41% (371/904) 2024-08-20T22:29:00.2689224Z remote: Compressing objects: 42% (380/904) 2024-08-20T22:29:00.2691140Z remote: Compressing objects: 43% (389/904) 2024-08-20T22:29:00.2692323Z remote: Compressing objects: 44% (398/904) 2024-08-20T22:29:00.2692991Z remote: Compressing objects: 45% (407/904) 2024-08-20T22:29:00.2693549Z remote: Compressing objects: 46% (416/904) 2024-08-20T22:29:00.2694539Z remote: Compressing objects: 47% (425/904) 2024-08-20T22:29:00.2695247Z remote: Compressing objects: 48% (434/904) 2024-08-20T22:29:00.2695956Z remote: Compressing objects: 49% (443/904) 2024-08-20T22:29:00.2696715Z remote: Compressing objects: 50% (452/904) 2024-08-20T22:29:00.2697424Z remote: Compressing objects: 51% (462/904) 2024-08-20T22:29:00.2697979Z remote: Compressing objects: 52% (471/904) 2024-08-20T22:29:00.2698651Z remote: Compressing objects: 53% (480/904) 2024-08-20T22:29:00.2699385Z remote: Compressing objects: 54% (489/904) 2024-08-20T22:29:00.2700106Z remote: Compressing objects: 55% (498/904) 2024-08-20T22:29:00.2700709Z remote: Compressing objects: 56% (507/904) 2024-08-20T22:29:00.2701241Z remote: Compressing objects: 57% (516/904) 2024-08-20T22:29:00.2701782Z remote: Compressing objects: 58% (525/904) 2024-08-20T22:29:00.2702326Z remote: Compressing objects: 59% (534/904) 2024-08-20T22:29:00.2702869Z remote: Compressing objects: 60% (543/904) 2024-08-20T22:29:00.2703521Z remote: Compressing objects: 61% (552/904) 2024-08-20T22:29:00.2704068Z remote: Compressing objects: 62% (561/904) 2024-08-20T22:29:00.2704610Z remote: Compressing objects: 63% (570/904) 2024-08-20T22:29:00.2705150Z remote: Compressing objects: 64% (579/904) 2024-08-20T22:29:00.2705857Z remote: Compressing objects: 65% (588/904) 2024-08-20T22:29:00.2706548Z remote: Compressing objects: 66% (597/904) 2024-08-20T22:29:00.2707099Z remote: Compressing objects: 67% (606/904) 2024-08-20T22:29:00.2707646Z remote: Compressing objects: 68% (615/904) 2024-08-20T22:29:00.2708175Z remote: Compressing objects: 69% (624/904) 2024-08-20T22:29:00.2708725Z remote: Compressing objects: 70% (633/904) 2024-08-20T22:29:00.2709264Z remote: Compressing objects: 71% (642/904) 2024-08-20T22:29:00.2709792Z remote: Compressing objects: 72% (651/904) 2024-08-20T22:29:00.2710325Z remote: Compressing objects: 73% (660/904) 2024-08-20T22:29:00.2710897Z remote: Compressing objects: 74% (669/904) 2024-08-20T22:29:00.2711518Z remote: Compressing objects: 75% (678/904) 2024-08-20T22:29:00.2713473Z remote: Compressing objects: 76% (688/904) 2024-08-20T22:29:00.2721238Z remote: Compressing objects: 77% (697/904) 2024-08-20T22:29:00.2725570Z remote: Compressing objects: 78% (706/904) 2024-08-20T22:29:00.2731489Z remote: Compressing objects: 79% (715/904) 2024-08-20T22:29:00.2738263Z remote: Compressing objects: 80% (724/904) 2024-08-20T22:29:00.2749653Z remote: Compressing objects: 81% (733/904) 2024-08-20T22:29:00.2756965Z remote: Compressing objects: 82% (742/904) 2024-08-20T22:29:00.2760364Z remote: Compressing objects: 83% (751/904) 2024-08-20T22:29:00.2763999Z remote: Compressing objects: 84% (760/904) 2024-08-20T22:29:00.2768152Z remote: Compressing objects: 85% (769/904) 2024-08-20T22:29:00.2772915Z remote: Compressing objects: 86% (778/904) 2024-08-20T22:29:00.2777182Z remote: Compressing objects: 87% (787/904) 2024-08-20T22:29:00.2781046Z remote: Compressing objects: 88% (796/904) 2024-08-20T22:29:00.2784579Z remote: Compressing objects: 89% (805/904) 2024-08-20T22:29:00.2787438Z remote: Compressing objects: 90% (814/904) 2024-08-20T22:29:00.2790866Z remote: Compressing objects: 91% (823/904) 2024-08-20T22:29:00.2794017Z remote: Compressing objects: 92% (832/904) 2024-08-20T22:29:00.2797208Z remote: Compressing objects: 93% (841/904) 2024-08-20T22:29:00.2799823Z remote: Compressing objects: 94% (850/904) 2024-08-20T22:29:00.2802222Z remote: Compressing objects: 95% (859/904) 2024-08-20T22:29:00.2804748Z remote: Compressing objects: 96% (868/904) 2024-08-20T22:29:00.2807773Z remote: Compressing objects: 97% (877/904) 2024-08-20T22:29:00.2808857Z remote: Compressing objects: 98% (886/904) 2024-08-20T22:29:00.2810179Z remote: Compressing objects: 99% (895/904) 2024-08-20T22:29:00.2811245Z remote: Compressing objects: 100% (904/904) 2024-08-20T22:29:00.2812214Z remote: Compressing objects: 100% (904/904), done. 2024-08-20T22:29:00.2890414Z Receiving objects: 0% (1/31006) 2024-08-20T22:29:00.2931903Z Receiving objects: 1% (311/31006) 2024-08-20T22:29:00.2972619Z Receiving objects: 2% (621/31006) 2024-08-20T22:29:00.3009197Z Receiving objects: 3% (931/31006) 2024-08-20T22:29:00.3042400Z Receiving objects: 4% (1241/31006) 2024-08-20T22:29:00.3076141Z Receiving objects: 5% (1551/31006) 2024-08-20T22:29:00.3113124Z Receiving objects: 6% (1861/31006) 2024-08-20T22:29:00.3152965Z Receiving objects: 7% (2171/31006) 2024-08-20T22:29:00.3183074Z Receiving objects: 8% (2481/31006) 2024-08-20T22:29:00.3216031Z Receiving objects: 9% (2791/31006) 2024-08-20T22:29:00.3287479Z Receiving objects: 10% (3101/31006) 2024-08-20T22:29:00.3335605Z Receiving objects: 11% (3411/31006) 2024-08-20T22:29:00.3402415Z Receiving objects: 12% (3721/31006) 2024-08-20T22:29:00.3533941Z Receiving objects: 13% (4031/31006) 2024-08-20T22:29:00.3679038Z Receiving objects: 14% (4341/31006) 2024-08-20T22:29:00.3750592Z Receiving objects: 15% (4651/31006) 2024-08-20T22:29:00.3823589Z Receiving objects: 16% (4961/31006) 2024-08-20T22:29:00.3883129Z Receiving objects: 17% (5272/31006) 2024-08-20T22:29:00.3910727Z Receiving objects: 18% (5582/31006) 2024-08-20T22:29:00.3969747Z Receiving objects: 19% (5892/31006) 2024-08-20T22:29:00.4002568Z Receiving objects: 20% (6202/31006) 2024-08-20T22:29:00.4017699Z Receiving objects: 21% (6512/31006) 2024-08-20T22:29:00.4217454Z Receiving objects: 22% (6822/31006) 2024-08-20T22:29:01.2838290Z Receiving objects: 23% (7132/31006) 2024-08-20T22:29:01.4596146Z Receiving objects: 23% (7374/31006), 71.96 MiB | 71.95 MiB/s 2024-08-20T22:29:02.0395376Z Receiving objects: 24% (7442/31006), 71.96 MiB | 71.95 MiB/s 2024-08-20T22:29:02.1157966Z Receiving objects: 25% (7752/31006), 108.43 MiB | 72.28 MiB/s 2024-08-20T22:29:02.1927441Z Receiving objects: 26% (8062/31006), 108.43 MiB | 72.28 MiB/s 2024-08-20T22:29:02.2687435Z Receiving objects: 27% (8372/31006), 108.43 MiB | 72.28 MiB/s 2024-08-20T22:29:02.2835806Z Receiving objects: 28% (8682/31006), 108.43 MiB | 72.28 MiB/s 2024-08-20T22:29:02.3441409Z Receiving objects: 28% (8741/31006), 108.43 MiB | 72.28 MiB/s 2024-08-20T22:29:02.4204738Z Receiving objects: 29% (8992/31006), 143.28 MiB | 71.64 MiB/s 2024-08-20T22:29:02.4968489Z Receiving objects: 30% (9302/31006), 143.28 MiB | 71.64 MiB/s 2024-08-20T22:29:02.5729487Z Receiving objects: 31% (9612/31006), 143.28 MiB | 71.64 MiB/s 2024-08-20T22:29:02.8193116Z Receiving objects: 32% (9922/31006), 143.28 MiB | 71.64 MiB/s 2024-08-20T22:29:02.8960940Z Receiving objects: 33% (10232/31006), 179.31 MiB | 71.72 MiB/s 2024-08-20T22:29:02.9089128Z Receiving objects: 34% (10543/31006), 179.31 MiB | 71.72 MiB/s 2024-08-20T22:29:03.1814062Z Receiving objects: 35% (10853/31006), 179.31 MiB | 71.72 MiB/s 2024-08-20T22:29:03.2838118Z Receiving objects: 36% (11163/31006), 179.31 MiB | 71.72 MiB/s 2024-08-20T22:29:03.4485933Z Receiving objects: 36% (11226/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4503957Z Receiving objects: 37% (11473/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4527589Z Receiving objects: 38% (11783/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4546116Z Receiving objects: 39% (12093/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4561646Z Receiving objects: 40% (12403/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4572990Z Receiving objects: 41% (12713/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4586470Z Receiving objects: 42% (13023/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4602222Z Receiving objects: 43% (13333/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4619270Z Receiving objects: 44% (13643/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4626930Z Receiving objects: 45% (13953/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4631936Z Receiving objects: 46% (14263/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4647391Z Receiving objects: 47% (14573/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4669316Z Receiving objects: 48% (14883/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4682917Z Receiving objects: 49% (15193/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4888337Z Receiving objects: 50% (15503/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4894768Z Receiving objects: 51% (15814/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4901425Z Receiving objects: 52% (16124/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4908057Z Receiving objects: 53% (16434/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4914071Z Receiving objects: 54% (16744/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4931874Z Receiving objects: 55% (17054/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4943050Z Receiving objects: 56% (17364/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4949589Z Receiving objects: 57% (17674/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4963510Z Receiving objects: 58% (17984/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4977050Z Receiving objects: 59% (18294/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.4988008Z Receiving objects: 60% (18604/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5136707Z Receiving objects: 61% (18914/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5142734Z Receiving objects: 62% (19224/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5149565Z Receiving objects: 63% (19534/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5158130Z Receiving objects: 64% (19844/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5166129Z Receiving objects: 65% (20154/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5175691Z Receiving objects: 66% (20464/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5449368Z Receiving objects: 67% (20775/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5860821Z Receiving objects: 68% (21085/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5869811Z Receiving objects: 69% (21395/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5884498Z Receiving objects: 70% (21705/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5895918Z Receiving objects: 71% (22015/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5911423Z Receiving objects: 72% (22325/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5920827Z Receiving objects: 73% (22635/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5937773Z Receiving objects: 74% (22945/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5948252Z Receiving objects: 75% (23255/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5956466Z Receiving objects: 76% (23565/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.5964159Z Receiving objects: 77% (23875/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6181836Z Receiving objects: 78% (24185/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6215907Z Receiving objects: 79% (24495/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6225767Z Receiving objects: 80% (24805/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6238532Z Receiving objects: 81% (25115/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6251877Z Receiving objects: 82% (25425/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6301305Z Receiving objects: 83% (25735/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6450066Z Receiving objects: 84% (26046/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6455878Z Receiving objects: 85% (26356/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6464595Z Receiving objects: 86% (26666/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6469799Z Receiving objects: 87% (26976/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6477845Z Receiving objects: 88% (27286/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6483861Z Receiving objects: 89% (27596/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6599001Z Receiving objects: 90% (27906/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6613287Z Receiving objects: 91% (28216/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6625467Z Receiving objects: 92% (28526/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6642423Z Receiving objects: 93% (28836/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6677224Z Receiving objects: 94% (29146/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6688603Z Receiving objects: 95% (29456/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6696594Z Receiving objects: 96% (29766/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6701820Z Receiving objects: 97% (30076/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6773743Z Receiving objects: 98% (30386/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6797057Z Receiving objects: 99% (30696/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6798099Z remote: Total 31006 (delta 1013), reused 1476 (delta 823), pack-reused 29260 (from 1) 2024-08-20T22:29:03.6814814Z Receiving objects: 100% (31006/31006), 208.31 MiB | 69.46 MiB/s 2024-08-20T22:29:03.6815518Z Receiving objects: 100% (31006/31006), 232.50 MiB | 68.43 MiB/s, done. 2024-08-20T22:29:03.6847739Z Resolving deltas: 0% (0/16726) 2024-08-20T22:29:03.6860783Z Resolving deltas: 1% (169/16726) 2024-08-20T22:29:03.6871835Z Resolving deltas: 2% (394/16726) 2024-08-20T22:29:03.6893902Z Resolving deltas: 3% (521/16726) 2024-08-20T22:29:03.6909228Z Resolving deltas: 4% (672/16726) 2024-08-20T22:29:03.6919114Z Resolving deltas: 5% (900/16726) 2024-08-20T22:29:03.6929043Z Resolving deltas: 6% (1021/16726) 2024-08-20T22:29:03.6940215Z Resolving deltas: 7% (1172/16726) 2024-08-20T22:29:03.6949884Z Resolving deltas: 8% (1347/16726) 2024-08-20T22:29:03.6962498Z Resolving deltas: 9% (1511/16726) 2024-08-20T22:29:03.6973097Z Resolving deltas: 10% (1677/16726) 2024-08-20T22:29:03.6985720Z Resolving deltas: 11% (1843/16726) 2024-08-20T22:29:03.6997837Z Resolving deltas: 12% (2010/16726) 2024-08-20T22:29:03.7011459Z Resolving deltas: 13% (2175/16726) 2024-08-20T22:29:03.7024164Z Resolving deltas: 14% (2404/16726) 2024-08-20T22:29:03.7039999Z Resolving deltas: 15% (2511/16726) 2024-08-20T22:29:03.7056939Z Resolving deltas: 16% (2683/16726) 2024-08-20T22:29:03.7068655Z Resolving deltas: 17% (2865/16726) 2024-08-20T22:29:03.7118883Z Resolving deltas: 18% (3017/16726) 2024-08-20T22:29:03.7139766Z Resolving deltas: 19% (3199/16726) 2024-08-20T22:29:03.7140569Z Resolving deltas: 20% (3497/16726) 2024-08-20T22:29:03.7150822Z Resolving deltas: 21% (3521/16726) 2024-08-20T22:29:03.7167941Z Resolving deltas: 22% (3685/16726) 2024-08-20T22:29:03.7170089Z Resolving deltas: 23% (3977/16726) 2024-08-20T22:29:03.7181337Z Resolving deltas: 24% (4020/16726) 2024-08-20T22:29:03.7221910Z Resolving deltas: 25% (4185/16726) 2024-08-20T22:29:03.7238403Z Resolving deltas: 26% (4365/16726) 2024-08-20T22:29:03.7252615Z Resolving deltas: 27% (4529/16726) 2024-08-20T22:29:03.7267521Z Resolving deltas: 28% (4691/16726) 2024-08-20T22:29:03.7283639Z Resolving deltas: 29% (4852/16726) 2024-08-20T22:29:03.7296602Z Resolving deltas: 30% (5036/16726) 2024-08-20T22:29:03.7306159Z Resolving deltas: 31% (5191/16726) 2024-08-20T22:29:03.7315913Z Resolving deltas: 32% (5353/16726) 2024-08-20T22:29:03.7330826Z Resolving deltas: 33% (5522/16726) 2024-08-20T22:29:03.7351978Z Resolving deltas: 34% (5697/16726) 2024-08-20T22:29:03.7367922Z Resolving deltas: 35% (5865/16726) 2024-08-20T22:29:03.7381103Z Resolving deltas: 36% (6033/16726) 2024-08-20T22:29:03.7391958Z Resolving deltas: 37% (6227/16726) 2024-08-20T22:29:03.7408319Z Resolving deltas: 38% (6358/16726) 2024-08-20T22:29:03.7419072Z Resolving deltas: 39% (6525/16726) 2024-08-20T22:29:03.7435805Z Resolving deltas: 40% (6693/16726) 2024-08-20T22:29:03.7447644Z Resolving deltas: 41% (6886/16726) 2024-08-20T22:29:03.7455063Z Resolving deltas: 42% (7029/16726) 2024-08-20T22:29:03.7464414Z Resolving deltas: 43% (7219/16726) 2024-08-20T22:29:03.7481407Z Resolving deltas: 44% (7367/16726) 2024-08-20T22:29:03.7503147Z Resolving deltas: 45% (7539/16726) 2024-08-20T22:29:03.7514607Z Resolving deltas: 46% (7697/16726) 2024-08-20T22:29:03.7538793Z Resolving deltas: 47% (7866/16726) 2024-08-20T22:29:03.7548386Z Resolving deltas: 48% (8029/16726) 2024-08-20T22:29:03.7558951Z Resolving deltas: 49% (8198/16726) 2024-08-20T22:29:03.7570390Z Resolving deltas: 50% (8366/16726) 2024-08-20T22:29:03.7602968Z Resolving deltas: 51% (8542/16726) 2024-08-20T22:29:03.7609356Z Resolving deltas: 53% (8887/16726) 2024-08-20T22:29:03.7618746Z Resolving deltas: 54% (9037/16726) 2024-08-20T22:29:03.7635096Z Resolving deltas: 55% (9202/16726) 2024-08-20T22:29:03.7644430Z Resolving deltas: 56% (9374/16726) 2024-08-20T22:29:03.7654989Z Resolving deltas: 57% (9572/16726) 2024-08-20T22:29:03.7659230Z Resolving deltas: 58% (9799/16726) 2024-08-20T22:29:03.7669687Z Resolving deltas: 59% (9889/16726) 2024-08-20T22:29:03.7682306Z Resolving deltas: 60% (10072/16726) 2024-08-20T22:29:03.7695400Z Resolving deltas: 61% (10281/16726) 2024-08-20T22:29:03.7698585Z Resolving deltas: 62% (10478/16726) 2024-08-20T22:29:03.7714312Z Resolving deltas: 63% (10538/16726) 2024-08-20T22:29:03.7726822Z Resolving deltas: 64% (10738/16726) 2024-08-20T22:29:03.7746237Z Resolving deltas: 65% (10935/16726) 2024-08-20T22:29:03.7746805Z Resolving deltas: 66% (11203/16726) 2024-08-20T22:29:03.7769574Z Resolving deltas: 67% (11208/16726) 2024-08-20T22:29:03.7779306Z Resolving deltas: 68% (11407/16726) 2024-08-20T22:29:03.7791005Z Resolving deltas: 69% (11588/16726) 2024-08-20T22:29:03.7805400Z Resolving deltas: 70% (11750/16726) 2024-08-20T22:29:03.7818412Z Resolving deltas: 71% (11876/16726) 2024-08-20T22:29:03.7828197Z Resolving deltas: 72% (12045/16726) 2024-08-20T22:29:03.7836296Z Resolving deltas: 73% (12214/16726) 2024-08-20T22:29:03.7851309Z Resolving deltas: 74% (12379/16726) 2024-08-20T22:29:03.7856393Z Resolving deltas: 75% (12641/16726) 2024-08-20T22:29:03.7862868Z Resolving deltas: 76% (12737/16726) 2024-08-20T22:29:03.7870808Z Resolving deltas: 77% (12897/16726) 2024-08-20T22:29:03.7882018Z Resolving deltas: 78% (13048/16726) 2024-08-20T22:29:03.7891055Z Resolving deltas: 79% (13216/16726) 2024-08-20T22:29:03.7900308Z Resolving deltas: 80% (13382/16726) 2024-08-20T22:29:03.7909670Z Resolving deltas: 81% (13570/16726) 2024-08-20T22:29:03.7918149Z Resolving deltas: 82% (13739/16726) 2024-08-20T22:29:03.7927593Z Resolving deltas: 83% (13883/16726) 2024-08-20T22:29:03.7955717Z Resolving deltas: 84% (14075/16726) 2024-08-20T22:29:03.7956158Z Resolving deltas: 85% (14249/16726) 2024-08-20T22:29:03.7956586Z Resolving deltas: 86% (14391/16726) 2024-08-20T22:29:03.7959513Z Resolving deltas: 87% (14552/16726) 2024-08-20T22:29:03.7970032Z Resolving deltas: 88% (14722/16726) 2024-08-20T22:29:03.7976375Z Resolving deltas: 89% (14904/16726) 2024-08-20T22:29:03.7983903Z Resolving deltas: 90% (15070/16726) 2024-08-20T22:29:03.7992938Z Resolving deltas: 91% (15222/16726) 2024-08-20T22:29:03.7999775Z Resolving deltas: 92% (15392/16726) 2024-08-20T22:29:03.8008865Z Resolving deltas: 93% (15559/16726) 2024-08-20T22:29:03.8017828Z Resolving deltas: 94% (15728/16726) 2024-08-20T22:29:03.8026620Z Resolving deltas: 95% (15916/16726) 2024-08-20T22:29:03.8035694Z Resolving deltas: 96% (16079/16726) 2024-08-20T22:29:03.8045001Z Resolving deltas: 97% (16260/16726) 2024-08-20T22:29:03.8089685Z Resolving deltas: 98% (16393/16726) 2024-08-20T22:29:03.8163797Z Resolving deltas: 99% (16562/16726) 2024-08-20T22:29:03.8164288Z Resolving deltas: 100% (16726/16726) 2024-08-20T22:29:03.8165054Z Resolving deltas: 100% (16726/16726), done. 2024-08-20T22:29:04.8655797Z + pushd torchbench 2024-08-20T22:29:04.8656324Z ~/workspace/torchbench ~/workspace 2024-08-20T22:29:04.8656961Z + git checkout 23512dbebd44a11eb84afbf53c3c071dd105297e 2024-08-20T22:29:04.9441206Z Note: switching to '23512dbebd44a11eb84afbf53c3c071dd105297e'. 2024-08-20T22:29:04.9441779Z 2024-08-20T22:29:04.9442329Z You are in 'detached HEAD' state. You can look around, make experimental 2024-08-20T22:29:04.9443303Z changes and commit them, and you can discard any commits you make in this 2024-08-20T22:29:04.9444504Z state without impacting any branches by switching back to a branch. 2024-08-20T22:29:04.9444951Z 2024-08-20T22:29:04.9445274Z If you want to create a new branch to retain commits you create, you may 2024-08-20T22:29:04.9446266Z do so (now or later) by using -c with the switch command. Example: 2024-08-20T22:29:04.9446701Z 2024-08-20T22:29:04.9446914Z git switch -c 2024-08-20T22:29:04.9447186Z 2024-08-20T22:29:04.9447357Z Or undo this operation with: 2024-08-20T22:29:04.9447610Z 2024-08-20T22:29:04.9447736Z git switch - 2024-08-20T22:29:04.9447923Z 2024-08-20T22:29:04.9448285Z Turn off this advice by setting config variable advice.detachedHead to false 2024-08-20T22:29:04.9448829Z 2024-08-20T22:29:04.9448999Z HEAD is now at 23512dbe Pin numpy version 2024-08-20T22:29:04.9452492Z + '[' '' ']' 2024-08-20T22:29:04.9453001Z + python install.py --continue_on_fail 2024-08-20T22:29:07.2240329Z checking packages torch, torchvision, torchaudio are installed...OK 2024-08-20T22:29:25.6229226Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch...OK 2024-08-20T22:29:28.7803493Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/Background_Matting...OK 2024-08-20T22:29:30.6775477Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/LearningToPaint...OK 2024-08-20T22:29:32.4469267Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/Super_SloMo...OK 2024-08-20T22:29:32.4707478Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/alexnet...OK 2024-08-20T22:29:40.1048645Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_edgecnn...OK 2024-08-20T22:29:42.8127567Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_gcn...OK 2024-08-20T22:29:45.5294026Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_gin...OK 2024-08-20T22:29:48.2513448Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_sage...OK 2024-08-20T22:29:48.2517306Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/cm3leon_generate...SKIP - No install.py is found 2024-08-20T22:29:49.2892059Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/dcgan...OK 2024-08-20T22:29:52.1119306Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/demucs...OK 2024-08-20T22:29:52.1357277Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/densenet121...OK 2024-08-20T22:30:48.8320518Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_c4...OK 2024-08-20T22:30:57.2173103Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_dc5...OK 2024-08-20T22:31:04.5319146Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_fpn...OK 2024-08-20T22:31:11.3758950Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_c4...OK 2024-08-20T22:31:20.0024120Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_dc5...OK 2024-08-20T22:31:26.9180372Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_fpn...OK 2024-08-20T22:31:33.8943979Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fcos_r_50_fpn...OK 2024-08-20T22:31:41.0265688Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn...OK 2024-08-20T22:31:48.1435288Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_101_c4...OK 2024-08-20T22:31:55.6979185Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_101_fpn...OK 2024-08-20T22:32:02.4564697Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_50_c4...OK 2024-08-20T22:32:09.4300581Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_50_fpn...OK 2024-08-20T22:32:10.6356161Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/dlrm...OK 2024-08-20T22:32:46.3665833Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/doctr_det_predictor...OK 2024-08-20T22:32:51.2346869Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/doctr_reco_predictor...OK 2024-08-20T22:32:56.4244767Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/drq...OK 2024-08-20T22:33:07.8405653Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/fastNLP_Bert...OK 2024-08-20T22:33:09.0135459Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/functorch_dp_cifar10...OK 2024-08-20T22:33:10.4718918Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/functorch_maml_omniglot...OK 2024-08-20T22:33:17.0764098Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Albert...OK 2024-08-20T22:33:24.1168453Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bart...OK 2024-08-20T22:33:29.5916471Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert...OK 2024-08-20T22:33:37.2716812Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert_large...OK 2024-08-20T22:33:43.1599674Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_BigBird...OK 2024-08-20T22:33:48.1572834Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_DistilBert...OK 2024-08-20T22:33:54.4129372Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_GPT2...OK 2024-08-20T22:34:11.1674478Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_GPT2_large...OK 2024-08-20T22:34:17.3641476Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Longformer...OK 2024-08-20T22:34:21.2856149Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Reformer...OK 2024-08-20T22:34:26.0433045Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5...OK 2024-08-20T22:34:32.8599052Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_base...OK 2024-08-20T22:34:32.8601133Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_generate...SKIP - No install.py is found 2024-08-20T22:34:45.9297239Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_large...OK 2024-08-20T22:34:49.8498125Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Whisper...OK 2024-08-20T22:34:49.8501246Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_clip...SKIP - No install.py is found 2024-08-20T22:34:57.4747387Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_distil_whisper...OK 2024-08-20T22:34:58.6969105Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/lennard_jones...OK 2024-08-20T22:34:59.9401111Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama...OK 2024-08-20T22:35:45.0906506Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama_v2_7b_16h...OK 2024-08-20T22:37:14.2488513Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llava...OK 2024-08-20T22:37:14.2796191Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/maml...OK 2024-08-20T22:37:15.8211498Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/maml_omniglot...OK 2024-08-20T22:37:15.8458967Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mnasnet1_0...OK 2024-08-20T22:37:15.8696313Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v2...OK 2024-08-20T22:37:15.8941502Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v2_quantized_qat...OK 2024-08-20T22:37:15.9177241Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v3_large...OK 2024-08-20T22:37:15.9423332Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moco...OK 2024-08-20T22:37:37.1244391Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moondream...OK 2024-08-20T22:37:37.1246910Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nanogpt...SKIP - No install.py is found 2024-08-20T22:37:38.3692814Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nvidia_deeprecommender...OK 2024-08-20T22:37:40.7805203Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/opacus_cifar10...OK 2024-08-20T22:37:40.8048468Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/phlippe_densenet...OK 2024-08-20T22:37:40.8285059Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/phlippe_resnet...OK 2024-08-20T22:37:40.8533840Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_equation_of_state...OK 2024-08-20T22:37:40.8777531Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_isoneutral_mixing...OK 2024-08-20T22:37:40.9027578Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_turbulent_kinetic_energy...OK 2024-08-20T22:37:47.5414627Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_CycleGAN_and_pix2pix...OK 2024-08-20T22:37:48.9740518Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_stargan...OK 2024-08-20T22:37:52.4490175Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_unet...OK 2024-08-20T22:37:52.4735276Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet152...OK 2024-08-20T22:37:52.4979388Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet18...OK 2024-08-20T22:37:52.5221500Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet50...OK 2024-08-20T22:37:52.5462652Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet50_quantized_qat...OK 2024-08-20T22:37:52.5702861Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnext50_32x4d...OK 2024-08-20T22:38:07.8106995Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/sam...OK 2024-08-20T22:38:24.3359346Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/sam_fast...OK 2024-08-20T22:38:24.3600797Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/shufflenet_v2_x1_0...OK 2024-08-20T22:38:24.3602984Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/simple_gpt...SKIP - No install.py is found 2024-08-20T22:38:24.3605228Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/simple_gpt_tp_manual...SKIP - No install.py is found 2024-08-20T22:38:26.8078605Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/soft_actor_critic...OK 2024-08-20T22:38:28.5569054Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/speech_transformer...OK 2024-08-20T22:38:28.5817390Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/squeezenet1_1...OK 2024-08-20T22:38:53.0064913Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/stable_diffusion_text_encoder...OK 2024-08-20T22:38:59.6487177Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/stable_diffusion_unet...OK 2024-08-20T22:39:04.1740890Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tacotron2...OK 2024-08-20T22:39:11.0776121Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_efficientdet...OK 2024-08-20T22:39:11.1029517Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_efficientnet...OK 2024-08-20T22:39:11.1271562Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_nfnet...OK 2024-08-20T22:39:11.1513599Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_regnet...OK 2024-08-20T22:39:11.1755750Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_resnest...OK 2024-08-20T22:39:11.1998244Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer...OK 2024-08-20T22:39:11.2246388Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer_large...OK 2024-08-20T22:39:11.2486269Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vovnet...OK 2024-08-20T22:39:20.7503440Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/torch_multimodal_clip...OK 2024-08-20T22:39:32.6103675Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tts_angular...OK 2024-08-20T22:39:32.6357490Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/vgg16...OK 2024-08-20T22:39:34.7059144Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/vision_maskrcnn...OK 2024-08-20T22:39:36.6777112Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/yolov3...OK 2024-08-20T22:39:37.1573509Z + echo 'Print all dependencies after TorchBench is installed' 2024-08-20T22:39:37.1574240Z Print all dependencies after TorchBench is installed 2024-08-20T22:39:37.1574821Z + python -mpip freeze 2024-08-20T22:39:37.6659546Z absl-py==2.1.0 2024-08-20T22:39:37.6660039Z accelerate==0.33.0 2024-08-20T22:39:37.6660448Z aiohappyeyeballs==2.4.0 2024-08-20T22:39:37.6660801Z aiohttp==3.10.5 2024-08-20T22:39:37.6661099Z aiosignal==1.3.1 2024-08-20T22:39:37.6670818Z annotated-types==0.7.0 2024-08-20T22:39:37.6671369Z antlr4-python3-runtime==4.9.3 2024-08-20T22:39:37.6671880Z anyascii==0.3.2 2024-08-20T22:39:37.6672278Z astroid==3.2.4 2024-08-20T22:39:37.6672706Z astunparse==1.6.3 2024-08-20T22:39:37.6673201Z async-timeout==4.0.3 2024-08-20T22:39:37.6673575Z attrs==23.1.0 2024-08-20T22:39:37.6673973Z audioread==3.0.1 2024-08-20T22:39:37.6674325Z babel==2.16.0 2024-08-20T22:39:37.6674730Z beautifulsoup4==4.12.3 2024-08-20T22:39:37.6675983Z -e git+https://github.com/pytorch/benchmark@23512dbebd44a11eb84afbf53c3c071dd105297e#egg=bert_pytorch&subdirectory=torchbenchmark/models/BERT_pytorch 2024-08-20T22:39:37.6677221Z bibtexparser==2.0.0b7 2024-08-20T22:39:37.6677566Z black==24.8.0 2024-08-20T22:39:37.6677949Z blinker==1.8.2 2024-08-20T22:39:37.6678293Z blis==0.7.11 2024-08-20T22:39:37.6678583Z blobfile==2.1.1 2024-08-20T22:39:37.6678960Z bokeh==3.5.1 2024-08-20T22:39:37.6679328Z boto3==1.19.12 2024-08-20T22:39:37.6679665Z botocore==1.22.12 2024-08-20T22:39:37.6680047Z bs4==0.0.2 2024-08-20T22:39:37.6680425Z cachetools==5.5.0 2024-08-20T22:39:37.6680816Z cardboardlint==1.3.1 2024-08-20T22:39:37.6681218Z catalogue==2.0.10 2024-08-20T22:39:37.6681527Z certifi==2024.7.4 2024-08-20T22:39:37.6682161Z cffi==1.17.0 2024-08-20T22:39:37.6682535Z charset-normalizer==3.3.2 2024-08-20T22:39:37.6682893Z click==8.1.7 2024-08-20T22:39:37.6683275Z clldutils==3.22.2 2024-08-20T22:39:37.6683769Z cloudpathlib==0.18.1 2024-08-20T22:39:37.6684167Z cloudpickle==3.0.0 2024-08-20T22:39:37.6684577Z colorama==0.4.6 2024-08-20T22:39:37.6684964Z colorlog==6.8.2 2024-08-20T22:39:37.6685342Z confection==0.1.5 2024-08-20T22:39:37.6685741Z contourpy==1.2.1 2024-08-20T22:39:37.6686133Z coremltools==5.0b5 2024-08-20T22:39:37.6686818Z cryptography==43.0.0 2024-08-20T22:39:37.6687141Z csvw==3.3.0 2024-08-20T22:39:37.6687417Z cycler==0.12.1 2024-08-20T22:39:37.6687705Z cymem==2.0.8 2024-08-20T22:39:37.6688035Z DALL-E==0.1 2024-08-20T22:39:37.6688319Z datasets==2.21.0 2024-08-20T22:39:37.6688625Z decorator==5.1.1 2024-08-20T22:39:37.6688928Z defusedxml==0.7.1 2024-08-20T22:39:37.6689226Z Deprecated==1.2.14 2024-08-20T22:39:37.6689947Z detectron2 @ git+https://github.com/facebookresearch/detectron2.git@1a4df4d954089534daebbd7a0a45f08ece87056e 2024-08-20T22:39:37.6690725Z diffusers==0.20.2 2024-08-20T22:39:37.6691029Z dill==0.3.8 2024-08-20T22:39:37.6691312Z diskcache==5.6.3 2024-08-20T22:39:37.6691613Z distro==1.9.0 2024-08-20T22:39:37.6691899Z dlinfo==1.2.1 2024-08-20T22:39:37.6692236Z docker-pycreds==0.4.0 2024-08-20T22:39:37.6692564Z dominate==2.9.1 2024-08-20T22:39:37.6693363Z effdet @ git+https://github.com/rwightman/efficientdet-pytorch.git@d43c9e34cd62d22b4205831bb735f6dd83b8e881 2024-08-20T22:39:37.6694154Z einops==0.8.0 2024-08-20T22:39:37.6694447Z exceptiongroup==1.2.2 2024-08-20T22:39:37.6694770Z execnet==2.1.1 2024-08-20T22:39:37.6695068Z expecttest==0.1.6 2024-08-20T22:39:37.6695366Z FastNLP==0.6.0 2024-08-20T22:39:37.6695706Z ffmpeg-python==0.2.0 2024-08-20T22:39:37.6696030Z filelock==3.13.1 2024-08-20T22:39:37.6696328Z Flask==3.0.3 2024-08-20T22:39:37.6696610Z flatbuffers==2.0 2024-08-20T22:39:37.6696911Z fonttools==4.53.1 2024-08-20T22:39:37.6697219Z frozenlist==1.4.1 2024-08-20T22:39:37.6697518Z fsspec==2024.6.1 2024-08-20T22:39:37.6697812Z ftfy==6.2.3 2024-08-20T22:39:37.6698420Z functorch @ git+https://github.com/pytorch/functorch.git@b71aa0b4387b86c278132209b99538be48ef4c74 2024-08-20T22:39:37.6699125Z future==1.0.0 2024-08-20T22:39:37.6699417Z fvcore==0.1.5.post20221221 2024-08-20T22:39:37.6699767Z gdown==5.2.0 2024-08-20T22:39:37.6700052Z geojson==2.5.0 2024-08-20T22:39:37.6700348Z ghstack==0.8.0 2024-08-20T22:39:37.6700630Z gitdb==4.0.11 2024-08-20T22:39:37.6700917Z GitPython==3.1.43 2024-08-20T22:39:37.6701274Z google-auth==2.34.0 2024-08-20T22:39:37.6701745Z google-auth-oauthlib==1.0.0 2024-08-20T22:39:37.6702102Z grpcio==1.65.5 2024-08-20T22:39:37.6702382Z gym==0.26.2 2024-08-20T22:39:37.6702697Z gym-notices==0.0.8 2024-08-20T22:39:37.6702999Z h5py==3.11.0 2024-08-20T22:39:37.6703281Z higher==0.2.1 2024-08-20T22:39:37.6703611Z huggingface-hub==0.24.6 2024-08-20T22:39:37.6703981Z hydra-core==1.3.2 2024-08-20T22:39:37.6704287Z hypothesis==5.35.1 2024-08-20T22:39:37.6704582Z idna==3.7 2024-08-20T22:39:37.6704863Z imageio==2.35.1 2024-08-20T22:39:37.6705176Z importlib_metadata==8.4.0 2024-08-20T22:39:37.6705520Z inflect==7.3.1 2024-08-20T22:39:37.6705804Z iniconfig==2.0.0 2024-08-20T22:39:37.6706103Z iopath==0.1.9 2024-08-20T22:39:37.6706391Z isodate==0.6.1 2024-08-20T22:39:37.6706673Z isort==5.13.2 2024-08-20T22:39:37.6706970Z itsdangerous==2.2.0 2024-08-20T22:39:37.6707277Z Jinja2==3.1.4 2024-08-20T22:39:37.6707565Z jmespath==0.10.0 2024-08-20T22:39:37.6707856Z joblib==1.4.2 2024-08-20T22:39:37.6708154Z jsonpatch==1.33 2024-08-20T22:39:37.6708456Z jsonpointer==3.0.0 2024-08-20T22:39:37.6708757Z jsonschema==4.23.0 2024-08-20T22:39:37.6709152Z jsonschema-specifications==2023.12.1 2024-08-20T22:39:37.6709563Z junitparser==2.1.1 2024-08-20T22:39:37.6709906Z kaldi-io==0.9.8 2024-08-20T22:39:37.6710194Z kiwisolver==1.4.5 2024-08-20T22:39:37.6710497Z kornia==0.7.3 2024-08-20T22:39:37.6710793Z kornia_rs==0.1.5 2024-08-20T22:39:37.6711082Z lameenc==1.7.0 2024-08-20T22:39:37.6711375Z langcodes==3.4.0 2024-08-20T22:39:37.6711860Z langdetect==1.0.9 2024-08-20T22:39:37.6712214Z language-tags==1.2.0 2024-08-20T22:39:37.6712530Z language_data==1.2.0 2024-08-20T22:39:37.6712846Z lark==0.12.0 2024-08-20T22:39:37.6713129Z lazy_loader==0.4 2024-08-20T22:39:37.6713418Z libcst==1.4.0 2024-08-20T22:39:37.6713706Z librosa==0.9.2 2024-08-20T22:39:37.6713998Z lintrunner==0.12.5 2024-08-20T22:39:37.6714303Z llvmlite==0.38.1 2024-08-20T22:39:37.6714591Z lxml==4.9.4 2024-08-20T22:39:37.6714906Z marisa-trie==1.2.0 2024-08-20T22:39:37.6715321Z Markdown==3.7 2024-08-20T22:39:37.6715655Z markdown-it-py==3.0.0 2024-08-20T22:39:37.6715977Z MarkupSafe==2.1.5 2024-08-20T22:39:37.6716285Z matplotlib==3.8.4 2024-08-20T22:39:37.6716586Z mccabe==0.7.0 2024-08-20T22:39:37.6716866Z mdurl==0.1.2 2024-08-20T22:39:37.6717186Z ml-dtypes==0.4.0 2024-08-20T22:39:37.6717489Z MonkeyType==23.3.0 2024-08-20T22:39:37.6717842Z more-itertools==10.4.0 2024-08-20T22:39:37.6718207Z mpmath==1.3.0 2024-08-20T22:39:37.6718496Z msgpack==1.0.8 2024-08-20T22:39:37.6718795Z multidict==6.0.5 2024-08-20T22:39:37.6719093Z multiprocess==0.70.16 2024-08-20T22:39:37.6719428Z murmurhash==1.0.10 2024-08-20T22:39:37.6719739Z musdb==0.4.2 2024-08-20T22:39:37.6720021Z museval==0.4.1 2024-08-20T22:39:37.6720303Z mypy==1.10.0 2024-08-20T22:39:37.6720635Z mypy-extensions==1.0.0 2024-08-20T22:39:37.6720967Z networkx==2.8.8 2024-08-20T22:39:37.6721251Z ninja==1.10.2 2024-08-20T22:39:37.6721538Z nose==1.3.7 2024-08-20T22:39:37.6721811Z numba==0.55.2 2024-08-20T22:39:37.6722101Z numpy==1.21.2 2024-08-20T22:39:37.6722442Z nvidia-cublas-cu12==12.1.3.1 2024-08-20T22:39:37.6722870Z nvidia-cuda-cupti-cu12==12.1.105 2024-08-20T22:39:37.6723318Z nvidia-cuda-nvrtc-cu12==12.1.105 2024-08-20T22:39:37.6723917Z nvidia-cuda-runtime-cu12==12.1.105 2024-08-20T22:39:37.6724361Z nvidia-cudnn-cu12==9.1.0.70 2024-08-20T22:39:37.6724767Z nvidia-cufft-cu12==11.0.2.54 2024-08-20T22:39:37.6725184Z nvidia-curand-cu12==10.3.2.106 2024-08-20T22:39:37.6725605Z nvidia-cusolver-cu12==11.4.5.107 2024-08-20T22:39:37.6726051Z nvidia-cusparse-cu12==12.1.0.106 2024-08-20T22:39:37.6726476Z nvidia-nccl-cu12==2.20.5 2024-08-20T22:39:37.6726875Z nvidia-nvjitlink-cu12==12.6.20 2024-08-20T22:39:37.6727288Z nvidia-nvtx-cu12==12.1.105 2024-08-20T22:39:37.6727635Z oauthlib==3.2.2 2024-08-20T22:39:37.6727936Z omegaconf==2.3.0 2024-08-20T22:39:37.6728225Z onnx==1.16.1 2024-08-20T22:39:37.6728524Z onnxscript==0.1.0.dev20240817 2024-08-20T22:39:37.6728882Z opacus==1.5.2 2024-08-20T22:39:37.6729220Z opencv-python==4.8.0.74 2024-08-20T22:39:37.6729588Z opt-einsum==3.3.0 2024-08-20T22:39:37.6729891Z optree==0.12.1 2024-08-20T22:39:37.6730189Z packaging==24.1 2024-08-20T22:39:37.6730478Z pandas==2.0.3 2024-08-20T22:39:37.6730767Z patch==1.16 2024-08-20T22:39:37.6731050Z pathspec==0.12.1 2024-08-20T22:39:37.6731357Z phonemizer==3.3.0 2024-08-20T22:39:37.6731653Z pillow==10.3.0 2024-08-20T22:39:37.6731950Z platformdirs==4.2.2 2024-08-20T22:39:37.6732267Z pluggy==1.5.0 2024-08-20T22:39:37.6732546Z pooch==1.8.2 2024-08-20T22:39:37.6732853Z portalocker==2.10.1 2024-08-20T22:39:37.6733167Z preshed==3.0.9 2024-08-20T22:39:37.6733465Z prettytable==3.11.0 2024-08-20T22:39:37.6733767Z protobuf==3.20.2 2024-08-20T22:39:37.6734069Z psutil==6.0.0 2024-08-20T22:39:37.6734400Z py-cpuinfo==9.0.0 2024-08-20T22:39:37.6734700Z pyaml==24.7.0 2024-08-20T22:39:37.6734992Z pyarrow==17.0.0 2024-08-20T22:39:37.6735285Z pyasn1==0.6.0 2024-08-20T22:39:37.6735588Z pyasn1_modules==0.4.0 2024-08-20T22:39:37.6735915Z pyclipper==1.3.0.post5 2024-08-20T22:39:37.6736254Z pycocotools==2.0.8 2024-08-20T22:39:37.6736566Z pycparser==2.22 2024-08-20T22:39:37.6736867Z pycryptodomex==3.20.0 2024-08-20T22:39:37.6737196Z pydantic==2.8.2 2024-08-20T22:39:37.6737503Z pydantic_core==2.20.1 2024-08-20T22:39:37.6737827Z pydot==3.0.1 2024-08-20T22:39:37.6738109Z pygame==2.6.0 2024-08-20T22:39:37.6738404Z PyGithub==2.3.0 2024-08-20T22:39:37.6738702Z Pygments==2.15.0 2024-08-20T22:39:37.6739003Z PyJWT==2.9.0 2024-08-20T22:39:37.6739285Z pylatexenc==2.10 2024-08-20T22:39:37.6739587Z pylint==3.2.6 2024-08-20T22:39:37.6740037Z PyNaCl==1.5.0 2024-08-20T22:39:37.6740323Z pynvml==11.5.3 2024-08-20T22:39:37.6740621Z pyparsing==3.1.2 2024-08-20T22:39:37.6740926Z pypdfium2==4.30.0 2024-08-20T22:39:37.6741233Z pysbd==0.3.4 2024-08-20T22:39:37.6741518Z PySocks==1.7.1 2024-08-20T22:39:37.6741810Z pytest==7.3.2 2024-08-20T22:39:37.6742155Z pytest-benchmark==4.0.0 2024-08-20T22:39:37.6742525Z pytest-cpp==2.3.0 2024-08-20T22:39:37.6742892Z pytest-flakefinder==1.1.0 2024-08-20T22:39:37.6743300Z pytest-rerunfailures==14.0 2024-08-20T22:39:37.6743794Z pytest-xdist==3.3.1 2024-08-20T22:39:37.6744158Z python-dateutil==2.9.0.post0 2024-08-20T22:39:37.6744955Z python-doctr @ git+https://github.com/xuzhao9/doctr.git@7f8a3e61bd6480c4e7b7cde47af0aa2153309c56 2024-08-20T22:39:37.6746919Z pytorch-labs-segment-anything-fast @ git+https://github.com/pytorch-labs/segment-anything-fast.git@de861af9badd03e8b40f5e063d70754a3dc6b4f4 2024-08-20T22:39:37.6747904Z pytz==2024.1 2024-08-20T22:39:37.6748189Z PyWavelets==1.4.1 2024-08-20T22:39:37.6748620Z PyYAML @ file:///croot/pyyaml_1698096049011/work 2024-08-20T22:39:37.6749071Z rapidfuzz==3.9.6 2024-08-20T22:39:37.6749360Z rdflib==7.0.0 2024-08-20T22:39:37.6749646Z redis==5.0.8 2024-08-20T22:39:37.6749934Z referencing==0.35.1 2024-08-20T22:39:37.6750239Z regex==2024.7.24 2024-08-20T22:39:37.6750527Z requests==2.32.3 2024-08-20T22:39:37.6750882Z requests-oauthlib==2.0.0 2024-08-20T22:39:37.6751226Z resampy==0.4.3 2024-08-20T22:39:37.6751507Z rfc3986==1.5.0 2024-08-20T22:39:37.6751799Z rich==13.7.1 2024-08-20T22:39:37.6752081Z rockset==1.0.3 2024-08-20T22:39:37.6752405Z rpds-py==0.20.0 2024-08-20T22:39:37.6752687Z rsa==4.9 2024-08-20T22:39:37.6752959Z s3transfer==0.5.2 2024-08-20T22:39:37.6753270Z safetensors==0.4.4 2024-08-20T22:39:37.6753609Z scikit-image==0.21.0 2024-08-20T22:39:37.6753974Z scikit-learn==1.5.1 2024-08-20T22:39:37.6754287Z scipy==1.10.1 2024-08-20T22:39:37.6754574Z segments==2.2.1 2024-08-20T22:39:37.6754866Z sentencepiece==0.2.0 2024-08-20T22:39:37.6755222Z sentry-sdk==2.13.0 2024-08-20T22:39:37.6755537Z setproctitle==1.3.3 2024-08-20T22:39:37.6755844Z shapely==2.0.6 2024-08-20T22:39:37.6756130Z shellingham==1.5.4 2024-08-20T22:39:37.6756439Z simplejson==3.19.3 2024-08-20T22:39:37.6756832Z six @ file:///tmp/build/80754af9/six_1644875935023/work 2024-08-20T22:39:37.6757330Z smart-open==7.0.4 2024-08-20T22:39:37.6757628Z smmap==5.0.1 2024-08-20T22:39:37.6757932Z sortedcontainers==2.4.0 2024-08-20T22:39:37.6758277Z soundfile==0.12.1 2024-08-20T22:39:37.6758583Z soupsieve==2.6 2024-08-20T22:39:37.6758889Z soxr==0.4.0 2024-08-20T22:39:37.6759173Z spacy==3.7.6 2024-08-20T22:39:37.6759499Z spacy-legacy==3.0.12 2024-08-20T22:39:37.6759868Z spacy-loggers==1.0.5 2024-08-20T22:39:37.6760194Z srsly==2.4.8 2024-08-20T22:39:37.6760487Z stempeg==0.2.3 2024-08-20T22:39:37.6760781Z submitit==1.5.1 2024-08-20T22:39:37.6761083Z sympy==1.13.1 2024-08-20T22:39:37.6761378Z tabulate==0.9.0 2024-08-20T22:39:37.6761727Z tb-nightly==2.13.0a20230426 2024-08-20T22:39:37.6762088Z tensorboard==2.13.0 2024-08-20T22:39:37.6762476Z tensorboard-data-server==0.7.2 2024-08-20T22:39:37.6762862Z tensorboardX==2.6.2.2 2024-08-20T22:39:37.6763187Z termcolor==2.4.0 2024-08-20T22:39:37.6763625Z thinc==8.2.5 2024-08-20T22:39:37.6763930Z threadpoolctl==3.5.0 2024-08-20T22:39:37.6764254Z tifffile==2024.8.10 2024-08-20T22:39:37.6765074Z timm @ git+https://github.com/huggingface/pytorch-image-models.git@730b907b4d45a4713cbc425cbf224c46089fd514 2024-08-20T22:39:37.6765862Z tlparse==0.3.25 2024-08-20T22:39:37.6766176Z tokenizers==0.15.2 2024-08-20T22:39:37.6766479Z tomli==2.0.1 2024-08-20T22:39:37.6766771Z tomlkit==0.13.2 2024-08-20T22:39:37.6767986Z torch @ file:///var/lib/jenkins/workspace/dist/torch-2.5.0a0%2Bgit40ec5f6-cp310-cp310-linux_x86_64.whl#sha256=b444e33ace8c516878aca2862d707788a4c01d8c6190fb13e010fb9a131a57c5 2024-08-20T22:39:37.6769624Z torch_geometric @ git+https://github.com/pyg-team/pytorch_geometric.git@cabcd4097442ba60aa1efa11e1619dd9bb8fb527 2024-08-20T22:39:37.6771082Z torchao @ git+https://github.com/pytorch/ao.git@2c8e3f368613f896fde14cc24b9268c9e125575e 2024-08-20T22:39:37.6772083Z torchaudio @ git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:39:37.6773096Z torchdata @ git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:39:37.6774212Z torchmultimodal @ git+https://github.com/facebookresearch/multimodal.git@e4d288b45b89cee462a21ab264405f3f368adc21 2024-08-20T22:39:37.6775317Z torchtext @ git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:39:37.6776447Z torchvision @ git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:39:37.6777155Z tornado==6.4.1 2024-08-20T22:39:37.6777457Z tqdm==4.66.5 2024-08-20T22:39:37.6777759Z transformers==4.38.1 2024-08-20T22:39:37.6778132Z treetable==0.2.5 2024-08-20T22:39:37.6778947Z -e git+https://github.com/openai/triton@dedb7bdf339a3546896d4820366ca562c586bfa0#egg=triton&subdirectory=python 2024-08-20T22:39:37.6779744Z typeguard==4.3.0 2024-08-20T22:39:37.6780044Z typer==0.12.4 2024-08-20T22:39:37.6780357Z typing_extensions==4.12.2 2024-08-20T22:39:37.6780705Z tzdata==2024.1 2024-08-20T22:39:37.6781002Z Unidecode==1.3.8 2024-08-20T22:39:37.6781372Z unittest-xml-reporting==3.2.0 2024-08-20T22:39:37.6781755Z uritemplate==4.1.1 2024-08-20T22:39:37.6782070Z urllib3==1.26.19 2024-08-20T22:39:37.6782374Z visdom==0.2.4 2024-08-20T22:39:37.6782661Z wandb==0.17.7 2024-08-20T22:39:37.6782951Z wasabi==1.1.3 2024-08-20T22:39:37.6783247Z wcwidth==0.2.13 2024-08-20T22:39:37.6783544Z weasel==0.4.1 2024-08-20T22:39:37.6783887Z websocket-client==1.8.0 2024-08-20T22:39:37.6784233Z Werkzeug==3.0.3 2024-08-20T22:39:37.6784531Z wrapt==1.16.0 2024-08-20T22:39:37.6784819Z xdoctest==1.1.0 2024-08-20T22:39:37.6785118Z xxhash==3.5.0 2024-08-20T22:39:37.6785419Z xyzservices==2024.6.0 2024-08-20T22:39:37.6785741Z yacs==0.1.8 2024-08-20T22:39:37.6786027Z yarl==1.9.4 2024-08-20T22:39:37.6786353Z z3-solver==4.12.2.0 2024-08-20T22:39:37.6786670Z zipp==3.20.0 2024-08-20T22:39:37.6975113Z + popd 2024-08-20T22:39:37.6975558Z ~/workspace 2024-08-20T22:39:37.6975894Z + [[ aot_inductor_torchbench != *cpu* ]] 2024-08-20T22:39:37.6976330Z + install_torchrec_and_fbgemm 2024-08-20T22:39:37.6976710Z + local torchrec_commit 2024-08-20T22:39:37.6977538Z ++ get_pinned_commit torchrec 2024-08-20T22:39:37.6977947Z ++ cat .github/ci_commit_pins/torchrec.txt 2024-08-20T22:39:37.7027297Z + torchrec_commit=6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T22:39:37.7027834Z + local fbgemm_commit 2024-08-20T22:39:37.7030129Z ++ get_pinned_commit fbgemm 2024-08-20T22:39:37.7030531Z ++ cat .github/ci_commit_pins/fbgemm.txt 2024-08-20T22:39:37.7059211Z + fbgemm_commit=de731af65b4f04696e85c729e3282450b51b95fd 2024-08-20T22:39:37.7060154Z + pip_uninstall torchrec-nightly 2024-08-20T22:39:37.7060630Z + pip uninstall -y torchrec-nightly 2024-08-20T22:39:38.0829118Z WARNING: Skipping torchrec-nightly as it is not installed. 2024-08-20T22:39:38.1173026Z + pip_uninstall fbgemm-gpu-nightly 2024-08-20T22:39:38.1173623Z + pip uninstall -y fbgemm-gpu-nightly 2024-08-20T22:39:38.4850912Z WARNING: Skipping fbgemm-gpu-nightly as it is not installed. 2024-08-20T22:39:38.5207027Z + pip_install setuptools-git-versioning scikit-build pyre-extensions 2024-08-20T22:39:38.5208019Z + pip install --progress-bar off setuptools-git-versioning scikit-build pyre-extensions 2024-08-20T22:39:39.0076890Z Collecting setuptools-git-versioning 2024-08-20T22:39:39.0238297Z Downloading setuptools_git_versioning-2.0.0-py3-none-any.whl.metadata (5.8 kB) 2024-08-20T22:39:39.1402700Z Collecting scikit-build 2024-08-20T22:39:39.1432203Z Downloading scikit_build-0.18.0-py3-none-any.whl.metadata (17 kB) 2024-08-20T22:39:39.2595180Z Collecting pyre-extensions 2024-08-20T22:39:39.2626184Z Downloading pyre_extensions-0.0.30-py3-none-any.whl.metadata (4.0 kB) 2024-08-20T22:39:39.3131429Z Requirement already satisfied: packaging in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from setuptools-git-versioning) (24.1) 2024-08-20T22:39:39.3133949Z Requirement already satisfied: setuptools in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from setuptools-git-versioning) (72.1.0) 2024-08-20T22:39:39.3709503Z Collecting toml>=0.10.2 (from setuptools-git-versioning) 2024-08-20T22:39:39.3737745Z Downloading toml-0.10.2-py2.py3-none-any.whl.metadata (7.1 kB) 2024-08-20T22:39:39.4238619Z Requirement already satisfied: distro in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from scikit-build) (1.9.0) 2024-08-20T22:39:39.4241422Z Requirement already satisfied: tomli in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from scikit-build) (2.0.1) 2024-08-20T22:39:39.4245948Z Requirement already satisfied: wheel>=0.32.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from scikit-build) (0.43.0) 2024-08-20T22:39:39.4811444Z Collecting typing-inspect (from pyre-extensions) 2024-08-20T22:39:39.4840516Z Downloading typing_inspect-0.9.0-py3-none-any.whl.metadata (1.5 kB) 2024-08-20T22:39:39.5374614Z Requirement already satisfied: typing-extensions in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pyre-extensions) (4.12.2) 2024-08-20T22:39:39.5452520Z Requirement already satisfied: mypy-extensions>=0.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from typing-inspect->pyre-extensions) (1.0.0) 2024-08-20T22:39:39.5488590Z Downloading setuptools_git_versioning-2.0.0-py3-none-any.whl (10 kB) 2024-08-20T22:39:39.6031525Z Downloading scikit_build-0.18.0-py3-none-any.whl (85 kB) 2024-08-20T22:39:39.6527122Z Downloading pyre_extensions-0.0.30-py3-none-any.whl (12 kB) 2024-08-20T22:39:39.7063626Z Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) 2024-08-20T22:39:39.7792096Z Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB) 2024-08-20T22:39:40.6564356Z Installing collected packages: typing-inspect, toml, scikit-build, setuptools-git-versioning, pyre-extensions 2024-08-20T22:39:41.0125452Z Successfully installed pyre-extensions-0.0.30 scikit-build-0.18.0 setuptools-git-versioning-2.0.0 toml-0.10.2 typing-inspect-0.9.0 2024-08-20T22:39:41.1225212Z + CUDA_PATH=/usr/local/cuda-12.1 2024-08-20T22:39:41.1227490Z + pip_install --no-use-pep517 --user 'git+https://github.com/pytorch/FBGEMM.git@de731af65b4f04696e85c729e3282450b51b95fd#egg=fbgemm-gpu&subdirectory=fbgemm_gpu' 2024-08-20T22:39:41.1230495Z + pip install --progress-bar off --no-use-pep517 --user 'git+https://github.com/pytorch/FBGEMM.git@de731af65b4f04696e85c729e3282450b51b95fd#egg=fbgemm-gpu&subdirectory=fbgemm_gpu' 2024-08-20T22:39:41.4946359Z Collecting fbgemm-gpu 2024-08-20T22:39:41.4949779Z Cloning https://github.com/pytorch/FBGEMM.git (to revision de731af65b4f04696e85c729e3282450b51b95fd) to /tmp/pip-install-0474pm72/fbgemm-gpu_fe22bda0cff14ee39292f73cf2d71dc1 2024-08-20T22:39:41.4981450Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/FBGEMM.git /tmp/pip-install-0474pm72/fbgemm-gpu_fe22bda0cff14ee39292f73cf2d71dc1 2024-08-20T22:39:42.6494198Z Running command git rev-parse -q --verify 'sha^de731af65b4f04696e85c729e3282450b51b95fd' 2024-08-20T22:39:42.6523112Z Running command git fetch -q https://github.com/pytorch/FBGEMM.git de731af65b4f04696e85c729e3282450b51b95fd 2024-08-20T22:39:43.1502124Z Running command git checkout -q de731af65b4f04696e85c729e3282450b51b95fd 2024-08-20T22:39:43.6907333Z Resolved https://github.com/pytorch/FBGEMM.git to commit de731af65b4f04696e85c729e3282450b51b95fd 2024-08-20T22:39:43.6908633Z Running command git submodule update --init --recursive -q 2024-08-20T22:39:53.2272177Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:39:53.2291537Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbgemm-gpu) (1.21.2) 2024-08-20T22:39:53.2298483Z Building wheels for collected packages: fbgemm-gpu 2024-08-20T23:24:37.9114808Z Building wheel for fbgemm-gpu (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | done 2024-08-20T23:24:38.0950328Z [?25h Created wheel for fbgemm-gpu: filename=fbgemm_gpu-0.4.1rc0.post421-cp310-cp310-linux_x86_64.whl size=252619800 sha256=90dfdb95027fbeb7f931e2fee2f7bb7369173247a73b7ba108c4d423ab07ebc6 2024-08-20T23:24:38.0951997Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/d8/d9/f3/f5c260aba9c9f0a533444c12621a0c2ceb8fbcffce2b5beb39 2024-08-20T23:24:38.0971988Z Successfully built fbgemm-gpu 2024-08-20T23:24:38.9281919Z Installing collected packages: fbgemm-gpu 2024-08-20T23:24:42.2604333Z Successfully installed fbgemm-gpu-0.4.1rc0.post421 2024-08-20T23:24:43.1668265Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/torchrec.git@6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:24:43.1669814Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/torchrec.git@6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:24:43.5391161Z Collecting git+https://github.com/pytorch/torchrec.git@6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:24:43.5393921Z Cloning https://github.com/pytorch/torchrec.git (to revision 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2) to /tmp/pip-req-build-u3im_agk 2024-08-20T23:24:43.5422870Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/torchrec.git /tmp/pip-req-build-u3im_agk 2024-08-20T23:24:44.3379050Z Running command git rev-parse -q --verify 'sha^6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2' 2024-08-20T23:24:44.3406876Z Running command git fetch -q https://github.com/pytorch/torchrec.git 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:24:44.6599516Z Running command git checkout -q 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:24:45.0135616Z Resolved https://github.com/pytorch/torchrec.git to commit 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:24:45.2862013Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T23:24:45.2896105Z [?25hRequirement already satisfied: iopath in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (0.1.9) 2024-08-20T23:24:45.2898661Z Requirement already satisfied: pandas in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (2.0.3) 2024-08-20T23:24:45.2901753Z Requirement already satisfied: tabulate in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (0.9.0) 2024-08-20T23:24:45.3399377Z Collecting torchmetrics (from torchrec==0.3.2) 2024-08-20T23:24:45.3565311Z Downloading torchmetrics-1.4.1-py3-none-any.whl.metadata (20 kB) 2024-08-20T23:24:45.3955197Z Requirement already satisfied: tqdm in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (4.66.5) 2024-08-20T23:24:45.3956830Z Requirement already satisfied: fbgemm-gpu in /var/lib/jenkins/.local/lib/python3.10/site-packages (from torchrec==0.3.2) (0.4.1rc0.post421) 2024-08-20T23:24:45.3966624Z Requirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbgemm-gpu->torchrec==0.3.2) (1.21.2) 2024-08-20T23:24:45.3975993Z Requirement already satisfied: portalocker in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from iopath->torchrec==0.3.2) (2.10.1) 2024-08-20T23:24:45.4240838Z Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pandas->torchrec==0.3.2) (2.9.0.post0) 2024-08-20T23:24:45.4244209Z Requirement already satisfied: pytz>=2020.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pandas->torchrec==0.3.2) (2024.1) 2024-08-20T23:24:45.4247818Z Requirement already satisfied: tzdata>=2022.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pandas->torchrec==0.3.2) (2024.1) 2024-08-20T23:24:45.4269738Z Requirement already satisfied: packaging>17.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchmetrics->torchrec==0.3.2) (24.1) 2024-08-20T23:24:45.4273468Z Requirement already satisfied: torch>=1.10.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchmetrics->torchrec==0.3.2) (2.5.0a0+git40ec5f6) 2024-08-20T23:24:45.4427750Z Collecting lightning-utilities>=0.8.0 (from torchmetrics->torchrec==0.3.2) 2024-08-20T23:24:45.4454495Z Downloading lightning_utilities-0.11.6-py3-none-any.whl.metadata (5.2 kB) 2024-08-20T23:24:45.4548183Z Requirement already satisfied: setuptools in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from lightning-utilities>=0.8.0->torchmetrics->torchrec==0.3.2) (72.1.0) 2024-08-20T23:24:45.4552310Z Requirement already satisfied: typing-extensions in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from lightning-utilities>=0.8.0->torchmetrics->torchrec==0.3.2) (4.12.2) 2024-08-20T23:24:45.4572746Z Requirement already satisfied: six>=1.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->torchrec==0.3.2) (1.16.0) 2024-08-20T23:24:45.4605513Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (3.13.1) 2024-08-20T23:24:45.4609159Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (2.8.8) 2024-08-20T23:24:45.4612529Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (3.1.4) 2024-08-20T23:24:45.4615045Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (2024.6.1) 2024-08-20T23:24:45.4620424Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (1.13.1) 2024-08-20T23:24:45.4635457Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch>=1.10.0->torchmetrics->torchrec==0.3.2) (1.3.0) 2024-08-20T23:24:45.5045391Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch>=1.10.0->torchmetrics->torchrec==0.3.2) (2.1.5) 2024-08-20T23:24:45.5184898Z Downloading torchmetrics-1.4.1-py3-none-any.whl (866 kB) 2024-08-20T23:24:45.5402876Z Downloading lightning_utilities-0.11.6-py3-none-any.whl (26 kB) 2024-08-20T23:24:45.5506277Z Building wheels for collected packages: torchrec 2024-08-20T23:24:46.0653057Z Building wheel for torchrec (setup.py) ... [?25l- \ | done 2024-08-20T23:24:46.0661527Z [?25h Created wheel for torchrec: filename=torchrec-0.3.2-py3-none-any.whl size=374488 sha256=4c84425540ab2ccf4116dc15497456549f585d375f130530e0c77e68c809c876 2024-08-20T23:24:46.0663223Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/8e/a1/47/39ede01672ba82c08fb521bfc057cc4347e4b0e951738c8ca8 2024-08-20T23:24:46.0684381Z Successfully built torchrec 2024-08-20T23:24:46.8441384Z Installing collected packages: lightning-utilities, torchmetrics, torchrec 2024-08-20T23:24:47.4597204Z Successfully installed lightning-utilities-0.11.6 torchmetrics-1.4.1 torchrec-0.3.2 2024-08-20T23:24:47.5991025Z ++ pwd 2024-08-20T23:24:47.5995059Z + PYTHONPATH=/var/lib/jenkins/workspace/torchbench 2024-08-20T23:24:47.5995729Z + test_dynamo_benchmark torchbench 1 2024-08-20T23:24:47.6000042Z ++ pwd 2024-08-20T23:24:47.6004174Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T23:24:47.6004823Z + local suite=torchbench 2024-08-20T23:24:47.6005164Z + shift 2024-08-20T23:24:47.6005431Z + local shard_id=1 2024-08-20T23:24:47.6008111Z + shift 2024-08-20T23:24:47.6008925Z + [[ aot_inductor_torchbench == *perf_compare* ]] 2024-08-20T23:24:47.6009551Z + [[ aot_inductor_torchbench == *perf* ]] 2024-08-20T23:24:47.6010131Z + [[ aot_inductor_torchbench == *cpu* ]] 2024-08-20T23:24:47.6010942Z + [[ aot_inductor_torchbench == *aot_inductor* ]] 2024-08-20T23:24:47.6012125Z + test_single_dynamo_benchmark inference torchbench 1 --inference --bfloat16 2024-08-20T23:24:47.6012756Z ++ pwd 2024-08-20T23:24:47.6013591Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T23:24:47.6014323Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-08-20T23:24:47.6063943Z + local name=inference 2024-08-20T23:24:47.6064308Z + shift 2024-08-20T23:24:47.6064630Z + local suite=torchbench 2024-08-20T23:24:47.6065030Z + shift 2024-08-20T23:24:47.6065312Z + local shard_id=1 2024-08-20T23:24:47.6065620Z + shift 2024-08-20T23:24:47.6065893Z + partition_flags=() 2024-08-20T23:24:47.6066245Z + local partition_flags 2024-08-20T23:24:47.6066663Z + [[ -n 2 ]] 2024-08-20T23:24:47.6066978Z + [[ -n 1 ]] 2024-08-20T23:24:47.6067602Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-08-20T23:24:47.6068301Z + [[ aot_inductor_torchbench == *perf_compare* ]] 2024-08-20T23:24:47.6068811Z + [[ aot_inductor_torchbench == *perf* ]] 2024-08-20T23:24:47.6069293Z + [[ aot_inductor_torchbench == *aot_inductor* ]] 2024-08-20T23:24:47.6069818Z + [[ aot_inductor_torchbench != *cpu_aot_inductor* ]] 2024-08-20T23:24:47.6070404Z + export TORCHINDUCTOR_ABI_COMPATIBLE=1 2024-08-20T23:24:47.6070863Z + TORCHINDUCTOR_ABI_COMPATIBLE=1 2024-08-20T23:24:47.6071284Z + [[ aot_inductor_torchbench == *_avx2* ]] 2024-08-20T23:24:47.6071749Z + [[ aot_inductor_torchbench == *_avx512* ]] 2024-08-20T23:24:47.6073445Z + python benchmarks/dynamo/torchbench.py --ci --accuracy --timing --explain --export-aot-inductor --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 1 --output /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv 2024-08-20T23:24:53.5839356Z 2024-08-20T23:24:54.3964175Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:24:54.3964652Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:24:54.3997504Z cuda eval llama 2024-08-20T23:24:55.2196712Z ERROR:common: 2024-08-20T23:24:55.2197069Z Traceback (most recent call last): 2024-08-20T23:24:55.2197943Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T23:24:55.2198731Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T23:24:55.2199598Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T23:24:55.2200529Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T23:24:55.2201359Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T23:24:55.2202159Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T23:24:55.2203984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T23:24:55.2205226Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T23:24:55.2206417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T23:24:55.2207241Z result_traced = opt_f(*args, **kwargs) 2024-08-20T23:24:55.2208249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:24:55.2209145Z return self._call_impl(*args, **kwargs) 2024-08-20T23:24:55.2210297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:24:55.2211152Z return forward_call(*args, **kwargs) 2024-08-20T23:24:55.2212049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T23:24:55.2212837Z return fn(*args, **kwargs) 2024-08-20T23:24:55.2213784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:24:55.2214704Z return self._call_impl(*args, **kwargs) 2024-08-20T23:24:55.2215692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:24:55.2216545Z return forward_call(*args, **kwargs) 2024-08-20T23:24:55.2217486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T23:24:55.2218371Z return self._torchdynamo_orig_callable( 2024-08-20T23:24:55.2219329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T23:24:55.2220214Z return _compile( 2024-08-20T23:24:55.2221060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T23:24:55.2222021Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T23:24:55.2223110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T23:24:55.2224068Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T23:24:55.2225156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T23:24:55.2226012Z return function(*args, **kwargs) 2024-08-20T23:24:55.2226960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T23:24:55.2227899Z out_code = transform_code_object(code, transform) 2024-08-20T23:24:55.2229049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T23:24:55.2230060Z transformations(instructions, code_options) 2024-08-20T23:24:55.2231003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T23:24:55.2231818Z return fn(*args, **kwargs) 2024-08-20T23:24:55.2232719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T23:24:55.2233547Z tracer.run() 2024-08-20T23:24:55.2234376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T23:24:55.2235179Z super().run() 2024-08-20T23:24:55.2236005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:24:55.2236819Z while self.step(): 2024-08-20T23:24:55.2237670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:24:55.2238543Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:24:55.2239629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T23:24:55.2240482Z return inner_fn(self, inst) 2024-08-20T23:24:55.2241440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T23:24:55.2242325Z self.call_function(fn, args, {}) 2024-08-20T23:24:55.2243298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T23:24:55.2244550Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T23:24:55.2246101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 442, in call_function 2024-08-20T23:24:55.2247026Z return tx.inline_user_function_return( 2024-08-20T23:24:55.2248126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T23:24:55.2249277Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T23:24:55.2250437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T23:24:55.2251384Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T23:24:55.2252425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T23:24:55.2253274Z tracer.run() 2024-08-20T23:24:55.2254100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:24:55.2254919Z while self.step(): 2024-08-20T23:24:55.2255762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:24:55.2256640Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:24:55.2257632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T23:24:55.2258481Z return inner_fn(self, inst) 2024-08-20T23:24:55.2259446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1664, in CALL_FUNCTION_EX 2024-08-20T23:24:55.2260421Z self.call_function(fn, argsvars.items, kwargsvars) 2024-08-20T23:24:55.2261472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T23:24:55.2262603Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T23:24:55.2263776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T23:24:55.2264761Z return super().call_function(tx, args, kwargs) 2024-08-20T23:24:55.2265840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T23:24:55.2266786Z return super().call_function(tx, args, kwargs) 2024-08-20T23:24:55.2267846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T23:24:55.2268938Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T23:24:55.2270211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T23:24:55.2271366Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T23:24:55.2272561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T23:24:55.2273507Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T23:24:55.2274573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T23:24:55.2275445Z tracer.run() 2024-08-20T23:24:55.2276424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:24:55.2277333Z while self.step(): 2024-08-20T23:24:55.2278191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:24:55.2279088Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:24:55.2280098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T23:24:55.2281081Z return inner_fn(self, inst) 2024-08-20T23:24:55.2282046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T23:24:55.2282959Z self.call_function(fn, args, {}) 2024-08-20T23:24:55.2284007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T23:24:55.2285198Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T23:24:55.2286383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T23:24:55.2287347Z return super().call_function(tx, args, kwargs) 2024-08-20T23:24:55.2288426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T23:24:55.2289384Z return super().call_function(tx, args, kwargs) 2024-08-20T23:24:55.2290445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T23:24:55.2291543Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T23:24:55.2292836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T23:24:55.2294012Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T23:24:55.2295200Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T23:24:55.2296155Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T23:24:55.2297235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T23:24:55.2298129Z tracer.run() 2024-08-20T23:24:55.2298978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:24:55.2299797Z while self.step(): 2024-08-20T23:24:55.2300659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:24:55.2301557Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:24:55.2302603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1739, in STORE_ATTR 2024-08-20T23:24:55.2303438Z assert ( 2024-08-20T23:24:55.2303914Z AssertionError: Mutating module attribute cache_k during export. 2024-08-20T23:24:55.2304364Z 2024-08-20T23:24:55.2304490Z from user code: 2024-08-20T23:24:55.2305282Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama/model.py", line 238, in forward 2024-08-20T23:24:55.2306143Z h = layer(h, start_pos, freqs_cis, mask) 2024-08-20T23:24:55.2307136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:24:55.2308017Z return forward_call(*args, **kwargs) 2024-08-20T23:24:55.2308891Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama/model.py", line 192, in forward 2024-08-20T23:24:55.2309973Z h = x + self.attention.forward(self.attention_norm(x), start_pos, freqs_cis, mask) 2024-08-20T23:24:55.2311147Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama/model.py", line 121, in forward 2024-08-20T23:24:55.2311997Z self.cache_k = self.cache_k.to(xq) 2024-08-20T23:24:55.2312303Z 2024-08-20T23:24:55.2312622Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T23:24:55.2313086Z 2024-08-20T23:24:55.2313411Z TorchDynamo optimized model failed to run because of following error 2024-08-20T23:24:55.2313987Z fail_to_run 2024-08-20T23:24:55.2784404Z TIMING: 2024-08-20T23:24:55.2785850Z STATS: call_* op count: 0 | FakeTensor.__torch_dispatch__:14 | FakeTensorMode.__torch_dispatch__:107 2024-08-20T23:24:55.2787405Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T23:24:58.9375926Z 2024-08-20T23:24:59.2405164Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:24:59.2406060Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:24:59.2406564Z cuda eval llama_v2_7b_16h 2024-08-20T23:24:59.2429492Z Traceback (most recent call last): 2024-08-20T23:24:59.2430466Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:24:59.2431273Z ) = runner.load_model( 2024-08-20T23:24:59.2431956Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 309, in load_model 2024-08-20T23:24:59.2432688Z benchmark = benchmark_cls( 2024-08-20T23:24:59.2433422Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 24, in __call__ 2024-08-20T23:24:59.2434213Z obj = type.__call__(cls, *args, **kwargs) 2024-08-20T23:24:59.2435141Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama_v2_7b_16h/__init__.py", line 12, in __init__ 2024-08-20T23:24:59.2436346Z super().__init__(name="llama_v2_7b_16h", test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:24:59.2437646Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/framework/huggingface/model_factory.py", line 81, in __init__ 2024-08-20T23:24:59.2438844Z super().__init__(test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:24:59.2439820Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 92, in __init__ 2024-08-20T23:24:59.2440571Z self._skip_by_device_name() 2024-08-20T23:24:59.2441383Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 165, in _skip_by_device_name 2024-08-20T23:24:59.2442630Z raise NotImplementedError(f"The current device {current_device_name} is skipped by its `{self.name}/metadata.yaml`.") 2024-08-20T23:24:59.2443855Z NotImplementedError: The current device NVIDIA A10G is skipped by its `llama_v2_7b_16h/metadata.yaml`. 2024-08-20T23:24:59.2444476Z 2024-08-20T23:24:59.2444598Z model_fail_to_load 2024-08-20T23:25:02.6833792Z 2024-08-20T23:25:02.9876592Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:25:02.9877125Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:25:02.9877567Z cuda eval llava 2024-08-20T23:25:02.9884158Z Traceback (most recent call last): 2024-08-20T23:25:02.9885104Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:25:02.9885981Z ) = runner.load_model( 2024-08-20T23:25:02.9886704Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 309, in load_model 2024-08-20T23:25:02.9887464Z benchmark = benchmark_cls( 2024-08-20T23:25:02.9888198Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 24, in __call__ 2024-08-20T23:25:02.9888999Z obj = type.__call__(cls, *args, **kwargs) 2024-08-20T23:25:02.9889864Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llava/__init__.py", line 11, in __init__ 2024-08-20T23:25:02.9890985Z super().__init__(name="llava", test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:25:02.9892262Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/framework/huggingface/model_factory.py", line 81, in __init__ 2024-08-20T23:25:02.9893776Z super().__init__(test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:25:02.9894784Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 92, in __init__ 2024-08-20T23:25:02.9895549Z self._skip_by_device_name() 2024-08-20T23:25:02.9896368Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 165, in _skip_by_device_name 2024-08-20T23:25:02.9897609Z raise NotImplementedError(f"The current device {current_device_name} is skipped by its `{self.name}/metadata.yaml`.") 2024-08-20T23:25:02.9898847Z NotImplementedError: The current device NVIDIA A10G is skipped by its `llava/metadata.yaml`. 2024-08-20T23:25:02.9899427Z 2024-08-20T23:25:02.9899552Z model_fail_to_load 2024-08-20T23:25:06.4412008Z 2024-08-20T23:25:07.1909869Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:25:07.1910462Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:25:07.1915857Z cuda eval maml 2024-08-20T23:25:07.1919890Z pass_due_to_skip 2024-08-20T23:25:07.1928787Z TIMING: 2024-08-20T23:25:07.1929200Z STATS: call_* op count: 0 2024-08-20T23:25:07.1929911Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T23:25:10.7678668Z 2024-08-20T23:25:11.3132531Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:25:11.3133030Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:25:11.3137882Z cuda eval maml_omniglot 2024-08-20T23:25:27.1045764Z pass 2024-08-20T23:25:27.1047099Z TIMING: backend_compile:8e-05 entire_frame_compile:0.16602 total_wall_time:0.16602 2024-08-20T23:25:27.1048281Z STATS: call_* op count: 14 | FakeTensor.__torch_dispatch__:202 | FakeTensorMode.__torch_dispatch__:1607 | ProxyTorchDispatchMode.__torch_dispatch__:339 2024-08-20T23:25:27.1049407Z Dynamo produced 1 graphs covering 14 ops with 0 graph breaks (0 unique) 2024-08-20T23:25:30.9339734Z 2024-08-20T23:25:31.1518805Z loading model: 0it [00:00, ?it/s]Downloading: "https://download.pytorch.org/models/mnasnet1.0_top1_73.512-f206786ef8.pth" to /var/lib/jenkins/.cache/torch/hub/checkpoints/mnasnet1.0_top1_73.512-f206786ef8.pth 2024-08-20T23:25:31.1772025Z 2024-08-20T23:25:31.1772123Z 2024-08-20T23:25:31.2377292Z 0% 0.00/16.9M [00:00 will be ignored 2024-08-20T23:27:12.9535272Z ERROR:common: 2024-08-20T23:27:12.9535767Z Traceback (most recent call last): 2024-08-20T23:27:12.9536525Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T23:27:12.9537424Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T23:27:12.9539873Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T23:27:12.9540975Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T23:27:12.9542103Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T23:27:12.9543025Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T23:27:12.9544457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T23:27:12.9545871Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T23:27:12.9546848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T23:27:12.9547661Z result_traced = opt_f(*args, **kwargs) 2024-08-20T23:27:12.9548674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:27:12.9549863Z return self._call_impl(*args, **kwargs) 2024-08-20T23:27:12.9550877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:27:12.9551716Z return forward_call(*args, **kwargs) 2024-08-20T23:27:12.9552589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T23:27:12.9553371Z return fn(*args, **kwargs) 2024-08-20T23:27:12.9554562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T23:27:12.9555687Z return self._torchdynamo_orig_callable( 2024-08-20T23:27:12.9556886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T23:27:12.9557798Z return _compile( 2024-08-20T23:27:12.9559191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T23:27:12.9560461Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T23:27:12.9561926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T23:27:12.9563160Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T23:27:12.9564822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T23:27:12.9565732Z return function(*args, **kwargs) 2024-08-20T23:27:12.9566693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T23:27:12.9567604Z out_code = transform_code_object(code, transform) 2024-08-20T23:27:12.9568751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T23:27:12.9569754Z transformations(instructions, code_options) 2024-08-20T23:27:12.9570692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T23:27:12.9571484Z return fn(*args, **kwargs) 2024-08-20T23:27:12.9572431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T23:27:12.9573246Z tracer.run() 2024-08-20T23:27:12.9574049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T23:27:12.9574850Z super().run() 2024-08-20T23:27:12.9575803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:27:12.9576828Z while self.step(): 2024-08-20T23:27:12.9577681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:27:12.9578550Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:27:12.9579532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T23:27:12.9580385Z return inner_fn(self, inst) 2024-08-20T23:27:12.9581355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1664, in CALL_FUNCTION_EX 2024-08-20T23:27:12.9582310Z self.call_function(fn, argsvars.items, kwargsvars) 2024-08-20T23:27:12.9583333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T23:27:12.9584430Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T23:27:12.9585580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 899, in call_function 2024-08-20T23:27:12.9586635Z return variables.UserFunctionVariable(fn, source=source).call_function( 2024-08-20T23:27:12.9587797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T23:27:12.9588718Z return super().call_function(tx, args, kwargs) 2024-08-20T23:27:12.9589749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T23:27:12.9590812Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T23:27:12.9592052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T23:27:12.9593172Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T23:27:12.9594574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T23:27:12.9595807Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T23:27:12.9597190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T23:27:12.9598032Z tracer.run() 2024-08-20T23:27:12.9598834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:27:12.9599621Z while self.step(): 2024-08-20T23:27:12.9600455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:27:12.9601414Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:27:12.9602382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T23:27:12.9603203Z return inner_fn(self, inst) 2024-08-20T23:27:12.9604267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1664, in CALL_FUNCTION_EX 2024-08-20T23:27:12.9605221Z self.call_function(fn, argsvars.items, kwargsvars) 2024-08-20T23:27:12.9606246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T23:27:12.9607349Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T23:27:12.9608500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T23:27:12.9609420Z return super().call_function(tx, args, kwargs) 2024-08-20T23:27:12.9610451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T23:27:12.9611374Z return super().call_function(tx, args, kwargs) 2024-08-20T23:27:12.9612396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T23:27:12.9613459Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T23:27:12.9614692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T23:27:12.9615806Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T23:27:12.9616951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T23:27:12.9617871Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T23:27:12.9618895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T23:27:12.9619724Z tracer.run() 2024-08-20T23:27:12.9620522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:27:12.9621307Z while self.step(): 2024-08-20T23:27:12.9622140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:27:12.9623000Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:27:12.9623974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T23:27:12.9624788Z return inner_fn(self, inst) 2024-08-20T23:27:12.9625722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T23:27:12.9626594Z self.call_function(fn, args, {}) 2024-08-20T23:27:12.9627547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T23:27:12.9628637Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T23:27:12.9629796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T23:27:12.9630813Z return super().call_function(tx, args, kwargs) 2024-08-20T23:27:12.9631842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T23:27:12.9632759Z return super().call_function(tx, args, kwargs) 2024-08-20T23:27:12.9633774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T23:27:12.9634940Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T23:27:12.9636178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T23:27:12.9637290Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T23:27:12.9638428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T23:27:12.9639357Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T23:27:12.9640388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T23:27:12.9641225Z tracer.run() 2024-08-20T23:27:12.9642028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:27:12.9642812Z while self.step(): 2024-08-20T23:27:12.9643725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:27:12.9644599Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:27:12.9645803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T23:27:12.9646662Z return inner_fn(self, inst) 2024-08-20T23:27:12.9647623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T23:27:12.9648493Z self.call_function(fn, args, {}) 2024-08-20T23:27:12.9649444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T23:27:12.9650540Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T23:27:12.9651697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T23:27:12.9652625Z return super().call_function(tx, args, kwargs) 2024-08-20T23:27:12.9653645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T23:27:12.9654558Z return super().call_function(tx, args, kwargs) 2024-08-20T23:27:12.9655579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T23:27:12.9656638Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T23:27:12.9657869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T23:27:12.9658977Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T23:27:12.9660110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T23:27:12.9661033Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T23:27:12.9662061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T23:27:12.9662899Z tracer.run() 2024-08-20T23:27:12.9663688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:27:12.9664480Z while self.step(): 2024-08-20T23:27:12.9665476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:27:12.9666341Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:27:12.9667326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1744, in STORE_ATTR 2024-08-20T23:27:12.9668196Z BuiltinVariable(setattr).call_function( 2024-08-20T23:27:12.9669180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T23:27:12.9670199Z return handler(tx, args, kwargs) 2024-08-20T23:27:12.9671180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builtin.py", line 854, in builtin_dispatch 2024-08-20T23:27:12.9672053Z rv = handler(tx, args, kwargs) 2024-08-20T23:27:12.9673024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builtin.py", line 781, in call_self_handler 2024-08-20T23:27:12.9673948Z result = self_handler(tx, *args, **kwargs) 2024-08-20T23:27:12.9674946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builtin.py", line 1779, in call_setattr 2024-08-20T23:27:12.9675918Z return obj.call_method(tx, "__setattr__", [name_var, val], {}) 2024-08-20T23:27:12.9677007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 1051, in call_method 2024-08-20T23:27:12.9677936Z return super().call_method(tx, name, args, kwargs) 2024-08-20T23:27:12.9678984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/user_defined.py", line 736, in call_method 2024-08-20T23:27:12.9680017Z return UserMethodVariable(method, self, source=source).call_function( 2024-08-20T23:27:12.9681203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T23:27:12.9682124Z return super().call_function(tx, args, kwargs) 2024-08-20T23:27:12.9683154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T23:27:12.9684152Z return super().call_function(tx, args, kwargs) 2024-08-20T23:27:12.9685165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T23:27:12.9686218Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T23:27:12.9687447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T23:27:12.9688560Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T23:27:12.9689691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T23:27:12.9690643Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T23:27:12.9691692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T23:27:12.9692524Z tracer.run() 2024-08-20T23:27:12.9693322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:27:12.9694106Z while self.step(): 2024-08-20T23:27:12.9694934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:27:12.9695791Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:27:12.9696759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T23:27:12.9697570Z return inner_fn(self, inst) 2024-08-20T23:27:12.9698501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T23:27:12.9699458Z self.call_function(fn, args, {}) 2024-08-20T23:27:12.9700416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T23:27:12.9701504Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T23:27:12.9702625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/misc.py", line 970, in call_function 2024-08-20T23:27:12.9703565Z return self.obj.call_method(tx, self.name, args, kwargs) 2024-08-20T23:27:12.9704677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/misc.py", line 211, in call_method 2024-08-20T23:27:12.9705640Z return self.objvar.method_setattr_standard(tx, *args, **kwargs) 2024-08-20T23:27:12.9706820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/user_defined.py", line 752, in method_setattr_standard 2024-08-20T23:27:12.9707805Z unimplemented(f"setattr({self}, {name}, ...)") 2024-08-20T23:27:12.9708740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/exc.py", line 289, in unimplemented 2024-08-20T23:27:12.9709573Z raise Unsupported(msg, case_name=case_name) 2024-08-20T23:27:12.9710443Z torch._dynamo.exc.Unsupported: setattr(UnspecializedNNModuleVariable(DistributedDataParallel), modules_buffers, ...) 2024-08-20T23:27:12.9711147Z 2024-08-20T23:27:12.9711263Z from user code: 2024-08-20T23:27:12.9712055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 39, in inner 2024-08-20T23:27:12.9712859Z return fn(*args, **kwargs) 2024-08-20T23:27:12.9713745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1639, in forward 2024-08-20T23:27:12.9714664Z inputs, kwargs = self._pre_forward(*inputs, **kwargs) 2024-08-20T23:27:12.9715713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1535, in _pre_forward 2024-08-20T23:27:12.9716578Z self._sync_buffers() 2024-08-20T23:27:12.9717486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 2171, in _sync_buffers 2024-08-20T23:27:12.9718364Z self._assign_modules_buffers() 2024-08-20T23:27:12.9719376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1335, in _assign_modules_buffers 2024-08-20T23:27:12.9720306Z self.modules_buffers = [ 2024-08-20T23:27:12.9721216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2016, in __setattr__ 2024-08-20T23:27:12.9722046Z super().__setattr__(name, value) 2024-08-20T23:27:12.9722331Z 2024-08-20T23:27:12.9722622Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T23:27:12.9723065Z 2024-08-20T23:27:12.9723364Z TorchDynamo optimized model failed to run because of following error 2024-08-20T23:27:12.9724013Z fail_to_run 2024-08-20T23:27:12.9724286Z TIMING: 2024-08-20T23:27:12.9724573Z STATS: call_* op count: 0 2024-08-20T23:27:12.9725102Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T23:27:13.2319339Z [rank0]:[W820 23:27:13.196623528 ProcessGroupNCCL.cpp:1223] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present, but this warning has only been added since PyTorch 2.4 (function operator()) 2024-08-20T23:27:17.1864332Z 2024-08-20T23:27:17.4913764Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:27:17.4914373Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:27:17.4914873Z cuda eval moondream 2024-08-20T23:27:17.4921755Z Traceback (most recent call last): 2024-08-20T23:27:17.4922969Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:27:17.4923998Z ) = runner.load_model( 2024-08-20T23:27:17.4924675Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 309, in load_model 2024-08-20T23:27:17.4925399Z benchmark = benchmark_cls( 2024-08-20T23:27:17.4926113Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 24, in __call__ 2024-08-20T23:27:17.4927087Z obj = type.__call__(cls, *args, **kwargs) 2024-08-20T23:27:17.4927968Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moondream/__init__.py", line 11, in __init__ 2024-08-20T23:27:17.4929129Z super().__init__(name="moondream", test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:27:17.4930399Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/framework/huggingface/model_factory.py", line 81, in __init__ 2024-08-20T23:27:17.4931590Z super().__init__(test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:27:17.4932563Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 92, in __init__ 2024-08-20T23:27:17.4933316Z self._skip_by_device_name() 2024-08-20T23:27:17.4934122Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 165, in _skip_by_device_name 2024-08-20T23:27:17.4935343Z raise NotImplementedError(f"The current device {current_device_name} is skipped by its `{self.name}/metadata.yaml`.") 2024-08-20T23:27:17.4936452Z NotImplementedError: The current device NVIDIA A10G is skipped by its `moondream/metadata.yaml`. 2024-08-20T23:27:17.4937043Z 2024-08-20T23:27:17.4937166Z model_fail_to_load 2024-08-20T23:27:20.9926751Z 2024-08-20T23:27:22.9739154Z loading model: 0it [00:00, ?it/s]number of parameters: 123.69M 2024-08-20T23:27:23.2458734Z num decayed parameter tensors: 50, with 124,354,560 parameters 2024-08-20T23:27:23.2459968Z num non-decayed parameter tensors: 98, with 121,344 parameters 2024-08-20T23:27:23.2460551Z using fused AdamW: True 2024-08-20T23:27:23.5817000Z 2024-08-20T23:27:23.5817587Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:27:23.5852391Z cuda eval nanogpt 2024-08-20T23:27:50.4480166Z pass 2024-08-20T23:27:50.4560243Z TIMING: backend_compile:0.00017 entire_frame_compile:1.52406 total_wall_time:1.52406 2024-08-20T23:27:50.4562814Z STATS: call_* op count: 393 | FakeTensorMode.__torch_dispatch__:13878 | FakeTensor.__torch_dispatch__:2358 | ProxyTorchDispatchMode.__torch_dispatch__:4667 2024-08-20T23:27:50.4564065Z Dynamo produced 1 graphs covering 393 ops with 0 graph breaks (0 unique) 2024-08-20T23:27:54.5835258Z 2024-08-20T23:27:57.0097789Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:27:57.0098390Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:27:57.0100586Z cuda eval nvidia_deeprecommender 2024-08-20T23:28:15.6213748Z pass 2024-08-20T23:28:15.6281917Z TIMING: backend_compile:0.00011 entire_frame_compile:0.18704 total_wall_time:0.18704 2024-08-20T23:28:15.6283397Z STATS: call_* op count: 13 | FakeTensorMode.__torch_dispatch__:1116 | ProxyTorchDispatchMode.__torch_dispatch__:352 | FakeTensor.__torch_dispatch__:168 2024-08-20T23:28:15.6284526Z Dynamo produced 1 graphs covering 13 ops with 0 graph breaks (0 unique) 2024-08-20T23:28:19.4978307Z 2024-08-20T23:28:20.1637103Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:28:20.1637571Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:28:20.1700024Z cuda eval phlippe_densenet 2024-08-20T23:28:49.6914782Z pass 2024-08-20T23:28:49.6927392Z TIMING: backend_compile:0.00011 entire_frame_compile:1.62422 total_wall_time:1.62422 2024-08-20T23:28:49.6928607Z STATS: call_* op count: 186 | FakeTensor.__torch_dispatch__:3002 | FakeTensorMode.__torch_dispatch__:16473 | ProxyTorchDispatchMode.__torch_dispatch__:4512 2024-08-20T23:28:49.6934139Z Dynamo produced 1 graphs covering 186 ops with 0 graph breaks (0 unique) 2024-08-20T23:28:53.9516025Z 2024-08-20T23:28:54.5209419Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:28:54.5210008Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:28:54.5243077Z cuda eval phlippe_resnet 2024-08-20T23:29:10.7534765Z pass 2024-08-20T23:29:10.7547369Z TIMING: backend_compile:9e-05 entire_frame_compile:0.54344 total_wall_time:0.54344 2024-08-20T23:29:10.7549009Z STATS: call_* op count: 71 | FakeTensor.__torch_dispatch__:1101 | FakeTensorMode.__torch_dispatch__:6348 | ProxyTorchDispatchMode.__torch_dispatch__:1708 2024-08-20T23:29:10.7550924Z Dynamo produced 1 graphs covering 71 ops with 0 graph breaks (0 unique) 2024-08-20T23:29:14.7039434Z 2024-08-20T23:29:15.0784405Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:29:15.0784958Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:29:15.0785424Z cuda eval pyhpc_equation_of_state 2024-08-20T23:29:30.7588013Z pass 2024-08-20T23:29:30.7588723Z TIMING: backend_compile:0.00013 entire_frame_compile:0.33815 total_wall_time:0.33815 2024-08-20T23:29:30.7592100Z STATS: call_* op count: 366 | FakeTensorMode.__torch_dispatch__:5088 | ProxyTorchDispatchMode.__torch_dispatch__:1885 | FakeTensor.__torch_dispatch__:1086 2024-08-20T23:29:30.7593270Z Dynamo produced 1 graphs covering 366 ops with 0 graph breaks (0 unique) 2024-08-20T23:29:34.6649273Z 2024-08-20T23:29:35.0178714Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:29:35.0179185Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:29:35.0181079Z cuda eval pyhpc_isoneutral_mixing 2024-08-20T23:29:58.4699729Z pass 2024-08-20T23:29:58.4700596Z TIMING: backend_compile:0.00023 entire_frame_compile:1.18605 total_wall_time:1.18605 2024-08-20T23:29:58.4702312Z STATS: call_* op count: 744 | FakeTensorMode.__torch_dispatch__:23645 | FakeTensor.__torch_dispatch__:2600 | ProxyTorchDispatchMode.__torch_dispatch__:7030 2024-08-20T23:29:58.4703951Z Dynamo produced 1 graphs covering 744 ops with 0 graph breaks (0 unique) 2024-08-20T23:30:02.7703834Z 2024-08-20T23:30:03.5540815Z loading model: 0it [00:00, ?it/s]WARNING:common:Model pyhpc_turbulent_kinetic_energy does not support bfloat16, running with amp instead 2024-08-20T23:30:03.6711402Z 2024-08-20T23:30:03.6711713Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:30:03.6712711Z WARNING:common:Model pyhpc_turbulent_kinetic_energy does not support bfloat16, running with amp instead 2024-08-20T23:30:03.6713542Z cuda eval pyhpc_turbulent_kinetic_energy 2024-08-20T23:30:03.7168862Z WARNING:common:Model pyhpc_turbulent_kinetic_energy does not support bfloat16, running with amp instead 2024-08-20T23:30:44.0004787Z pass 2024-08-20T23:30:44.0175877Z TIMING: backend_compile:0.00027 entire_frame_compile:1.30924 total_wall_time:1.30924 2024-08-20T23:30:44.0180848Z STATS: call_* op count: 994 | FakeTensorMode.__torch_dispatch__:24908 | FakeTensor.__torch_dispatch__:2539 | ProxyTorchDispatchMode.__torch_dispatch__:7366 2024-08-20T23:30:44.0182006Z Dynamo produced 1 graphs covering 994 ops with 0 graph breaks (0 unique) 2024-08-20T23:30:48.6710162Z 2024-08-20T23:30:49.4982767Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:30:49.4983231Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:30:49.4994660Z cuda eval pytorch_CycleGAN_and_pix2pix 2024-08-20T23:31:16.4396646Z pass 2024-08-20T23:31:16.4397786Z TIMING: backend_compile:0.0001 entire_frame_compile:0.46518 total_wall_time:0.46518 2024-08-20T23:31:16.4398960Z STATS: call_* op count: 91 | FakeTensorMode.__torch_dispatch__:5716 | FakeTensor.__torch_dispatch__:1356 | ProxyTorchDispatchMode.__torch_dispatch__:2000 2024-08-20T23:31:16.4400444Z Dynamo produced 1 graphs covering 91 ops with 0 graph breaks (0 unique) 2024-08-20T23:31:20.5489010Z 2024-08-20T23:31:21.7820398Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:31:21.7820983Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:31:21.7840800Z cuda eval pytorch_stargan 2024-08-20T23:31:56.5903348Z pass 2024-08-20T23:31:56.5923071Z TIMING: backend_compile:9e-05 entire_frame_compile:0.54109 total_wall_time:0.54109 2024-08-20T23:31:56.5924612Z STATS: call_* op count: 56 | FakeTensorMode.__torch_dispatch__:8478 | FakeTensor.__torch_dispatch__:1291 | ProxyTorchDispatchMode.__torch_dispatch__:2504 2024-08-20T23:31:56.5925751Z Dynamo produced 1 graphs covering 56 ops with 0 graph breaks (0 unique) 2024-08-20T23:32:00.7657548Z 2024-08-20T23:32:01.4675573Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:32:01.4676056Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:32:01.4696854Z cuda eval pytorch_unet 2024-08-20T23:32:34.0790235Z pass 2024-08-20T23:32:34.0920471Z TIMING: backend_compile:0.0001 entire_frame_compile:0.74739 total_wall_time:0.74739 2024-08-20T23:32:34.0923164Z STATS: call_* op count: 71 | FakeTensor.__torch_dispatch__:1467 | FakeTensorMode.__torch_dispatch__:8418 | ProxyTorchDispatchMode.__torch_dispatch__:2554 2024-08-20T23:32:34.0924340Z Dynamo produced 1 graphs covering 71 ops with 0 graph breaks (0 unique) 2024-08-20T23:32:38.1512453Z 2024-08-20T23:32:38.9612128Z loading model: 0it [00:00, ?it/s]Downloading: "https://download.pytorch.org/models/resnet152-394f9c45.pth" to /var/lib/jenkins/.cache/torch/hub/checkpoints/resnet152-394f9c45.pth 2024-08-20T23:32:38.9786287Z 2024-08-20T23:32:38.9786292Z 2024-08-20T23:32:39.0791868Z 0% 0.00/230M [00:00` is not supported or pytree-flattenable. 2024-08-20T23:47:44.4861850Z Exported graphs outputs can only contain the following supported types: [, , , , , , , , , , , , , , , , , , ]. 2024-08-20T23:47:44.4865018Z If you are using a custom class object, please register a pytree_flatten/unflatten function using `torch.utils._pytree.register_pytree_node` or `torch.export.register_dataclass`. 2024-08-20T23:47:44.4866345Z TorchDynamo optimized model failed to run because of following error 2024-08-20T23:47:44.4866904Z fail_to_run 2024-08-20T23:47:44.4867845Z TIMING: backend_compile:8e-05 entire_frame_compile:0.15178 total_wall_time:0.15178 2024-08-20T23:47:44.4868768Z STATS: call_* op count: 16 | FakeTensor.__torch_dispatch__:23 | FakeTensorMode.__torch_dispatch__:128 2024-08-20T23:47:44.4869653Z Dynamo produced 1 graphs covering 16 ops with 0 graph breaks (0 unique) 2024-08-20T23:47:48.1118285Z 2024-08-20T23:47:48.2854832Z loading model: 0it [00:00, ?it/s]Downloading: "https://download.pytorch.org/models/squeezenet1_1-b8a52dc0.pth" to /var/lib/jenkins/.cache/torch/hub/checkpoints/squeezenet1_1-b8a52dc0.pth 2024-08-20T23:47:48.3048039Z 2024-08-20T23:47:48.3048279Z 2024-08-20T23:47:48.3237628Z 0% 0.00/4.73M [00:00at(301); 2024-08-20T23:53:17.7375658Z | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-08-20T23:53:22.4002610Z pass 2024-08-20T23:53:22.4148376Z TIMING: backend_compile:9e-05 entire_frame_compile:1.25657 total_wall_time:1.25657 2024-08-20T23:53:22.4149582Z STATS: call_* op count: 36 | FakeTensor.__torch_dispatch__:5640 | FakeTensorMode.__torch_dispatch__:27031 | ProxyTorchDispatchMode.__torch_dispatch__:10083 2024-08-20T23:53:22.4151746Z Dynamo produced 1 graphs covering 36 ops with 0 graph breaks (0 unique) 2024-08-20T23:53:27.0593045Z 2024-08-20T23:53:27.4689646Z loading model: 0it [00:00, ?it/s]WARNING:common:Model tts_angular does not support bfloat16, running with amp instead 2024-08-20T23:53:27.6192626Z 2024-08-20T23:53:27.6193319Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:53:27.6194041Z WARNING:common:Model tts_angular does not support bfloat16, running with amp instead 2024-08-20T23:53:27.6194750Z cuda eval tts_angular 2024-08-20T23:53:27.6650574Z WARNING:common:Model tts_angular does not support bfloat16, running with amp instead 2024-08-20T23:53:27.9266213Z ERROR:common: 2024-08-20T23:53:27.9266604Z Traceback (most recent call last): 2024-08-20T23:53:27.9267346Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T23:53:27.9268106Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T23:53:27.9268884Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T23:53:27.9270747Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T23:53:27.9271608Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T23:53:27.9272340Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T23:53:27.9273515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T23:53:27.9274448Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T23:53:27.9275749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T23:53:27.9276636Z result_traced = opt_f(*args, **kwargs) 2024-08-20T23:53:27.9277640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:53:27.9278586Z return self._call_impl(*args, **kwargs) 2024-08-20T23:53:27.9279531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:53:27.9280560Z return forward_call(*args, **kwargs) 2024-08-20T23:53:27.9281758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T23:53:27.9282575Z return fn(*args, **kwargs) 2024-08-20T23:53:27.9283645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:53:27.9284546Z return self._call_impl(*args, **kwargs) 2024-08-20T23:53:27.9285493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:53:27.9286335Z return forward_call(*args, **kwargs) 2024-08-20T23:53:27.9287256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T23:53:27.9288114Z return self._torchdynamo_orig_callable( 2024-08-20T23:53:27.9289287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T23:53:27.9290095Z return _compile( 2024-08-20T23:53:27.9290924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T23:53:27.9292072Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T23:53:27.9293151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T23:53:27.9294260Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T23:53:27.9295265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T23:53:27.9296098Z return function(*args, **kwargs) 2024-08-20T23:53:27.9297040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T23:53:27.9297948Z out_code = transform_code_object(code, transform) 2024-08-20T23:53:27.9299068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T23:53:27.9300057Z transformations(instructions, code_options) 2024-08-20T23:53:27.9301031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T23:53:27.9301822Z return fn(*args, **kwargs) 2024-08-20T23:53:27.9302702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T23:53:27.9303512Z tracer.run() 2024-08-20T23:53:27.9304313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T23:53:27.9305101Z super().run() 2024-08-20T23:53:27.9305903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:53:27.9306780Z while self.step(): 2024-08-20T23:53:27.9307755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:53:27.9308627Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:53:27.9309602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T23:53:27.9310423Z return inner_fn(self, inst) 2024-08-20T23:53:27.9311351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T23:53:27.9312226Z self.call_function(fn, args, {}) 2024-08-20T23:53:27.9313177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T23:53:27.9314276Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T23:53:27.9315428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 371, in call_function 2024-08-20T23:53:27.9316284Z tx.call_function( 2024-08-20T23:53:27.9317166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T23:53:27.9318261Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T23:53:27.9319418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 442, in call_function 2024-08-20T23:53:27.9320307Z return tx.inline_user_function_return( 2024-08-20T23:53:27.9321359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T23:53:27.9322474Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T23:53:27.9323767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T23:53:27.9324806Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T23:53:27.9325853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T23:53:27.9326684Z tracer.run() 2024-08-20T23:53:27.9327481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:53:27.9328264Z while self.step(): 2024-08-20T23:53:27.9329099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:53:27.9330046Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:53:27.9331019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T23:53:27.9331835Z return inner_fn(self, inst) 2024-08-20T23:53:27.9332795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1664, in CALL_FUNCTION_EX 2024-08-20T23:53:27.9333741Z self.call_function(fn, argsvars.items, kwargsvars) 2024-08-20T23:53:27.9334769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T23:53:27.9335866Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T23:53:27.9337018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T23:53:27.9337954Z return super().call_function(tx, args, kwargs) 2024-08-20T23:53:27.9338983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T23:53:27.9339912Z return super().call_function(tx, args, kwargs) 2024-08-20T23:53:27.9340931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T23:53:27.9342005Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T23:53:27.9343239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T23:53:27.9344353Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T23:53:27.9345491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T23:53:27.9346885Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T23:53:27.9347926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T23:53:27.9348793Z tracer.run() 2024-08-20T23:53:27.9349602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T23:53:27.9350387Z while self.step(): 2024-08-20T23:53:27.9351220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T23:53:27.9352076Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T23:53:27.9353039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T23:53:27.9353850Z return inner_fn(self, inst) 2024-08-20T23:53:27.9354776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T23:53:27.9355645Z self.call_function(fn, args, {}) 2024-08-20T23:53:27.9356595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T23:53:27.9357681Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T23:53:27.9359016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 370, in call_function 2024-08-20T23:53:27.9359915Z return self.obj.call_method( 2024-08-20T23:53:27.9360847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 760, in call_method 2024-08-20T23:53:27.9361773Z return super().call_method(tx, name, args, kwargs) 2024-08-20T23:53:27.9362763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/base.py", line 327, in call_method 2024-08-20T23:53:27.9363798Z unimplemented(f"call_method {self} {name} {args} {kwargs}") 2024-08-20T23:53:27.9364920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/exc.py", line 289, in unimplemented 2024-08-20T23:53:27.9365752Z raise Unsupported(msg, case_name=case_name) 2024-08-20T23:53:27.9366449Z torch._dynamo.exc.Unsupported: call_method NNModuleVariable() flatten_parameters [] {} 2024-08-20T23:53:27.9366997Z 2024-08-20T23:53:27.9367116Z from user code: 2024-08-20T23:53:27.9367861Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tts_angular/model.py", line 59, in forward 2024-08-20T23:53:27.9368684Z d = self.layers(x) 2024-08-20T23:53:27.9369536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:53:27.9370387Z return forward_call(*args, **kwargs) 2024-08-20T23:53:27.9371239Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tts_angular/model.py", line 17, in forward 2024-08-20T23:53:27.9372096Z self.lstm.flatten_parameters() 2024-08-20T23:53:27.9372371Z 2024-08-20T23:53:27.9372667Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T23:53:27.9373122Z 2024-08-20T23:53:27.9373417Z TorchDynamo optimized model failed to run because of following error 2024-08-20T23:53:27.9373978Z fail_to_run 2024-08-20T23:53:27.9374269Z TIMING: 2024-08-20T23:53:27.9374544Z STATS: call_* op count: 0 2024-08-20T23:53:27.9375088Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T23:53:31.4774408Z 2024-08-20T23:53:32.4792568Z loading model: 0it [00:00, ?it/s]Downloading: "https://download.pytorch.org/models/vgg16-397923af.pth" to /var/lib/jenkins/.cache/torch/hub/checkpoints/vgg16-397923af.pth 2024-08-20T23:53:32.4979829Z 2024-08-20T23:53:32.4979993Z 2024-08-20T23:53:32.5982825Z 0% 0.00/528M [00:00at(225); 2024-08-20T23:55:03.0682075Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-08-20T23:55:03.0683846Z /tmp/tmpk_lj_y5y/ck7hfnpcnprhpegncrillgsfaenz3ecmk6ck6l7sj6ch4co2kr22/c53mix67zok3aw2gdlry6y3hrgw2qfbdwwox732mmebykofyx7nr.cpp:4269:10: warning: variable ‘L__self___module_list_89_anchor_wh’ set but not used [-Wunused-but-set-variable] 2024-08-20T23:55:03.0685494Z 4269 | auto L__self___module_list_89_anchor_wh = constants_->at(226); 2024-08-20T23:55:03.0686080Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-08-20T23:55:03.0687708Z /tmp/tmpk_lj_y5y/ck7hfnpcnprhpegncrillgsfaenz3ecmk6ck6l7sj6ch4co2kr22/c53mix67zok3aw2gdlry6y3hrgw2qfbdwwox732mmebykofyx7nr.cpp:4270:10: warning: variable ‘L__self___module_list_101_anchor_vec’ set but not used [-Wunused-but-set-variable] 2024-08-20T23:55:03.0689478Z 4270 | auto L__self___module_list_101_anchor_vec = constants_->at(227); 2024-08-20T23:55:03.0690069Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-08-20T23:55:03.0691697Z /tmp/tmpk_lj_y5y/ck7hfnpcnprhpegncrillgsfaenz3ecmk6ck6l7sj6ch4co2kr22/c53mix67zok3aw2gdlry6y3hrgw2qfbdwwox732mmebykofyx7nr.cpp:4271:10: warning: variable ‘L__self___module_list_101_anchor_wh’ set but not used [-Wunused-but-set-variable] 2024-08-20T23:55:03.0693327Z 4271 | auto L__self___module_list_101_anchor_wh = constants_->at(228); 2024-08-20T23:55:03.0693918Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-08-20T23:55:03.0695554Z /tmp/tmpk_lj_y5y/ck7hfnpcnprhpegncrillgsfaenz3ecmk6ck6l7sj6ch4co2kr22/c53mix67zok3aw2gdlry6y3hrgw2qfbdwwox732mmebykofyx7nr.cpp:4272:10: warning: variable ‘L__self___module_list_113_anchor_vec’ set but not used [-Wunused-but-set-variable] 2024-08-20T23:55:03.0697196Z 4272 | auto L__self___module_list_113_anchor_vec = constants_->at(229); 2024-08-20T23:55:03.0697782Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-08-20T23:55:03.0699407Z /tmp/tmpk_lj_y5y/ck7hfnpcnprhpegncrillgsfaenz3ecmk6ck6l7sj6ch4co2kr22/c53mix67zok3aw2gdlry6y3hrgw2qfbdwwox732mmebykofyx7nr.cpp:4273:10: warning: variable ‘L__self___module_list_113_anchor_wh’ set but not used [-Wunused-but-set-variable] 2024-08-20T23:55:03.0701042Z 4273 | auto L__self___module_list_113_anchor_wh = constants_->at(230); 2024-08-20T23:55:03.0701623Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-08-20T23:55:05.5600994Z pass 2024-08-20T23:55:05.5623494Z TIMING: backend_compile:0.00014 entire_frame_compile:3.29285 total_wall_time:3.29285 2024-08-20T23:55:05.5625108Z STATS: call_* op count: 341 | FakeTensor.__torch_dispatch__:5245 | FakeTensorMode.__torch_dispatch__:27646 | ProxyTorchDispatchMode.__torch_dispatch__:7390 2024-08-20T23:55:05.5626353Z Dynamo produced 1 graphs covering 341 ops with 0 graph breaks (0 unique) 2024-08-20T23:55:07.3918728Z accuracy pass_rate=65.91% 2024-08-20T23:55:07.3921827Z calls_captured gmean=0.00x mean=369.136x 2024-08-20T23:55:07.3925742Z unique_graphs gmean=0.00x mean=0.773x 2024-08-20T23:55:07.3928541Z graph_breaks gmean=0.00x mean=0.000x 2024-08-20T23:55:07.3931694Z unique_graph_breaks gmean=0.00x mean=0.000x 2024-08-20T23:55:07.3935427Z autograd_captures gmean=0.00x mean=0.000x 2024-08-20T23:55:07.3938533Z autograd_compiles gmean=0.00x mean=0.000x 2024-08-20T23:55:07.3941939Z cudagraph_skips gmean=0.00x mean=0.000x 2024-08-20T23:55:09.2343680Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy/aot_inductor_torchbench_inference.csv 2024-08-20T23:55:09.5753825Z llama XFAIL 2024-08-20T23:55:09.5757562Z llama_v2_7b_16h XFAIL 2024-08-20T23:55:09.5762425Z llava XFAIL 2024-08-20T23:55:09.5767847Z maml XFAIL 2024-08-20T23:55:09.5772916Z maml_omniglot PASS 2024-08-20T23:55:09.5777908Z mnasnet1_0 PASS 2024-08-20T23:55:09.5782867Z mobilenet_v2 PASS 2024-08-20T23:55:09.5787997Z mobilenet_v2_quantized_qat XFAIL 2024-08-20T23:55:09.5792951Z mobilenet_v3_large PASS 2024-08-20T23:55:09.5797786Z moco XFAIL 2024-08-20T23:55:09.5802832Z moondream XFAIL 2024-08-20T23:55:09.5808043Z nanogpt PASS 2024-08-20T23:55:09.5812852Z nvidia_deeprecommender PASS 2024-08-20T23:55:09.5817819Z phlippe_densenet PASS 2024-08-20T23:55:09.5822736Z phlippe_resnet PASS 2024-08-20T23:55:09.5827712Z pyhpc_equation_of_state PASS 2024-08-20T23:55:09.5832819Z pyhpc_isoneutral_mixing PASS 2024-08-20T23:55:09.5837647Z pyhpc_turbulent_kinetic_energy PASS 2024-08-20T23:55:09.5842516Z pytorch_CycleGAN_and_pix2pix PASS 2024-08-20T23:55:09.5848080Z pytorch_stargan PASS 2024-08-20T23:55:09.5852976Z pytorch_unet PASS 2024-08-20T23:55:09.5857892Z resnet152 PASS 2024-08-20T23:55:09.5862725Z resnet18 PASS 2024-08-20T23:55:09.5867658Z resnet50 PASS 2024-08-20T23:55:09.5872616Z resnet50_quantized_qat XFAIL 2024-08-20T23:55:09.5877546Z resnext50_32x4d PASS 2024-08-20T23:55:09.5882395Z sam XFAIL 2024-08-20T23:55:09.5887645Z sam_fast XFAIL 2024-08-20T23:55:09.5892488Z shufflenet_v2_x1_0 PASS 2024-08-20T23:55:09.5897265Z soft_actor_critic XFAIL 2024-08-20T23:55:09.5902160Z squeezenet1_1 PASS 2024-08-20T23:55:09.5907036Z stable_diffusion_text_encoder PASS 2024-08-20T23:55:09.5911945Z stable_diffusion_unet XFAIL 2024-08-20T23:55:09.5916894Z timm_efficientnet PASS 2024-08-20T23:55:09.5921698Z timm_regnet PASS 2024-08-20T23:55:09.5927189Z timm_resnest PASS 2024-08-20T23:55:09.5931717Z timm_vision_transformer PASS 2024-08-20T23:55:09.5936634Z timm_vision_transformer_large XFAIL 2024-08-20T23:55:09.5941497Z timm_vovnet PASS 2024-08-20T23:55:09.5946658Z torch_multimodal_clip PASS 2024-08-20T23:55:09.5951617Z tts_angular XFAIL 2024-08-20T23:55:09.5956583Z vgg16 PASS 2024-08-20T23:55:09.5961573Z vision_maskrcnn XFAIL 2024-08-20T23:55:09.5967081Z yolov3 PASS 2024-08-20T23:55:09.6488661Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy/aot_inductor_torchbench_inference.csv 2024-08-20T23:55:09.9861490Z llama PASS 2024-08-20T23:55:09.9865217Z llama_v2_7b_16h PASS 2024-08-20T23:55:09.9870000Z llava PASS 2024-08-20T23:55:09.9874906Z maml PASS 2024-08-20T23:55:09.9879559Z maml_omniglot PASS 2024-08-20T23:55:09.9884670Z mnasnet1_0 PASS 2024-08-20T23:55:09.9889658Z mobilenet_v2 PASS 2024-08-20T23:55:09.9894094Z mobilenet_v2_quantized_qat PASS 2024-08-20T23:55:09.9898842Z mobilenet_v3_large PASS 2024-08-20T23:55:09.9903556Z moco PASS 2024-08-20T23:55:09.9908211Z moondream PASS 2024-08-20T23:55:09.9912849Z nanogpt PASS 2024-08-20T23:55:09.9917616Z nvidia_deeprecommender PASS 2024-08-20T23:55:09.9922309Z phlippe_densenet PASS 2024-08-20T23:55:09.9927139Z phlippe_resnet PASS 2024-08-20T23:55:09.9931973Z pyhpc_equation_of_state PASS 2024-08-20T23:55:09.9936844Z pyhpc_isoneutral_mixing PASS 2024-08-20T23:55:09.9941529Z pyhpc_turbulent_kinetic_energy PASS 2024-08-20T23:55:09.9946727Z pytorch_CycleGAN_and_pix2pix PASS 2024-08-20T23:55:09.9951902Z pytorch_stargan PASS 2024-08-20T23:55:09.9956618Z pytorch_unet PASS 2024-08-20T23:55:09.9961312Z resnet152 PASS 2024-08-20T23:55:09.9966258Z resnet18 PASS 2024-08-20T23:55:09.9970940Z resnet50 PASS 2024-08-20T23:55:09.9975724Z resnet50_quantized_qat PASS 2024-08-20T23:55:09.9980457Z resnext50_32x4d PASS 2024-08-20T23:55:09.9984971Z sam PASS 2024-08-20T23:55:09.9989739Z sam_fast PASS 2024-08-20T23:55:09.9994460Z shufflenet_v2_x1_0 PASS 2024-08-20T23:55:09.9999162Z soft_actor_critic PASS 2024-08-20T23:55:10.0004236Z squeezenet1_1 PASS 2024-08-20T23:55:10.0008957Z stable_diffusion_text_encoder PASS 2024-08-20T23:55:10.0013715Z stable_diffusion_unet PASS 2024-08-20T23:55:10.0018377Z timm_efficientnet PASS 2024-08-20T23:55:10.0023195Z timm_regnet PASS 2024-08-20T23:55:10.0027937Z timm_resnest PASS 2024-08-20T23:55:10.0032829Z timm_vision_transformer PASS 2024-08-20T23:55:10.0037436Z timm_vision_transformer_large PASS 2024-08-20T23:55:10.0042286Z timm_vovnet PASS 2024-08-20T23:55:10.0047566Z torch_multimodal_clip PASS 2024-08-20T23:55:10.0053412Z tts_angular PASS 2024-08-20T23:55:10.0057915Z vgg16 PASS 2024-08-20T23:55:10.0062529Z vision_maskrcnn PASS 2024-08-20T23:55:10.0067224Z yolov3 PASS 2024-08-20T23:55:10.0587222Z + cleanup_workspace 2024-08-20T23:55:10.0588169Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-20T23:55:10.0589270Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-20T23:55:10.0590201Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-20T23:55:10.0590901Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-20T23:55:10.0591719Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-20T23:55:10.0592606Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-20T23:55:10.0593523Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-20T23:55:10.8284627Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-20T23:55:10.8285142Z cat test/**/*_toprint.log || true 2024-08-20T23:55:10.8298757Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:55:10.8299256Z env: 2024-08-20T23:55:10.8299531Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:10.8299979Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:10.8300932Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:10.8301611Z ##[endgroup] 2024-08-20T23:55:10.8404210Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-20T23:55:10.8436467Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:55:10.8436935Z kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:55:10.8445290Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:55:10.8446026Z env: 2024-08-20T23:55:10.8446298Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:10.8446753Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:10.8447500Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:10.8448164Z MONITOR_SCRIPT_PID: 52473 2024-08-20T23:55:10.8448508Z ##[endgroup] 2024-08-20T23:55:10.8654392Z Prepare all required actions 2024-08-20T23:55:10.8654846Z Getting action download info 2024-08-20T23:55:11.0710439Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-20T23:55:11.2271855Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-20T23:55:11.2272334Z with: 2024-08-20T23:55:11.2272939Z file-suffix: test-aot_inductor_torchbench-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026530815 2024-08-20T23:55:11.2273671Z s3-bucket: gha-artifacts 2024-08-20T23:55:11.2274024Z env: 2024-08-20T23:55:11.2274297Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:11.2274757Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:11.2275510Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:11.2276146Z ##[endgroup] 2024-08-20T23:55:11.2302729Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-20T23:55:11.2303388Z # Remove any previous test jsons if they exist 2024-08-20T23:55:11.2303895Z rm -f test-jsons-*.zip 2024-08-20T23:55:11.2304421Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-20T23:55:11.2313772Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:55:11.2314272Z env: 2024-08-20T23:55:11.2314534Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:11.2314990Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:11.2315736Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:11.2316716Z FILE_SUFFIX: test-aot_inductor_torchbench-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026530815 2024-08-20T23:55:11.2317409Z ##[endgroup] 2024-08-20T23:55:11.2661626Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-20T23:55:11.2691055Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-20T23:55:11.2696397Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-20T23:55:11.2709347Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-20T23:55:11.2720711Z adding: test/slow_tests.json (deflated 82%) 2024-08-20T23:55:11.2754479Z ##[group]Run # Remove any previous test reports if they exist 2024-08-20T23:55:11.2755120Z # Remove any previous test reports if they exist 2024-08-20T23:55:11.2755652Z rm -f test-reports-*.zip 2024-08-20T23:55:11.2756249Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-20T23:55:11.2765419Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:55:11.2765920Z env: 2024-08-20T23:55:11.2766192Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:11.2766650Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:11.2767402Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:11.2768374Z FILE_SUFFIX: test-aot_inductor_torchbench-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026530815 2024-08-20T23:55:11.2769076Z ##[endgroup] 2024-08-20T23:55:11.3033453Z adding: test/test-reports/inference_torchbench.csv (deflated 70%) 2024-08-20T23:55:11.3065753Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-20T23:55:11.3066399Z # Remove any previous usage logs if they exist 2024-08-20T23:55:11.3066914Z rm -f logs-*.zip 2024-08-20T23:55:11.3067590Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-20T23:55:11.3068395Z # so check to see if the file exists first 2024-08-20T23:55:11.3068904Z if [ -f 'usage_log.txt' ]; then 2024-08-20T23:55:11.3069444Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-20T23:55:11.3069942Z fi 2024-08-20T23:55:11.3070312Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-20T23:55:11.3070896Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-20T23:55:11.3071404Z fi 2024-08-20T23:55:11.3080369Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:55:11.3080875Z env: 2024-08-20T23:55:11.3081148Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:11.3081623Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:11.3082380Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:11.3083442Z FILE_SUFFIX: test-aot_inductor_torchbench-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026530815 2024-08-20T23:55:11.3084151Z ##[endgroup] 2024-08-20T23:55:11.3182631Z adding: usage_log.txt (deflated 92%) 2024-08-20T23:55:11.3270653Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-20T23:55:11.3271378Z # Remove any previous debugging artifacts if they exist 2024-08-20T23:55:11.3271931Z rm -f debug-*.zip 2024-08-20T23:55:11.3272306Z if [ -d 'test/debug' ]; then 2024-08-20T23:55:11.3272818Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-20T23:55:11.3273312Z fi 2024-08-20T23:55:11.3281441Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:55:11.3281934Z env: 2024-08-20T23:55:11.3282221Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:11.3282785Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:11.3283534Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:11.3284499Z FILE_SUFFIX: test-aot_inductor_torchbench-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026530815 2024-08-20T23:55:11.3285210Z ##[endgroup] 2024-08-20T23:55:11.3385534Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:55:11.3385966Z with: 2024-08-20T23:55:11.3386246Z s3-bucket: gha-artifacts 2024-08-20T23:55:11.3386679Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:55:11.3387150Z retention-days: 14 2024-08-20T23:55:11.3387488Z if-no-files-found: warn 2024-08-20T23:55:11.3387838Z path: test-jsons-*.zip 2024-08-20T23:55:11.3388183Z name: artifact 2024-08-20T23:55:11.3388487Z region: us-east-1 2024-08-20T23:55:11.3388779Z env: 2024-08-20T23:55:11.3389050Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:11.3389507Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:11.3390245Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:11.3390884Z ##[endgroup] 2024-08-20T23:55:11.7253102Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:55:11.7253986Z With the provided path, there will be 1 file uploaded 2024-08-20T23:55:11.7254782Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:55:11.8653906Z Starting upload of test-jsons-test-aot_inductor_torchbench-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026530815.zip 2024-08-20T23:55:12.0154381Z Finished upload of test-jsons-test-aot_inductor_torchbench-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026530815.zip 2024-08-20T23:55:12.0443166Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:55:12.0443626Z with: 2024-08-20T23:55:12.0443909Z s3-bucket: gha-artifacts 2024-08-20T23:55:12.0444347Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:55:12.0446682Z retention-days: 14 2024-08-20T23:55:12.0447030Z if-no-files-found: error 2024-08-20T23:55:12.0447397Z path: test-reports-*.zip 2024-08-20T23:55:12.0447745Z name: artifact 2024-08-20T23:55:12.0448044Z region: us-east-1 2024-08-20T23:55:12.0448346Z env: 2024-08-20T23:55:12.0448618Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:12.0449075Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:12.0449819Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:12.0450468Z ##[endgroup] 2024-08-20T23:55:12.4045402Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:55:12.4046258Z With the provided path, there will be 1 file uploaded 2024-08-20T23:55:12.4046882Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:55:12.4104010Z Starting upload of test-reports-test-aot_inductor_torchbench-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026530815.zip 2024-08-20T23:55:12.5319509Z Finished upload of test-reports-test-aot_inductor_torchbench-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026530815.zip 2024-08-20T23:55:12.5607853Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:55:12.5608311Z with: 2024-08-20T23:55:12.5608588Z s3-bucket: gha-artifacts 2024-08-20T23:55:12.5609034Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:55:12.5609521Z retention-days: 14 2024-08-20T23:55:12.5609862Z if-no-files-found: ignore 2024-08-20T23:55:12.5610217Z path: logs-*.zip 2024-08-20T23:55:12.5610546Z name: artifact 2024-08-20T23:55:12.5610861Z region: us-east-1 2024-08-20T23:55:12.5611161Z env: 2024-08-20T23:55:12.5611450Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:12.5611916Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:12.5612677Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:12.5613324Z ##[endgroup] 2024-08-20T23:55:12.9201418Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:55:12.9202193Z With the provided path, there will be 1 file uploaded 2024-08-20T23:55:12.9202911Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:55:12.9259037Z Starting upload of logs-test-aot_inductor_torchbench-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026530815.zip 2024-08-20T23:55:13.0720207Z Finished upload of logs-test-aot_inductor_torchbench-2-2-amz2023.linux.g5.4xlarge.nvidia.gpu_29026530815.zip 2024-08-20T23:55:13.1007129Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:55:13.1007574Z with: 2024-08-20T23:55:13.1007853Z s3-bucket: gha-artifacts 2024-08-20T23:55:13.1008298Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T23:55:13.1008782Z retention-days: 14 2024-08-20T23:55:13.1009125Z if-no-files-found: ignore 2024-08-20T23:55:13.1009479Z path: debug-*.zip 2024-08-20T23:55:13.1009812Z name: artifact 2024-08-20T23:55:13.1010117Z region: us-east-1 2024-08-20T23:55:13.1010412Z env: 2024-08-20T23:55:13.1010675Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:13.1011142Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:13.1011893Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:13.1012597Z ##[endgroup] 2024-08-20T23:55:13.4478651Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-20T23:55:13.4763652Z ##[group]Run # shellcheck disable=SC2156 2024-08-20T23:55:13.4764164Z # shellcheck disable=SC2156 2024-08-20T23:55:13.4764960Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-20T23:55:13.4774204Z shell: /usr/bin/bash -e {0} 2024-08-20T23:55:13.4774559Z env: 2024-08-20T23:55:13.4774836Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:13.4775289Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:13.4776040Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:13.4776872Z ##[endgroup] 2024-08-20T23:55:13.7493876Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-20T23:55:13.7494448Z with: 2024-08-20T23:55:13.7494701Z env: 2024-08-20T23:55:13.7494976Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:13.7495425Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:13.7496171Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:13.7496818Z ##[endgroup] 2024-08-20T23:55:13.7516023Z ##[group]Run set -eou pipefail 2024-08-20T23:55:13.7516441Z set -eou pipefail 2024-08-20T23:55:13.7516792Z  2024-08-20T23:55:13.7517315Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-20T23:55:13.7517953Z for _ in $(seq 1440); do 2024-08-20T23:55:13.7518434Z  # Break if no ssh session exists anymore 2024-08-20T23:55:13.7518938Z  if [ "$(who)" = "" ]; then 2024-08-20T23:55:13.7519346Z  break 2024-08-20T23:55:13.7519697Z  fi 2024-08-20T23:55:13.7519994Z  echo "." 2024-08-20T23:55:13.7520322Z  sleep 5 2024-08-20T23:55:13.7520644Z done 2024-08-20T23:55:13.7529550Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:55:13.7530040Z env: 2024-08-20T23:55:13.7530328Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:13.7530794Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:13.7531546Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:13.7532214Z ##[endgroup] 2024-08-20T23:55:13.7562116Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-20T23:55:13.7651197Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:55:13.7652006Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:55:13.7652601Z # shellcheck disable=SC2046 2024-08-20T23:55:13.7653074Z docker stop $(docker ps -q) || true 2024-08-20T23:55:13.7653547Z # Prune all of the docker images 2024-08-20T23:55:13.7654001Z docker system prune -af 2024-08-20T23:55:13.7662234Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:55:13.7662729Z env: 2024-08-20T23:55:13.7662999Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:55:13.7663451Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:55:13.7664197Z DOCKER_CONTAINER_ID: d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:13.7664839Z ##[endgroup] 2024-08-20T23:55:14.4426328Z d9bd82139fb7 2024-08-20T23:55:18.4734833Z Deleted Containers: 2024-08-20T23:55:18.4735549Z d9bd82139fb7f6309f704bd8aacc3001d2e9d7cecc058f2f25f0b99ecd20dc58 2024-08-20T23:55:18.4736095Z 2024-08-20T23:55:27.4028560Z Deleted Images: 2024-08-20T23:55:27.4030579Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T23:55:27.4033609Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-20T23:55:27.4035204Z deleted: sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb 2024-08-20T23:55:27.4036053Z deleted: sha256:c2ce10014dd98f6abbfbf2265f9780a8204c2a7a27463e61ee565573a23d141d 2024-08-20T23:55:27.4036957Z deleted: sha256:ae249c0e6e6013bd6ffea2ca0b0160c015c9ebf7c03070bc2f983a9ff2415bf5 2024-08-20T23:55:27.4037794Z deleted: sha256:8a73883c1067c0e29ff128f553518682b47ed9707d2f35e7d5138e2caf029d61 2024-08-20T23:55:27.4038612Z deleted: sha256:9c040db729fa084584aaf5573082408a2196bc64e5952270c79dc610ed4680b6 2024-08-20T23:55:27.4039432Z deleted: sha256:17311eeeb8b71b16699880c0f646e185107496c4d41da5d23370603768ec030c 2024-08-20T23:55:27.4040251Z deleted: sha256:07cc10bece8501b1c8035d4385ad9e9867dd6e92fa32218a4dcb3e42af7e0411 2024-08-20T23:55:27.4041623Z deleted: sha256:f3e32d05099d1a3115a4722bc2e94a48752c3e965446773673808252671bb967 2024-08-20T23:55:27.4042487Z deleted: sha256:be7c3f41475e3e72ffe3ac2f92ab2f4cf49d5496b2aae738942ac3bb687d742c 2024-08-20T23:55:27.4043421Z deleted: sha256:4fcac490be825129b468cf458089f1fcf8fd56afc44ef4ab27c2eff41ab1ac90 2024-08-20T23:55:27.4044253Z deleted: sha256:31c66d80a2a4792897f267d363c0cf84fb821c5d4a275ab1636f778fd0284c08 2024-08-20T23:55:27.4045065Z deleted: sha256:8a8bcb6085e7e438e5533e8a5613e2ae5888543853568d023af3e2623a1593a9 2024-08-20T23:55:27.4046141Z deleted: sha256:2f6cb60029c79d11567b738f5bdbfb9cb1838cc0f194c4a5e6ae685178d2ed34 2024-08-20T23:55:27.4046973Z deleted: sha256:e546391804d3313f558e1ac16e93cb29a417a92e4eccaf0d4bd12af3bbcf39e3 2024-08-20T23:55:27.4047793Z deleted: sha256:1623b2aa16f2bdc6b572f6799498d9a92fa8114d07c440599b17aea72e22f8b1 2024-08-20T23:55:27.4048626Z deleted: sha256:c0386ca58b7cb7afee015b7e58d2f70525eaf73d6da2cc63868b20e14b1f9497 2024-08-20T23:55:27.4049478Z deleted: sha256:dcbf534ff289186e246531925fed90f72516eb3d63663bad234d611cdc025d62 2024-08-20T23:55:27.4050306Z deleted: sha256:0bab537af579f56c9cd51c7073fd53635fffb424b363236e0de38a7dd0e60ae8 2024-08-20T23:55:27.4051142Z deleted: sha256:d85a852ae1ee027e2fc17240ed68b87957b0a6d210c9119752d8979a0dcd55cf 2024-08-20T23:55:27.4051972Z deleted: sha256:f3821d761b555362e44cf9a1447d8fd2d95b37d54aea67c7519cd9e5db0c4d7e 2024-08-20T23:55:27.4052800Z deleted: sha256:c05e84a545db5dfcb0847c73d2b0f4749fa67aabce05705632ff7f76c84d4688 2024-08-20T23:55:27.4053637Z deleted: sha256:27054eae7ca67b2bd5ca0fd3f4488bf5c5f78e95487d8c45616f3d753d48a8b5 2024-08-20T23:55:27.4054462Z deleted: sha256:9804139f6ae97796aaa993fe4b89ea0cf111eb4d30a79f4c87c339c2e1e04fa0 2024-08-20T23:55:27.4055281Z deleted: sha256:ad6ae2fc594aaa359ff4e287323836570454eb24dd162930655e414d1d77c6dd 2024-08-20T23:55:27.4056101Z deleted: sha256:9cfbb36e328af52a248853a46aab88e266c09e2b44a03e0a42194a031b9b9453 2024-08-20T23:55:27.4056931Z deleted: sha256:a8a842bd7f37b8d50b6d12d6f95f46bbba57953db466fcbde9ba522625f024ef 2024-08-20T23:55:27.4057873Z deleted: sha256:da9738654b3c86f89ebeed2560f1bae093f1219f1eff90b0394b7f211ecabd04 2024-08-20T23:55:27.4058831Z deleted: sha256:cc683bf0cb245394e0f568984b2dd4fce1b6a3a7fe7c44734bae1aa1caaa3251 2024-08-20T23:55:27.4059663Z deleted: sha256:965f220e71c7d1ec19aaf41b1529ab287beb9d86eb8482c4c14094485ab34475 2024-08-20T23:55:27.4060483Z deleted: sha256:da32ecec41a73ce7dd29464f90c7951510a18cd378753659a78cd7fabd10dc36 2024-08-20T23:55:27.4061310Z deleted: sha256:d828d187cee9ad48dbc3702fb61d359619dd8a1ed582878680b51686bf117e5a 2024-08-20T23:55:27.4062131Z deleted: sha256:712111f4b12ff8e2787597d5fdcb043bd0824e55ddd97f262ef550506ca195f0 2024-08-20T23:55:27.4062950Z deleted: sha256:a064eea405958ad14dc6c794e5df9fd1d4515424272801ec559d45a59317309a 2024-08-20T23:55:27.4063762Z deleted: sha256:e124382d136bd5e9a8b409ca79c1f1ef76458deaf444b1b2060d3fa99dbeb506 2024-08-20T23:55:27.4064594Z deleted: sha256:6e10fe757cc190a1b31b47010f4f7610e02cb1b4b438f190dcd301aa4fe2278e 2024-08-20T23:55:27.4065425Z deleted: sha256:4cea352e10f46ca4729f96ac6c4600fc53d4c36e7658c2e0f19f4f309cd4a2a7 2024-08-20T23:55:27.4066278Z deleted: sha256:518bba00fe1d0933053066d21cac0235c9fb485abbc41557ec184b293095c924 2024-08-20T23:55:27.4067125Z deleted: sha256:a64f9ca42e9f27ff2690c15a5c84312e6efaccad45668d9b496cb542165cfb5a 2024-08-20T23:55:27.4067954Z deleted: sha256:096ea0fa14a6c420658f5276b8a8f5d264c0de7e18161ceda2cfc2f4a92cd9bf 2024-08-20T23:55:27.4068792Z deleted: sha256:6f91c94ce6e3f8034c91c66ee25c66a3d85dba9d813b749bd6d2000942a99edc 2024-08-20T23:55:27.4069626Z deleted: sha256:8f8d255eeae546cead8f9a94d237fa57a99940c0f5d51bc29a9d41c7df9cffd9 2024-08-20T23:55:27.4070471Z deleted: sha256:fe05276f4a49f91eef56bfbc3da87cfff5eb116a827fa2d30efe42ce1c134b40 2024-08-20T23:55:27.4071301Z deleted: sha256:5f83cd75d0669c813d4f58978c92e14305cd8f6a07eb9a60665a583460fa7b59 2024-08-20T23:55:27.4072156Z deleted: sha256:8f7ab6366169d39d8953129f650f1927d2fb1328b3f11e2fd9076adb5f46b93a 2024-08-20T23:55:27.4073277Z deleted: sha256:4146cfd5dba0f5308954749f98596b14a1d8f6d23bd93f7ad86c6c3d72239328 2024-08-20T23:55:27.4074125Z deleted: sha256:c6d83cb9ccc606ff63bb3c4d5a5e1ff1450c619b5aae494303a94ae25971cca0 2024-08-20T23:55:27.4074954Z deleted: sha256:71e037e0a3e0963c4565534a7387c082afcb04475d3ac629acf7a0ab2f49fadc 2024-08-20T23:55:27.4075765Z deleted: sha256:4b31135c564686bdaf23262b901237880aea635655721cc7e8c95dc42b05e06d 2024-08-20T23:55:27.4076628Z deleted: sha256:d09bd5c2f35cedcb846543941ee64ce1f684b52e4285696c93b3fb2032da8579 2024-08-20T23:55:27.4077465Z deleted: sha256:4de7fb7d4fc22212aae5fc1ceb637526916d0cced240d7c93d5edd6b0ea28013 2024-08-20T23:55:27.4078289Z deleted: sha256:69890e3e82746e10aff2f1084f11afb4b95a4a6c26b30f64b7001bc0a5fe32bf 2024-08-20T23:55:27.4079103Z deleted: sha256:0156df53b3224e645336ea52b4bb816f1e8d8c9b591748cd645e5e5649fa507b 2024-08-20T23:55:27.4079915Z deleted: sha256:0664624cb9cc4a386f31bf49919a8d8a11d9fea428d05585d9e6228f754bc7cd 2024-08-20T23:55:27.4080753Z deleted: sha256:8c2ddd5fa535c768c058a23f5cc4b6c64cdd71bb7458491282aabc7f7bb94d69 2024-08-20T23:55:27.4081842Z deleted: sha256:8dd03dbe0e1c5f47dd333074dff4bd00415f2f63727a65bf17866774e2ebbd0d 2024-08-20T23:55:27.4082764Z deleted: sha256:3c56b63e094bd18fad9da7a541d537210bca3b730c590d3641213259f6908d6f 2024-08-20T23:55:27.4083589Z deleted: sha256:51484fef07bd1266d4d6ff3d767751172660fc4cc6a1ffe883523a63c985e31e 2024-08-20T23:55:27.4084417Z deleted: sha256:03e83c94023982ab7dd8dc65a6df2024e342f1b760c91ec7a680ee45824869c7 2024-08-20T23:55:27.4085223Z deleted: sha256:58440ab3e8175b5de94a76485c8689a6dc186015475ebf4b386ebba309c5da34 2024-08-20T23:55:27.4086044Z deleted: sha256:9a980f3383e8aa511f0ef68982cb6dfc53b38d1ac8c3d673cb3ba3d45158c91e 2024-08-20T23:55:27.4086884Z deleted: sha256:cb0269c9cf1c47b6c4188ca7be4db261b5d3fc4b4a5153629e222dba7b1f9f98 2024-08-20T23:55:27.4087735Z deleted: sha256:41f1ba870cdffc05e0bf8d87b640fb9d4c153d7f4f3b775f37f94e9f3d584aad 2024-08-20T23:55:27.4088577Z deleted: sha256:7b502ab352f48e8b6eb5e6824a29f308952a5ee7c7d987562e1df7e47639d7f1 2024-08-20T23:55:27.4089399Z deleted: sha256:6ba5217e18738660e6ca8b7a7ec88845e70b0fd781301520bf46c71d99df3144 2024-08-20T23:55:27.4090229Z deleted: sha256:dd5a1c233594780163726bdbbf5de00f7b6b8dca1dace8a001340714c54baedb 2024-08-20T23:55:27.4091066Z deleted: sha256:c27f0db27da73e01f67f6cb78d10f355e350d829479ba4351d82055bb29f6597 2024-08-20T23:55:27.4091897Z deleted: sha256:4e3ce3c854f35c08be3dd95e0d6e718512a885e4614291d7d8b7cd23f48b7dd8 2024-08-20T23:55:27.4092727Z deleted: sha256:c2d6fb088bbec16e60cf0ad74b016f7287c92e659ec2db01271bc309b4dc6d83 2024-08-20T23:55:27.4093573Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-08-20T23:55:27.4094403Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-08-20T23:55:27.4095224Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-08-20T23:55:27.4096054Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-08-20T23:55:27.4096952Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-08-20T23:55:27.4097783Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-08-20T23:55:27.4098630Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-08-20T23:55:27.4099470Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-08-20T23:55:27.4100322Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-08-20T23:55:27.4101170Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-08-20T23:55:27.4102003Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-08-20T23:55:27.4102503Z 2024-08-20T23:55:27.4102659Z Total reclaimed space: 49.95GB 2024-08-20T23:55:27.4171718Z Post job cleanup. 2024-08-20T23:55:27.4222572Z Post job cleanup. 2024-08-20T23:55:27.5118364Z [command]/usr/bin/git version 2024-08-20T23:55:27.5184083Z git version 2.40.1 2024-08-20T23:55:27.5221822Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/7936ea2d-90f8-49db-86d6-c4e8eb87b770' before making global git config changes 2024-08-20T23:55:27.5223091Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T23:55:27.5226987Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T23:55:27.5275007Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T23:55:27.5319007Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T23:55:27.5709694Z Entering 'android/libs/fbjni' 2024-08-20T23:55:27.5777078Z Entering 'third_party/FP16' 2024-08-20T23:55:27.5846216Z Entering 'third_party/FXdiv' 2024-08-20T23:55:27.5918533Z Entering 'third_party/NNPACK' 2024-08-20T23:55:27.5987326Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:55:27.6055346Z Entering 'third_party/XNNPACK' 2024-08-20T23:55:27.6138067Z Entering 'third_party/benchmark' 2024-08-20T23:55:27.6206521Z Entering 'third_party/cpp-httplib' 2024-08-20T23:55:27.6278272Z Entering 'third_party/cpuinfo' 2024-08-20T23:55:27.6346006Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:55:27.6413457Z Entering 'third_party/cutlass' 2024-08-20T23:55:27.6489354Z Entering 'third_party/eigen' 2024-08-20T23:55:27.6562881Z Entering 'third_party/fbgemm' 2024-08-20T23:55:27.6629910Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:55:27.6695426Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:55:27.6761485Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:55:27.6838162Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:55:27.6904953Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:55:27.6974472Z Entering 'third_party/flatbuffers' 2024-08-20T23:55:27.7044640Z Entering 'third_party/fmt' 2024-08-20T23:55:27.7117611Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:55:27.7190372Z Entering 'third_party/gloo' 2024-08-20T23:55:27.7259610Z Entering 'third_party/googletest' 2024-08-20T23:55:27.7327169Z Entering 'third_party/ideep' 2024-08-20T23:55:27.7398264Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:55:27.7473126Z Entering 'third_party/ittapi' 2024-08-20T23:55:27.7542678Z Entering 'third_party/kineto' 2024-08-20T23:55:27.7612010Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:55:27.7677852Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:55:27.7747238Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:55:27.7815346Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:55:27.7885180Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:55:27.7951625Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:55:27.8023111Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:55:27.8091388Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:55:27.8159357Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:55:27.8228371Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:55:27.8298474Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:55:27.8364858Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:55:27.8434360Z Entering 'third_party/mimalloc' 2024-08-20T23:55:27.8505744Z Entering 'third_party/nccl/nccl' 2024-08-20T23:55:27.8575623Z Entering 'third_party/nlohmann' 2024-08-20T23:55:27.8643972Z Entering 'third_party/onnx' 2024-08-20T23:55:27.8726213Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:55:27.8792644Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:55:27.8869731Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:55:27.8938346Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:55:27.9004521Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:55:27.9073259Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:55:27.9139909Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:55:27.9210523Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:55:27.9276244Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:55:27.9341423Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:55:27.9406164Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:55:27.9475309Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:55:27.9543683Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:55:27.9635033Z Entering 'third_party/pocketfft' 2024-08-20T23:55:27.9703815Z Entering 'third_party/protobuf' 2024-08-20T23:55:27.9774766Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:55:27.9839777Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:55:27.9910145Z Entering 'third_party/psimd' 2024-08-20T23:55:27.9978721Z Entering 'third_party/pthreadpool' 2024-08-20T23:55:28.0048330Z Entering 'third_party/pybind11' 2024-08-20T23:55:28.0117384Z Entering 'third_party/python-peachpy' 2024-08-20T23:55:28.0184909Z Entering 'third_party/sleef' 2024-08-20T23:55:28.0252817Z Entering 'third_party/tensorpipe' 2024-08-20T23:55:28.0319089Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:55:28.0386154Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:55:28.0453165Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:55:28.0518589Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:55:28.0582341Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:55:28.0676519Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T23:55:28.0711998Z http.https://github.com/.extraheader 2024-08-20T23:55:28.0721064Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-20T23:55:28.0766972Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T23:55:28.1134197Z Entering 'android/libs/fbjni' 2024-08-20T23:55:28.1178448Z http.https://github.com/.extraheader 2024-08-20T23:55:28.1223460Z Entering 'third_party/FP16' 2024-08-20T23:55:28.1268518Z http.https://github.com/.extraheader 2024-08-20T23:55:28.1311946Z Entering 'third_party/FXdiv' 2024-08-20T23:55:28.1357246Z http.https://github.com/.extraheader 2024-08-20T23:55:28.1402558Z Entering 'third_party/NNPACK' 2024-08-20T23:55:28.1446900Z http.https://github.com/.extraheader 2024-08-20T23:55:28.1491109Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:55:28.1534754Z http.https://github.com/.extraheader 2024-08-20T23:55:28.1580636Z Entering 'third_party/XNNPACK' 2024-08-20T23:55:28.1624772Z http.https://github.com/.extraheader 2024-08-20T23:55:28.1686923Z Entering 'third_party/benchmark' 2024-08-20T23:55:28.1730865Z http.https://github.com/.extraheader 2024-08-20T23:55:28.1775191Z Entering 'third_party/cpp-httplib' 2024-08-20T23:55:28.1818856Z http.https://github.com/.extraheader 2024-08-20T23:55:28.1863099Z Entering 'third_party/cpuinfo' 2024-08-20T23:55:28.1906890Z http.https://github.com/.extraheader 2024-08-20T23:55:28.1953008Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:55:28.1996355Z http.https://github.com/.extraheader 2024-08-20T23:55:28.2041076Z Entering 'third_party/cutlass' 2024-08-20T23:55:28.2085484Z http.https://github.com/.extraheader 2024-08-20T23:55:28.2137451Z Entering 'third_party/eigen' 2024-08-20T23:55:28.2181775Z http.https://github.com/.extraheader 2024-08-20T23:55:28.2227114Z Entering 'third_party/fbgemm' 2024-08-20T23:55:28.2271228Z http.https://github.com/.extraheader 2024-08-20T23:55:28.2314057Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:55:28.2367149Z http.https://github.com/.extraheader 2024-08-20T23:55:28.2409327Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:55:28.2456979Z http.https://github.com/.extraheader 2024-08-20T23:55:28.2500351Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:55:28.2544558Z http.https://github.com/.extraheader 2024-08-20T23:55:28.2595700Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:55:28.2639175Z http.https://github.com/.extraheader 2024-08-20T23:55:28.2682846Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:55:28.2725232Z http.https://github.com/.extraheader 2024-08-20T23:55:28.2770728Z Entering 'third_party/flatbuffers' 2024-08-20T23:55:28.2816134Z http.https://github.com/.extraheader 2024-08-20T23:55:28.2863647Z Entering 'third_party/fmt' 2024-08-20T23:55:28.2907973Z http.https://github.com/.extraheader 2024-08-20T23:55:28.2952212Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:55:28.2995108Z http.https://github.com/.extraheader 2024-08-20T23:55:28.3039429Z Entering 'third_party/gloo' 2024-08-20T23:55:28.3083169Z http.https://github.com/.extraheader 2024-08-20T23:55:28.3126164Z Entering 'third_party/googletest' 2024-08-20T23:55:28.3170406Z http.https://github.com/.extraheader 2024-08-20T23:55:28.3213936Z Entering 'third_party/ideep' 2024-08-20T23:55:28.3259318Z http.https://github.com/.extraheader 2024-08-20T23:55:28.3301350Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:55:28.3345513Z http.https://github.com/.extraheader 2024-08-20T23:55:28.3402027Z Entering 'third_party/ittapi' 2024-08-20T23:55:28.3445447Z http.https://github.com/.extraheader 2024-08-20T23:55:28.3490659Z Entering 'third_party/kineto' 2024-08-20T23:55:28.3533903Z http.https://github.com/.extraheader 2024-08-20T23:55:28.3576671Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:55:28.3619536Z http.https://github.com/.extraheader 2024-08-20T23:55:28.3663019Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:55:28.3706109Z http.https://github.com/.extraheader 2024-08-20T23:55:28.3754648Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:55:28.3797741Z http.https://github.com/.extraheader 2024-08-20T23:55:28.3843572Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:55:28.3888756Z http.https://github.com/.extraheader 2024-08-20T23:55:28.3933803Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:55:28.3977045Z http.https://github.com/.extraheader 2024-08-20T23:55:28.4019283Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:55:28.4062998Z http.https://github.com/.extraheader 2024-08-20T23:55:28.4110924Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:55:28.4157043Z http.https://github.com/.extraheader 2024-08-20T23:55:28.4200917Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:55:28.4244165Z http.https://github.com/.extraheader 2024-08-20T23:55:28.4287704Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:55:28.4330384Z http.https://github.com/.extraheader 2024-08-20T23:55:28.4379107Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:55:28.4423587Z http.https://github.com/.extraheader 2024-08-20T23:55:28.4470259Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:55:28.4513356Z http.https://github.com/.extraheader 2024-08-20T23:55:28.4557927Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:55:28.4601050Z http.https://github.com/.extraheader 2024-08-20T23:55:28.4649974Z Entering 'third_party/mimalloc' 2024-08-20T23:55:28.4694468Z http.https://github.com/.extraheader 2024-08-20T23:55:28.4738610Z Entering 'third_party/nccl/nccl' 2024-08-20T23:55:28.4783196Z http.https://github.com/.extraheader 2024-08-20T23:55:28.4826845Z Entering 'third_party/nlohmann' 2024-08-20T23:55:28.4871234Z http.https://github.com/.extraheader 2024-08-20T23:55:28.4915834Z Entering 'third_party/onnx' 2024-08-20T23:55:28.4961287Z http.https://github.com/.extraheader 2024-08-20T23:55:28.5018702Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:55:28.5061731Z http.https://github.com/.extraheader 2024-08-20T23:55:28.5105799Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:55:28.5149272Z http.https://github.com/.extraheader 2024-08-20T23:55:28.5197295Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:55:28.5241910Z http.https://github.com/.extraheader 2024-08-20T23:55:28.5286571Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:55:28.5328710Z http.https://github.com/.extraheader 2024-08-20T23:55:28.5371561Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:55:28.5415697Z http.https://github.com/.extraheader 2024-08-20T23:55:28.5458759Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:55:28.5501235Z http.https://github.com/.extraheader 2024-08-20T23:55:28.5543535Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:55:28.5586952Z http.https://github.com/.extraheader 2024-08-20T23:55:28.5633194Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:55:28.5675943Z http.https://github.com/.extraheader 2024-08-20T23:55:28.5718115Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:55:28.5765481Z http.https://github.com/.extraheader 2024-08-20T23:55:28.5808115Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:55:28.5851312Z http.https://github.com/.extraheader 2024-08-20T23:55:28.5895036Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:55:28.5938057Z http.https://github.com/.extraheader 2024-08-20T23:55:28.5984273Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:55:28.6026599Z http.https://github.com/.extraheader 2024-08-20T23:55:28.6074189Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:55:28.6117764Z http.https://github.com/.extraheader 2024-08-20T23:55:28.6181708Z Entering 'third_party/pocketfft' 2024-08-20T23:55:28.6226072Z http.https://github.com/.extraheader 2024-08-20T23:55:28.6270139Z Entering 'third_party/protobuf' 2024-08-20T23:55:28.6314165Z http.https://github.com/.extraheader 2024-08-20T23:55:28.6959073Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:55:28.7004796Z http.https://github.com/.extraheader 2024-08-20T23:55:28.7046739Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:55:28.7091388Z http.https://github.com/.extraheader 2024-08-20T23:55:28.7139767Z Entering 'third_party/psimd' 2024-08-20T23:55:28.7185060Z http.https://github.com/.extraheader 2024-08-20T23:55:28.7228979Z Entering 'third_party/pthreadpool' 2024-08-20T23:55:28.7276969Z http.https://github.com/.extraheader 2024-08-20T23:55:28.7321250Z Entering 'third_party/pybind11' 2024-08-20T23:55:28.7366212Z http.https://github.com/.extraheader 2024-08-20T23:55:28.7410436Z Entering 'third_party/python-peachpy' 2024-08-20T23:55:28.7455333Z http.https://github.com/.extraheader 2024-08-20T23:55:28.7499214Z Entering 'third_party/sleef' 2024-08-20T23:55:28.7543496Z http.https://github.com/.extraheader 2024-08-20T23:55:28.7589104Z Entering 'third_party/tensorpipe' 2024-08-20T23:55:28.7632370Z http.https://github.com/.extraheader 2024-08-20T23:55:28.7675853Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:55:28.7718620Z http.https://github.com/.extraheader 2024-08-20T23:55:28.7761136Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:55:28.7805880Z http.https://github.com/.extraheader 2024-08-20T23:55:28.7848220Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:55:28.7894686Z http.https://github.com/.extraheader 2024-08-20T23:55:28.7936846Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:55:28.7979429Z http.https://github.com/.extraheader 2024-08-20T23:55:28.8019864Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:55:28.8063728Z http.https://github.com/.extraheader 2024-08-20T23:55:28.8227013Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-20T23:55:28.8248257Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-20T23:55:28.8256375Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:55:28.8256950Z ##[endgroup] 2024-08-20T23:55:35.7944846Z Cleaning up orphan processes