2024-06-26T07:01:04.4408448Z Current runner version: '2.317.0' 2024-06-26T07:01:04.4414656Z Runner name: 'i-083295e4fef5bcb20' 2024-06-26T07:01:04.4415525Z Runner group name: 'Default' 2024-06-26T07:01:04.4416521Z Machine name: 'ip-10-0-72-207' 2024-06-26T07:01:04.4421455Z ##[group]GITHUB_TOKEN Permissions 2024-06-26T07:01:04.4423493Z Actions: read 2024-06-26T07:01:04.4424058Z Attestations: read 2024-06-26T07:01:04.4424734Z Checks: read 2024-06-26T07:01:04.4425315Z Contents: read 2024-06-26T07:01:04.4425879Z Deployments: read 2024-06-26T07:01:04.4426446Z Discussions: read 2024-06-26T07:01:04.4427242Z Issues: read 2024-06-26T07:01:04.4427789Z Metadata: read 2024-06-26T07:01:04.4428321Z Packages: read 2024-06-26T07:01:04.4428895Z Pages: read 2024-06-26T07:01:04.4429432Z PullRequests: read 2024-06-26T07:01:04.4430088Z RepositoryProjects: read 2024-06-26T07:01:04.4430717Z SecurityEvents: read 2024-06-26T07:01:04.4431303Z Statuses: read 2024-06-26T07:01:04.4431866Z ##[endgroup] 2024-06-26T07:01:04.4435081Z Secret source: Actions 2024-06-26T07:01:04.4435993Z Prepare workflow directory 2024-06-26T07:01:04.8095273Z Prepare all required actions 2024-06-26T07:01:04.8282428Z Getting action download info 2024-06-26T07:01:05.0023132Z Download action repository 'pytorch/test-infra@main' (SHA:43a2ce341cc31288e9a38b65ce600a7f43021bd5) 2024-06-26T07:01:05.3214647Z Download action repository 'pytorch/pytorch@main' (SHA:c422a9549d2b320b47a9391f6cbcf592b2876c88) 2024-06-26T07:01:08.0622653Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-06-26T07:01:08.2165038Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-06-26T07:01:08.5172394Z Getting action download info 2024-06-26T07:01:08.6102648Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-06-26T07:01:08.7792484Z Getting action download info 2024-06-26T07:01:08.8879437Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-06-26T07:01:09.0396005Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/129470 (b8c4c54d347aa776934c60784e35936878ef18dc) 2024-06-26T07:01:09.0398238Z ##[group] Inputs 2024-06-26T07:01:09.0398746Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:01:09.0407454Z test-matrix: {"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-06-26T07:01:09.0416934Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:01:09.0418186Z sync-tag: 2024-06-26T07:01:09.0419036Z timeout-minutes: 240 2024-06-26T07:01:09.0419394Z use-gha: 2024-06-26T07:01:09.0419692Z dashboard-tag: 2024-06-26T07:01:09.0420027Z s3-bucket: gha-artifacts 2024-06-26T07:01:09.0420400Z aws-role-to-assume: 2024-06-26T07:01:09.0420749Z ##[endgroup] 2024-06-26T07:01:09.0421631Z Complete job name: cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:01:09.0988131Z A job started hook has been configured by the self-hosted runner administrator 2024-06-26T07:01:09.1126189Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T07:01:09.1136551Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:01:09.1137102Z ##[endgroup] 2024-06-26T07:01:10.2723294Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-06-26T07:01:10.2723852Z with: 2024-06-26T07:01:10.2724563Z github-secret: *** 2024-06-26T07:01:10.2725519Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-06-26T07:01:10.2726529Z activate-with-label: false 2024-06-26T07:01:10.2726893Z label: with-ssh 2024-06-26T07:01:10.2727226Z remove-existing-keys: true 2024-06-26T07:01:10.2727630Z fail-silently: true 2024-06-26T07:01:10.2727943Z env: 2024-06-26T07:01:10.2728244Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:01:10.2728601Z ##[endgroup] 2024-06-26T07:01:10.3602681Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-06-26T07:01:10.3603728Z ciflow reference detected, attempting to extract PR number 2024-06-26T07:01:10.6531315Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-06-26T07:01:10.7484681Z No SSH keys found for user pytorch-bot[bot] 2024-06-26T07:01:10.7485424Z Grabbing public ssh keys from https://github.com/leslie-fang-intel.keys 2024-06-26T07:01:10.8181548Z No SSH keys found for user leslie-fang-intel 2024-06-26T07:01:10.8287259Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-06-26T07:01:10.8287910Z with: 2024-06-26T07:01:10.8288222Z submodules: recursive 2024-06-26T07:01:10.8288615Z fetch-depth: 0 2024-06-26T07:01:10.8288976Z env: 2024-06-26T07:01:10.8289293Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:01:10.8289669Z ##[endgroup] 2024-06-26T07:01:10.8508314Z ##[group]Run retry () { 2024-06-26T07:01:10.8508712Z retry () { 2024-06-26T07:01:10.8509243Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-06-26T07:01:10.8509822Z } 2024-06-26T07:01:10.8510141Z echo "${GITHUB_WORKSPACE}" 2024-06-26T07:01:10.8510565Z if [ -z "${NO_SUDO}" ]; then 2024-06-26T07:01:10.8511054Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T07:01:10.8511519Z else 2024-06-26T07:01:10.8511868Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T07:01:10.8512303Z fi 2024-06-26T07:01:10.8512647Z mkdir "${GITHUB_WORKSPACE}" 2024-06-26T07:01:10.8520764Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:01:10.8521286Z env: 2024-06-26T07:01:10.8521564Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:01:10.8521920Z NO_SUDO: 2024-06-26T07:01:10.8522196Z ##[endgroup] 2024-06-26T07:01:10.8548032Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:01:13.1423408Z ##[group]Run malfet/checkout@silent-checkout 2024-06-26T07:01:13.1423856Z with: 2024-06-26T07:01:13.1424277Z ref: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:01:13.1424732Z fetch-depth: 0 2024-06-26T07:01:13.1425051Z submodules: recursive 2024-06-26T07:01:13.1425390Z quiet-checkout: true 2024-06-26T07:01:13.1425745Z repository: pytorch/pytorch 2024-06-26T07:01:13.1426238Z token: *** 2024-06-26T07:01:13.1426733Z ssh-strict: true 2024-06-26T07:01:13.1427070Z persist-credentials: true 2024-06-26T07:01:13.1427433Z clean: true 2024-06-26T07:01:13.1427774Z sparse-checkout-cone-mode: true 2024-06-26T07:01:13.1428169Z lfs: false 2024-06-26T07:01:13.1428473Z set-safe-directory: true 2024-06-26T07:01:13.1428825Z env: 2024-06-26T07:01:13.1429107Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:01:13.1429452Z ##[endgroup] 2024-06-26T07:01:13.2349553Z Syncing repository: pytorch/pytorch 2024-06-26T07:01:13.2351170Z ##[group]Getting Git version info 2024-06-26T07:01:13.2351963Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T07:01:13.2352834Z [command]/usr/bin/git version 2024-06-26T07:01:13.2353216Z git version 2.40.1 2024-06-26T07:01:13.2354546Z ##[endgroup] 2024-06-26T07:01:13.2365631Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/ec446aed-8138-44e5-8cbd-77bd1e3de3f3' before making global git config changes 2024-06-26T07:01:13.2366916Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T07:01:13.2370078Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:01:13.2396736Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T07:01:13.2400800Z ##[group]Initializing the repository 2024-06-26T07:01:13.2403781Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:01:13.2421190Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-06-26T07:01:13.2422307Z hint: is subject to change. To configure the initial branch name to use in all 2024-06-26T07:01:13.2423112Z hint: of your new repositories, which will suppress this warning, call: 2024-06-26T07:01:13.2423688Z hint: 2024-06-26T07:01:13.2424297Z hint: git config --global init.defaultBranch 2024-06-26T07:01:13.2424779Z hint: 2024-06-26T07:01:13.2425298Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-06-26T07:01:13.2426146Z hint: 'development'. The just-created branch can be renamed via this command: 2024-06-26T07:01:13.2427054Z hint: 2024-06-26T07:01:13.2427375Z hint: git branch -m 2024-06-26T07:01:13.2428125Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-06-26T07:01:13.2429899Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-06-26T07:01:13.2455614Z ##[endgroup] 2024-06-26T07:01:13.2456231Z ##[group]Disabling automatic garbage collection 2024-06-26T07:01:13.2459103Z [command]/usr/bin/git config --local gc.auto 0 2024-06-26T07:01:13.2483294Z ##[endgroup] 2024-06-26T07:01:13.2483835Z ##[group]Setting up auth 2024-06-26T07:01:13.2489421Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T07:01:13.2515747Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T07:01:13.2711099Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T07:01:13.2737749Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T07:01:13.2936500Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:01:13.2975570Z ##[endgroup] 2024-06-26T07:01:13.2976130Z ##[group]Fetching the repository 2024-06-26T07:01:13.2981437Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-06-26T07:01:15.5596129Z remote: Enumerating objects: 987392 2024-06-26T07:01:15.5596806Z remote: Enumerating objects: 990209, done. 2024-06-26T07:01:15.5609603Z remote: Counting objects: 0% (1/2817) 2024-06-26T07:01:15.5610311Z remote: Counting objects: 1% (29/2817) 2024-06-26T07:01:15.5611061Z remote: Counting objects: 2% (57/2817) 2024-06-26T07:01:15.5611611Z remote: Counting objects: 3% (85/2817) 2024-06-26T07:01:15.5612135Z remote: Counting objects: 4% (113/2817) 2024-06-26T07:01:15.5612645Z remote: Counting objects: 5% (141/2817) 2024-06-26T07:01:15.5613168Z remote: Counting objects: 6% (170/2817) 2024-06-26T07:01:15.5613687Z remote: Counting objects: 7% (198/2817) 2024-06-26T07:01:15.5614193Z remote: Counting objects: 8% (226/2817) 2024-06-26T07:01:15.5614691Z remote: Counting objects: 9% (254/2817) 2024-06-26T07:01:15.5615208Z remote: Counting objects: 10% (282/2817) 2024-06-26T07:01:15.5615717Z remote: Counting objects: 11% (310/2817) 2024-06-26T07:01:15.5616222Z remote: Counting objects: 12% (339/2817) 2024-06-26T07:01:15.5616716Z remote: Counting objects: 13% (367/2817) 2024-06-26T07:01:15.5617229Z remote: Counting objects: 14% (395/2817) 2024-06-26T07:01:15.5617744Z remote: Counting objects: 15% (423/2817) 2024-06-26T07:01:15.5618250Z remote: Counting objects: 16% (451/2817) 2024-06-26T07:01:15.5618746Z remote: Counting objects: 17% (479/2817) 2024-06-26T07:01:15.5619255Z remote: Counting objects: 18% (508/2817) 2024-06-26T07:01:15.5619761Z remote: Counting objects: 19% (536/2817) 2024-06-26T07:01:15.5620266Z remote: Counting objects: 20% (564/2817) 2024-06-26T07:01:15.5620949Z remote: Counting objects: 21% (592/2817) 2024-06-26T07:01:15.5621489Z remote: Counting objects: 22% (620/2817) 2024-06-26T07:01:15.5622259Z remote: Counting objects: 23% (648/2817) 2024-06-26T07:01:15.5622986Z remote: Counting objects: 24% (677/2817) 2024-06-26T07:01:15.5623603Z remote: Counting objects: 25% (705/2817) 2024-06-26T07:01:15.5624198Z remote: Counting objects: 26% (733/2817) 2024-06-26T07:01:15.5624735Z remote: Counting objects: 27% (761/2817) 2024-06-26T07:01:15.5625588Z remote: Counting objects: 28% (789/2817) 2024-06-26T07:01:15.5626268Z remote: Counting objects: 29% (817/2817) 2024-06-26T07:01:15.5627077Z remote: Counting objects: 30% (846/2817) 2024-06-26T07:01:15.5627705Z remote: Counting objects: 31% (874/2817) 2024-06-26T07:01:15.5628413Z remote: Counting objects: 32% (902/2817) 2024-06-26T07:01:15.5628964Z remote: Counting objects: 33% (930/2817) 2024-06-26T07:01:15.5629481Z remote: Counting objects: 34% (958/2817) 2024-06-26T07:01:15.5630058Z remote: Counting objects: 35% (986/2817) 2024-06-26T07:01:15.5630743Z remote: Counting objects: 36% (1015/2817) 2024-06-26T07:01:15.5631373Z remote: Counting objects: 37% (1043/2817) 2024-06-26T07:01:15.5632025Z remote: Counting objects: 38% (1071/2817) 2024-06-26T07:01:15.5632648Z remote: Counting objects: 39% (1099/2817) 2024-06-26T07:01:15.5633180Z remote: Counting objects: 40% (1127/2817) 2024-06-26T07:01:15.5633701Z remote: Counting objects: 41% (1155/2817) 2024-06-26T07:01:15.5634222Z remote: Counting objects: 42% (1184/2817) 2024-06-26T07:01:15.5634846Z remote: Counting objects: 43% (1212/2817) 2024-06-26T07:01:15.5635477Z remote: Counting objects: 44% (1240/2817) 2024-06-26T07:01:15.5636129Z remote: Counting objects: 45% (1268/2817) 2024-06-26T07:01:15.5636859Z remote: Counting objects: 46% (1296/2817) 2024-06-26T07:01:15.5637421Z remote: Counting objects: 47% (1324/2817) 2024-06-26T07:01:15.5638106Z remote: Counting objects: 48% (1353/2817) 2024-06-26T07:01:15.5638625Z remote: Counting objects: 49% (1381/2817) 2024-06-26T07:01:15.5639148Z remote: Counting objects: 50% (1409/2817) 2024-06-26T07:01:15.5639830Z remote: Counting objects: 51% (1437/2817) 2024-06-26T07:01:15.5640364Z remote: Counting objects: 52% (1465/2817) 2024-06-26T07:01:15.5641045Z remote: Counting objects: 53% (1494/2817) 2024-06-26T07:01:15.5641584Z remote: Counting objects: 54% (1522/2817) 2024-06-26T07:01:15.5642327Z remote: Counting objects: 55% (1550/2817) 2024-06-26T07:01:15.5643053Z remote: Counting objects: 56% (1578/2817) 2024-06-26T07:01:15.5643586Z remote: Counting objects: 57% (1606/2817) 2024-06-26T07:01:15.5644109Z remote: Counting objects: 58% (1634/2817) 2024-06-26T07:01:15.5644649Z remote: Counting objects: 59% (1663/2817) 2024-06-26T07:01:15.5645166Z remote: Counting objects: 60% (1691/2817) 2024-06-26T07:01:15.5645684Z remote: Counting objects: 61% (1719/2817) 2024-06-26T07:01:15.5646254Z remote: Counting objects: 62% (1747/2817) 2024-06-26T07:01:15.5646915Z remote: Counting objects: 63% (1775/2817) 2024-06-26T07:01:15.5647446Z remote: Counting objects: 64% (1803/2817) 2024-06-26T07:01:15.5647988Z remote: Counting objects: 65% (1832/2817) 2024-06-26T07:01:15.5648698Z remote: Counting objects: 66% (1860/2817) 2024-06-26T07:01:15.5649407Z remote: Counting objects: 67% (1888/2817) 2024-06-26T07:01:15.5649962Z remote: Counting objects: 68% (1916/2817) 2024-06-26T07:01:15.5650672Z remote: Counting objects: 69% (1944/2817) 2024-06-26T07:01:15.5651348Z remote: Counting objects: 70% (1972/2817) 2024-06-26T07:01:15.5652105Z remote: Counting objects: 71% (2001/2817) 2024-06-26T07:01:15.5652638Z remote: Counting objects: 72% (2029/2817) 2024-06-26T07:01:15.5653239Z remote: Counting objects: 73% (2057/2817) 2024-06-26T07:01:15.5653933Z remote: Counting objects: 74% (2085/2817) 2024-06-26T07:01:15.5654568Z remote: Counting objects: 75% (2113/2817) 2024-06-26T07:01:15.5655100Z remote: Counting objects: 76% (2141/2817) 2024-06-26T07:01:15.5655627Z remote: Counting objects: 77% (2170/2817) 2024-06-26T07:01:15.5656289Z remote: Counting objects: 78% (2198/2817) 2024-06-26T07:01:15.5656981Z remote: Counting objects: 79% (2226/2817) 2024-06-26T07:01:15.5657527Z remote: Counting objects: 80% (2254/2817) 2024-06-26T07:01:15.5658250Z remote: Counting objects: 81% (2282/2817) 2024-06-26T07:01:15.5658774Z remote: Counting objects: 82% (2310/2817) 2024-06-26T07:01:15.5659305Z remote: Counting objects: 83% (2339/2817) 2024-06-26T07:01:15.5659840Z remote: Counting objects: 84% (2367/2817) 2024-06-26T07:01:15.5660364Z remote: Counting objects: 85% (2395/2817) 2024-06-26T07:01:15.5660892Z remote: Counting objects: 86% (2423/2817) 2024-06-26T07:01:15.5661597Z remote: Counting objects: 87% (2451/2817) 2024-06-26T07:01:15.5662131Z remote: Counting objects: 88% (2479/2817) 2024-06-26T07:01:15.5662691Z remote: Counting objects: 89% (2508/2817) 2024-06-26T07:01:15.5663233Z remote: Counting objects: 90% (2536/2817) 2024-06-26T07:01:15.5663760Z remote: Counting objects: 91% (2564/2817) 2024-06-26T07:01:15.5664390Z remote: Counting objects: 92% (2592/2817) 2024-06-26T07:01:15.5664922Z remote: Counting objects: 93% (2620/2817) 2024-06-26T07:01:15.5665446Z remote: Counting objects: 94% (2648/2817) 2024-06-26T07:01:15.5665973Z remote: Counting objects: 95% (2677/2817) 2024-06-26T07:01:15.5666633Z remote: Counting objects: 96% (2705/2817) 2024-06-26T07:01:15.5667280Z remote: Counting objects: 97% (2733/2817) 2024-06-26T07:01:15.5667801Z remote: Counting objects: 98% (2761/2817) 2024-06-26T07:01:15.5668328Z remote: Counting objects: 99% (2789/2817) 2024-06-26T07:01:15.5668864Z remote: Counting objects: 100% (2817/2817) 2024-06-26T07:01:15.5669431Z remote: Counting objects: 100% (2817/2817), done. 2024-06-26T07:01:15.5830714Z remote: Compressing objects: 0% (1/1140) 2024-06-26T07:01:15.6399782Z remote: Compressing objects: 1% (12/1140) 2024-06-26T07:01:15.6953874Z remote: Compressing objects: 2% (23/1140) 2024-06-26T07:01:15.7318997Z remote: Compressing objects: 3% (35/1140) 2024-06-26T07:01:15.7598995Z remote: Compressing objects: 4% (46/1140) 2024-06-26T07:01:15.7911460Z remote: Compressing objects: 5% (57/1140) 2024-06-26T07:01:15.8153613Z remote: Compressing objects: 6% (69/1140) 2024-06-26T07:01:15.9419953Z remote: Compressing objects: 7% (80/1140) 2024-06-26T07:01:16.0051152Z remote: Compressing objects: 8% (92/1140) 2024-06-26T07:01:16.0544884Z remote: Compressing objects: 9% (103/1140) 2024-06-26T07:01:16.0717838Z remote: Compressing objects: 10% (114/1140) 2024-06-26T07:01:16.0821041Z remote: Compressing objects: 11% (126/1140) 2024-06-26T07:01:16.0900408Z remote: Compressing objects: 12% (137/1140) 2024-06-26T07:01:16.0949022Z remote: Compressing objects: 13% (149/1140) 2024-06-26T07:01:16.0949605Z remote: Compressing objects: 14% (160/1140) 2024-06-26T07:01:16.1021555Z remote: Compressing objects: 15% (171/1140) 2024-06-26T07:01:16.1022646Z remote: Compressing objects: 16% (183/1140) 2024-06-26T07:01:16.1027643Z remote: Compressing objects: 17% (194/1140) 2024-06-26T07:01:16.1053866Z remote: Compressing objects: 18% (206/1140) 2024-06-26T07:01:16.1081112Z remote: Compressing objects: 19% (217/1140) 2024-06-26T07:01:16.1105870Z remote: Compressing objects: 20% (228/1140) 2024-06-26T07:01:16.1119925Z remote: Compressing objects: 21% (240/1140) 2024-06-26T07:01:16.1136569Z remote: Compressing objects: 22% (251/1140) 2024-06-26T07:01:16.1150515Z remote: Compressing objects: 23% (263/1140) 2024-06-26T07:01:16.1158883Z remote: Compressing objects: 24% (274/1140) 2024-06-26T07:01:16.1168896Z remote: Compressing objects: 25% (285/1140) 2024-06-26T07:01:16.1177242Z remote: Compressing objects: 26% (297/1140) 2024-06-26T07:01:16.1183075Z remote: Compressing objects: 27% (308/1140) 2024-06-26T07:01:16.1187120Z remote: Compressing objects: 28% (320/1140) 2024-06-26T07:01:16.1195676Z remote: Compressing objects: 29% (331/1140) 2024-06-26T07:01:16.1209845Z remote: Compressing objects: 30% (342/1140) 2024-06-26T07:01:16.1213081Z remote: Compressing objects: 31% (354/1140) 2024-06-26T07:01:16.1228318Z remote: Compressing objects: 32% (365/1140) 2024-06-26T07:01:16.1241149Z remote: Compressing objects: 33% (377/1140) 2024-06-26T07:01:16.1256733Z remote: Compressing objects: 34% (388/1140) 2024-06-26T07:01:16.1270519Z remote: Compressing objects: 35% (399/1140) 2024-06-26T07:01:16.1293105Z remote: Compressing objects: 36% (411/1140) 2024-06-26T07:01:16.1312864Z remote: Compressing objects: 37% (422/1140) 2024-06-26T07:01:16.1326994Z remote: Compressing objects: 38% (434/1140) 2024-06-26T07:01:16.1341006Z remote: Compressing objects: 39% (445/1140) 2024-06-26T07:01:16.1358582Z remote: Compressing objects: 40% (456/1140) 2024-06-26T07:01:16.1389517Z remote: Compressing objects: 41% (468/1140) 2024-06-26T07:01:16.1409875Z remote: Compressing objects: 42% (479/1140) 2024-06-26T07:01:16.1420437Z remote: Compressing objects: 43% (491/1140) 2024-06-26T07:01:16.1430086Z remote: Compressing objects: 44% (502/1140) 2024-06-26T07:01:16.1436857Z remote: Compressing objects: 45% (513/1140) 2024-06-26T07:01:16.1445623Z remote: Compressing objects: 46% (525/1140) 2024-06-26T07:01:16.1460562Z remote: Compressing objects: 47% (536/1140) 2024-06-26T07:01:16.1461105Z remote: Compressing objects: 48% (548/1140) 2024-06-26T07:01:16.1463027Z remote: Compressing objects: 49% (559/1140) 2024-06-26T07:01:16.1468400Z remote: Compressing objects: 50% (570/1140) 2024-06-26T07:01:16.1482408Z remote: Compressing objects: 51% (582/1140) 2024-06-26T07:01:16.1508706Z remote: Compressing objects: 52% (593/1140) 2024-06-26T07:01:16.1517923Z remote: Compressing objects: 53% (605/1140) 2024-06-26T07:01:16.1540660Z remote: Compressing objects: 54% (616/1140) 2024-06-26T07:01:16.1547053Z remote: Compressing objects: 55% (627/1140) 2024-06-26T07:01:16.1555365Z remote: Compressing objects: 56% (639/1140) 2024-06-26T07:01:16.1564722Z remote: Compressing objects: 57% (650/1140) 2024-06-26T07:01:16.1577186Z remote: Compressing objects: 58% (662/1140) 2024-06-26T07:01:16.1594132Z remote: Compressing objects: 59% (673/1140) 2024-06-26T07:01:16.1600006Z remote: Compressing objects: 60% (684/1140) 2024-06-26T07:01:16.1607600Z remote: Compressing objects: 61% (696/1140) 2024-06-26T07:01:16.1620199Z remote: Compressing objects: 62% (707/1140) 2024-06-26T07:01:16.1635077Z remote: Compressing objects: 63% (719/1140) 2024-06-26T07:01:16.1642895Z remote: Compressing objects: 64% (730/1140) 2024-06-26T07:01:16.1652725Z remote: Compressing objects: 65% (741/1140) 2024-06-26T07:01:16.1666666Z remote: Compressing objects: 66% (753/1140) 2024-06-26T07:01:16.1672182Z remote: Compressing objects: 67% (764/1140) 2024-06-26T07:01:16.1672788Z remote: Compressing objects: 68% (776/1140) 2024-06-26T07:01:16.1683886Z remote: Compressing objects: 69% (787/1140) 2024-06-26T07:01:16.1685043Z remote: Compressing objects: 70% (798/1140) 2024-06-26T07:01:16.1688488Z remote: Compressing objects: 71% (810/1140) 2024-06-26T07:01:16.1692059Z remote: Compressing objects: 72% (821/1140) 2024-06-26T07:01:16.1693457Z remote: Compressing objects: 73% (833/1140) 2024-06-26T07:01:16.1694545Z remote: Compressing objects: 74% (844/1140) 2024-06-26T07:01:16.1695843Z remote: Compressing objects: 75% (855/1140) 2024-06-26T07:01:16.1696671Z remote: Compressing objects: 76% (867/1140) 2024-06-26T07:01:16.1697596Z remote: Compressing objects: 77% (878/1140) 2024-06-26T07:01:16.1698159Z remote: Compressing objects: 78% (890/1140) 2024-06-26T07:01:16.1698698Z remote: Compressing objects: 79% (901/1140) 2024-06-26T07:01:16.1701930Z remote: Compressing objects: 80% (912/1140) 2024-06-26T07:01:16.1702490Z remote: Compressing objects: 81% (924/1140) 2024-06-26T07:01:16.1705292Z remote: Compressing objects: 82% (935/1140) 2024-06-26T07:01:16.1708568Z remote: Compressing objects: 83% (947/1140) 2024-06-26T07:01:16.1709189Z remote: Compressing objects: 84% (958/1140) 2024-06-26T07:01:16.1711485Z remote: Compressing objects: 85% (969/1140) 2024-06-26T07:01:16.1713569Z remote: Compressing objects: 86% (981/1140) 2024-06-26T07:01:16.1715080Z remote: Compressing objects: 87% (992/1140) 2024-06-26T07:01:16.1720028Z remote: Compressing objects: 88% (1004/1140) 2024-06-26T07:01:16.1728483Z remote: Compressing objects: 89% (1015/1140) 2024-06-26T07:01:16.1729070Z remote: Compressing objects: 90% (1026/1140) 2024-06-26T07:01:16.1729625Z remote: Compressing objects: 91% (1038/1140) 2024-06-26T07:01:16.1730178Z remote: Compressing objects: 92% (1049/1140) 2024-06-26T07:01:16.1740100Z remote: Compressing objects: 93% (1061/1140) 2024-06-26T07:01:16.1740662Z remote: Compressing objects: 94% (1072/1140) 2024-06-26T07:01:16.1741480Z remote: Compressing objects: 95% (1083/1140) 2024-06-26T07:01:16.1742032Z remote: Compressing objects: 96% (1095/1140) 2024-06-26T07:01:16.1742580Z remote: Compressing objects: 97% (1106/1140) 2024-06-26T07:01:16.1743119Z remote: Compressing objects: 98% (1118/1140) 2024-06-26T07:01:16.1743661Z remote: Compressing objects: 99% (1129/1140) 2024-06-26T07:01:16.1744293Z remote: Compressing objects: 100% (1140/1140) 2024-06-26T07:01:16.1744886Z remote: Compressing objects: 100% (1140/1140), done. 2024-06-26T07:01:35.5844754Z remote: Total 990209 (delta 2280), reused 2016 (delta 1676), pack-reused 987392 2024-06-26T07:02:01.5177103Z [command]/usr/bin/git rev-parse --verify --quiet b8c4c54d347aa776934c60784e35936878ef18dc^{object} 2024-06-26T07:02:01.5198201Z b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:02:01.5203562Z ##[endgroup] 2024-06-26T07:02:01.5204391Z ##[group]Determining the checkout info 2024-06-26T07:02:01.5205403Z ##[endgroup] 2024-06-26T07:02:01.5206164Z ##[group]Checking out the ref 2024-06-26T07:02:01.5209832Z [command]/usr/bin/git checkout --quiet --force b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:02:02.7421803Z ##[endgroup] 2024-06-26T07:02:02.7422653Z ##[group]Setting up auth for fetching submodules 2024-06-26T07:02:02.7427862Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:02:02.7472043Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-06-26T07:02:02.7492896Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-06-26T07:02:02.7513511Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-06-26T07:02:02.7530497Z ##[endgroup] 2024-06-26T07:02:02.7531288Z ##[group]Fetching submodules 2024-06-26T07:02:02.7534909Z [command]/usr/bin/git submodule sync --recursive 2024-06-26T07:02:02.7738115Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-06-26T07:02:02.7938766Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-06-26T07:02:02.7940273Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-06-26T07:02:02.7941947Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-06-26T07:02:02.7943982Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-06-26T07:02:02.7947799Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-06-26T07:02:02.7950011Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-06-26T07:02:02.7952414Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-06-26T07:02:02.7955130Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-06-26T07:02:02.7957666Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-06-26T07:02:02.7960466Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-06-26T07:02:02.7963127Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-06-26T07:02:02.7966000Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-06-26T07:02:02.7968888Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-06-26T07:02:02.7971988Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-06-26T07:02:02.7974871Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-06-26T07:02:02.7977983Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-06-26T07:02:02.7981278Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:02:02.7984535Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-06-26T07:02:02.7988921Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-06-26T07:02:02.7991455Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-06-26T07:02:02.7994947Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-06-26T07:02:02.7998466Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-06-26T07:02:02.8002187Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-06-26T07:02:02.8005786Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-06-26T07:02:02.8009535Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-06-26T07:02:02.8013206Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-06-26T07:02:02.8017367Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-06-26T07:02:02.8021016Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-06-26T07:02:02.8025130Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-06-26T07:02:02.8030142Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-06-26T07:02:02.8034463Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-06-26T07:02:02.8038400Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-06-26T07:02:02.8042613Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-06-26T07:02:02.8046814Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-06-26T07:02:02.8051259Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-06-26T07:02:02.8072552Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-06-26T07:02:03.0601703Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-06-26T07:02:03.2269083Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-06-26T07:02:03.3879437Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-06-26T07:02:03.6903544Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-06-26T07:02:05.7944693Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-06-26T07:02:15.7111353Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-06-26T07:02:16.1015513Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-06-26T07:02:16.7085631Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-06-26T07:02:17.2798640Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-06-26T07:02:18.6379125Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-06-26T07:02:20.4308184Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-06-26T07:02:26.7194343Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-06-26T07:02:28.1270321Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-06-26T07:02:29.7832351Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-06-26T07:02:31.0307127Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-06-26T07:02:31.2287501Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-06-26T07:02:31.6208753Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-06-26T07:02:32.0007970Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-06-26T07:02:33.0761725Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-06-26T07:02:33.4373723Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-06-26T07:02:33.6979926Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-06-26T07:02:35.3283131Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-06-26T07:02:36.0900075Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-06-26T07:02:36.8605532Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-06-26T07:02:43.0248980Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-06-26T07:02:45.4562004Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-06-26T07:02:51.1098764Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-06-26T07:02:51.3032694Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-06-26T07:03:00.0438529Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-06-26T07:03:00.2147502Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-06-26T07:03:00.4085743Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-06-26T07:03:01.4490017Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-06-26T07:03:01.7264903Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-06-26T07:03:02.3809028Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-06-26T07:03:02.9192723Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-06-26T07:03:02.9272266Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-06-26T07:03:02.9331836Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-06-26T07:03:02.9510165Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-06-26T07:03:02.9816112Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-06-26T07:03:03.7561257Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-06-26T07:03:03.7735859Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-06-26T07:03:03.8090538Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-06-26T07:03:03.8945702Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-06-26T07:03:03.9213250Z Submodule path 'third_party/cudnn_frontend': checked out 'aa3abd4bc689d6412979c7f55f9cd132848c9c6a' 2024-06-26T07:03:04.3517688Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-06-26T07:03:04.5652673Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-06-26T07:03:04.6297242Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-06-26T07:03:04.6308314Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:03:04.6310023Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:03:04.6311628Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:03:04.6313604Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:03:04.6315798Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:03:04.6333325Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-06-26T07:03:05.6985013Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-06-26T07:03:06.3453970Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-06-26T07:03:08.3037488Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-06-26T07:03:09.3853778Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-06-26T07:03:09.7200985Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-06-26T07:03:09.8036413Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-06-26T07:03:10.1471190Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-06-26T07:03:10.2011507Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-06-26T07:03:10.2099634Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-06-26T07:03:10.3034542Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-06-26T07:03:10.3312593Z Submodule path 'third_party/fmt': checked out 'e69e5f977d458f2650bb346dadf2ad30c5320281' 2024-06-26T07:03:10.3375701Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-06-26T07:03:10.3674641Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-06-26T07:03:10.3871064Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-06-26T07:03:10.4230967Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:03:10.4328577Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-06-26T07:03:10.4339809Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-06-26T07:03:10.4356621Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-06-26T07:03:22.8154422Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-06-26T07:03:22.8289760Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-06-26T07:03:22.9069924Z Submodule path 'third_party/kineto': checked out '8681ff11e1fa54da39023076c5c43eddd87b7a8a' 2024-06-26T07:03:22.9080122Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:03:22.9081847Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:03:22.9083831Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:03:22.9101891Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-06-26T07:03:23.4632051Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-06-26T07:03:24.7070205Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-06-26T07:03:25.7619639Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-06-26T07:03:25.7630050Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:03:25.7632199Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:03:25.7634214Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:03:25.7662300Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:03:25.7664186Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:03:25.7666367Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:03:25.7668611Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:03:25.7670372Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:03:25.7672041Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-06-26T07:03:26.8199740Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-06-26T07:03:27.2386765Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-06-26T07:03:28.4676223Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-06-26T07:03:28.7728276Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-06-26T07:03:29.3058300Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-06-26T07:03:30.3446961Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-06-26T07:03:37.0883599Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-06-26T07:03:37.4663098Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-06-26T07:03:37.4797198Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-06-26T07:03:37.5082984Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-06-26T07:03:37.5176173Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-06-26T07:03:37.5184902Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:03:37.5200661Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-06-26T07:03:37.8381731Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-06-26T07:03:37.8518523Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-06-26T07:03:37.8849920Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-06-26T07:03:37.9664980Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-06-26T07:03:37.9781154Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-06-26T07:03:38.0072289Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out 'a33701196adfad74917046096bf5a2aa0ab0bb50' 2024-06-26T07:03:38.0557846Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-06-26T07:03:38.0857534Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-06-26T07:03:38.1052573Z Submodule path 'third_party/nccl/nccl': checked out '48bb7fec7953112ff37499a272317f6663f8f600' 2024-06-26T07:03:38.1905697Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-06-26T07:03:38.4652874Z Submodule path 'third_party/onnx': checked out '990217f043af7222348ca8f0301e17fa7b841781' 2024-06-26T07:03:38.4675786Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-06-26T07:03:38.4677658Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-06-26T07:03:38.4696690Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-06-26T07:03:38.8689325Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-06-26T07:03:39.8735304Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-06-26T07:03:39.9014465Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-06-26T07:03:39.9547863Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-06-26T07:03:39.9560308Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:03:39.9566900Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:03:39.9568613Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:03:39.9570431Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:03:39.9572408Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:03:39.9574397Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:03:39.9576275Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:03:39.9577855Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:03:39.9591491Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-06-26T07:03:40.3934377Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-06-26T07:03:41.3861387Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-06-26T07:03:41.6748036Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-06-26T07:03:47.8796353Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-06-26T07:03:48.1958187Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-06-26T07:03:48.3828032Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-06-26T07:03:48.7055858Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-06-26T07:03:56.3881560Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-06-26T07:03:56.4209094Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-06-26T07:03:56.4327422Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-06-26T07:03:56.5158964Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-06-26T07:03:56.5247241Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-06-26T07:03:56.5351419Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-06-26T07:03:56.5468370Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-06-26T07:03:56.5476887Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:03:56.5478948Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:03:56.5495060Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-06-26T07:03:58.5773085Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-06-26T07:03:59.8349529Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-06-26T07:03:59.8710758Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:04:00.2342091Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-06-26T07:04:00.2429820Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-06-26T07:04:00.4619525Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-06-26T07:04:00.4634290Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:04:00.4636252Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-06-26T07:04:00.4653862Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-06-26T07:04:00.8882702Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-06-26T07:04:01.9609781Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-06-26T07:04:02.0230363Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-06-26T07:04:02.0289218Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-06-26T07:04:02.0374810Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-06-26T07:04:02.0652926Z Submodule path 'third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b' 2024-06-26T07:04:02.0877178Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-06-26T07:04:02.1209589Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-06-26T07:04:02.1412948Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-06-26T07:04:02.1424422Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:04:02.1426103Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:04:02.1428251Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:04:02.1430318Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:04:02.1446850Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-06-26T07:04:03.1437703Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-06-26T07:04:03.3914667Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-06-26T07:04:04.4176528Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-06-26T07:04:05.5821972Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-06-26T07:04:05.5935675Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-06-26T07:04:05.6503240Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-06-26T07:04:05.6732133Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-06-26T07:04:05.6741453Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:04:05.6759189Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-06-26T07:04:05.8797573Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-06-26T07:04:05.8820230Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-06-26T07:04:05.9024624Z Entering 'android/libs/fbjni' 2024-06-26T07:04:05.9051816Z Entering 'third_party/FP16' 2024-06-26T07:04:05.9080111Z Entering 'third_party/FXdiv' 2024-06-26T07:04:05.9107157Z Entering 'third_party/NNPACK' 2024-06-26T07:04:05.9135412Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:04:05.9164052Z Entering 'third_party/XNNPACK' 2024-06-26T07:04:05.9205645Z Entering 'third_party/benchmark' 2024-06-26T07:04:05.9233782Z Entering 'third_party/cpp-httplib' 2024-06-26T07:04:05.9262624Z Entering 'third_party/cpuinfo' 2024-06-26T07:04:05.9292180Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:04:05.9319615Z Entering 'third_party/cutlass' 2024-06-26T07:04:05.9353138Z Entering 'third_party/eigen' 2024-06-26T07:04:05.9383058Z Entering 'third_party/fbgemm' 2024-06-26T07:04:05.9410507Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:04:05.9438649Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:04:05.9464155Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:04:05.9494723Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:04:05.9521791Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:04:05.9550195Z Entering 'third_party/flatbuffers' 2024-06-26T07:04:05.9581115Z Entering 'third_party/fmt' 2024-06-26T07:04:05.9606743Z Entering 'third_party/foxi' 2024-06-26T07:04:05.9632018Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:04:05.9661341Z Entering 'third_party/gloo' 2024-06-26T07:04:05.9688758Z Entering 'third_party/googletest' 2024-06-26T07:04:05.9719199Z Entering 'third_party/ideep' 2024-06-26T07:04:05.9749794Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:04:05.9781965Z Entering 'third_party/ittapi' 2024-06-26T07:04:05.9812251Z Entering 'third_party/kineto' 2024-06-26T07:04:05.9840871Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:04:05.9868664Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:04:05.9898140Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:04:05.9924305Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:04:05.9950530Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:04:05.9976918Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:04:06.0004563Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:04:06.0031591Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:04:06.0056883Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:04:06.0085812Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:04:06.0113298Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:04:06.0140102Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:04:06.0170142Z Entering 'third_party/mimalloc' 2024-06-26T07:04:06.0197344Z Entering 'third_party/nccl/nccl' 2024-06-26T07:04:06.0226315Z Entering 'third_party/nlohmann' 2024-06-26T07:04:06.0255450Z Entering 'third_party/onnx' 2024-06-26T07:04:06.0296080Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:04:06.0324416Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:04:06.0354758Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:04:06.0384964Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:04:06.0413144Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:04:06.0439640Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:04:06.0467644Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:04:06.0495866Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:04:06.0522063Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:04:06.0549355Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:04:06.0575433Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:04:06.0605152Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:04:06.0633664Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:04:06.0676894Z Entering 'third_party/pocketfft' 2024-06-26T07:04:06.0706917Z Entering 'third_party/protobuf' 2024-06-26T07:04:06.0738429Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:04:06.0764316Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:04:06.0791535Z Entering 'third_party/psimd' 2024-06-26T07:04:06.0820308Z Entering 'third_party/pthreadpool' 2024-06-26T07:04:06.0845746Z Entering 'third_party/pybind11' 2024-06-26T07:04:06.0870998Z Entering 'third_party/python-peachpy' 2024-06-26T07:04:06.0898520Z Entering 'third_party/sleef' 2024-06-26T07:04:06.0924028Z Entering 'third_party/tensorpipe' 2024-06-26T07:04:06.0950080Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:04:06.0977952Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:04:06.1005297Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:04:06.1031246Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:04:06.1056235Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:04:06.1096906Z ##[endgroup] 2024-06-26T07:04:06.1098856Z ##[group]Persisting credentials for submodules 2024-06-26T07:04:06.1102582Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-06-26T07:04:06.1311744Z Entering 'android/libs/fbjni' 2024-06-26T07:04:06.1346185Z Entering 'third_party/FP16' 2024-06-26T07:04:06.1380867Z Entering 'third_party/FXdiv' 2024-06-26T07:04:06.1416401Z Entering 'third_party/NNPACK' 2024-06-26T07:04:06.1453694Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:04:06.1486896Z Entering 'third_party/XNNPACK' 2024-06-26T07:04:06.1534503Z Entering 'third_party/benchmark' 2024-06-26T07:04:06.1570998Z Entering 'third_party/cpp-httplib' 2024-06-26T07:04:06.1606419Z Entering 'third_party/cpuinfo' 2024-06-26T07:04:06.1639803Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:04:06.1672568Z Entering 'third_party/cutlass' 2024-06-26T07:04:06.1712114Z Entering 'third_party/eigen' 2024-06-26T07:04:06.1750278Z Entering 'third_party/fbgemm' 2024-06-26T07:04:06.1784932Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:04:06.1820494Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:04:06.1855855Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:04:06.1893517Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:04:06.1927913Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:04:06.1964368Z Entering 'third_party/flatbuffers' 2024-06-26T07:04:06.2003585Z Entering 'third_party/fmt' 2024-06-26T07:04:06.2036847Z Entering 'third_party/foxi' 2024-06-26T07:04:06.2071707Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:04:06.2110157Z Entering 'third_party/gloo' 2024-06-26T07:04:06.2145081Z Entering 'third_party/googletest' 2024-06-26T07:04:06.2178377Z Entering 'third_party/ideep' 2024-06-26T07:04:06.2213705Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:04:06.2252829Z Entering 'third_party/ittapi' 2024-06-26T07:04:06.2286126Z Entering 'third_party/kineto' 2024-06-26T07:04:06.2321254Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:04:06.2355544Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:04:06.2390842Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:04:06.2426386Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:04:06.2461035Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:04:06.2493746Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:04:06.2530602Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:04:06.2565941Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:04:06.2600053Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:04:06.2635271Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:04:06.2669551Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:04:06.2703824Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:04:06.2740233Z Entering 'third_party/mimalloc' 2024-06-26T07:04:06.2775769Z Entering 'third_party/nccl/nccl' 2024-06-26T07:04:06.2811808Z Entering 'third_party/nlohmann' 2024-06-26T07:04:06.2848515Z Entering 'third_party/onnx' 2024-06-26T07:04:06.2894101Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:04:06.2929986Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:04:06.2968984Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:04:06.3006599Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:04:06.3039522Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:04:06.3073693Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:04:06.3106070Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:04:06.3141026Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:04:06.3175299Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:04:06.3210004Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:04:06.3244183Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:04:06.3277614Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:04:06.3313707Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:04:06.3363629Z Entering 'third_party/pocketfft' 2024-06-26T07:04:06.3399646Z Entering 'third_party/protobuf' 2024-06-26T07:04:06.3437972Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:04:06.3472666Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:04:06.3505888Z Entering 'third_party/psimd' 2024-06-26T07:04:06.3541185Z Entering 'third_party/pthreadpool' 2024-06-26T07:04:06.3576588Z Entering 'third_party/pybind11' 2024-06-26T07:04:06.3611896Z Entering 'third_party/python-peachpy' 2024-06-26T07:04:06.3646669Z Entering 'third_party/sleef' 2024-06-26T07:04:06.3682172Z Entering 'third_party/tensorpipe' 2024-06-26T07:04:06.3715412Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:04:06.3749345Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:04:06.3781157Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:04:06.3815783Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:04:06.3848921Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:04:06.3898502Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-06-26T07:04:06.4106272Z Entering 'android/libs/fbjni' 2024-06-26T07:04:06.4137971Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-06-26T07:04:06.4150876Z Entering 'third_party/FP16' 2024-06-26T07:04:06.4181857Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-06-26T07:04:06.4196577Z Entering 'third_party/FXdiv' 2024-06-26T07:04:06.4227435Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-06-26T07:04:06.4240421Z Entering 'third_party/NNPACK' 2024-06-26T07:04:06.4271363Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-06-26T07:04:06.4286113Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:04:06.4318802Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-06-26T07:04:06.4331893Z Entering 'third_party/XNNPACK' 2024-06-26T07:04:06.4362786Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-06-26T07:04:06.4387684Z Entering 'third_party/benchmark' 2024-06-26T07:04:06.4419542Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:04:06.4432428Z Entering 'third_party/cpp-httplib' 2024-06-26T07:04:06.4464263Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-06-26T07:04:06.4478783Z Entering 'third_party/cpuinfo' 2024-06-26T07:04:06.4511453Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:04:06.4525031Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:04:06.4556767Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-06-26T07:04:06.4569904Z Entering 'third_party/cutlass' 2024-06-26T07:04:06.4601298Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:04:06.4619839Z Entering 'third_party/eigen' 2024-06-26T07:04:06.4651050Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-06-26T07:04:06.4666891Z Entering 'third_party/fbgemm' 2024-06-26T07:04:06.4698227Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-06-26T07:04:06.4712982Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:04:06.4744149Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-06-26T07:04:06.4756970Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:04:06.4789126Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:04:06.4802264Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:04:06.4833305Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:04:06.4851152Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:04:06.4882218Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:04:06.4895427Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:04:06.4927157Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-06-26T07:04:06.4941028Z Entering 'third_party/flatbuffers' 2024-06-26T07:04:06.4974879Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-06-26T07:04:06.4989575Z Entering 'third_party/fmt' 2024-06-26T07:04:06.5020464Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:04:06.5032639Z Entering 'third_party/foxi' 2024-06-26T07:04:06.5064946Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-06-26T07:04:06.5079274Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:04:06.5110760Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-06-26T07:04:06.5123393Z Entering 'third_party/gloo' 2024-06-26T07:04:06.5154453Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-06-26T07:04:06.5167316Z Entering 'third_party/googletest' 2024-06-26T07:04:06.5199706Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:04:06.5213376Z Entering 'third_party/ideep' 2024-06-26T07:04:06.5244719Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-06-26T07:04:06.5256816Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:04:06.5287630Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-06-26T07:04:06.5305646Z Entering 'third_party/ittapi' 2024-06-26T07:04:06.5338837Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-06-26T07:04:06.5352997Z Entering 'third_party/kineto' 2024-06-26T07:04:06.5385562Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-06-26T07:04:06.5398709Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:04:06.5431609Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-06-26T07:04:06.5444021Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:04:06.5476778Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-06-26T07:04:06.5490586Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:04:06.5522013Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-06-26T07:04:06.5535472Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:04:06.5565640Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:04:06.5576782Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:04:06.5610460Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-06-26T07:04:06.5622017Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:04:06.5653905Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-06-26T07:04:06.5667758Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:04:06.5698926Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-06-26T07:04:06.5711990Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:04:06.5743187Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:04:06.5756105Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:04:06.5787563Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-06-26T07:04:06.5801088Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:04:06.5832854Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-06-26T07:04:06.5846155Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:04:06.5877394Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-06-26T07:04:06.5888459Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:04:06.5921195Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-06-26T07:04:06.5933890Z Entering 'third_party/mimalloc' 2024-06-26T07:04:06.5966220Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-06-26T07:04:06.5978454Z Entering 'third_party/nccl/nccl' 2024-06-26T07:04:06.6014567Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-06-26T07:04:06.6028347Z Entering 'third_party/nlohmann' 2024-06-26T07:04:06.6058148Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-06-26T07:04:06.6072092Z Entering 'third_party/onnx' 2024-06-26T07:04:06.6101286Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-06-26T07:04:06.6125358Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:04:06.6164129Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:04:06.6177361Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:04:06.6208760Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:04:06.6223764Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:04:06.6256473Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-06-26T07:04:06.6272563Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:04:06.6303881Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:04:06.6316820Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:04:06.6349454Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:04:06.6361619Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:04:06.6392285Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-06-26T07:04:06.6403924Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:04:06.6435477Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-06-26T07:04:06.6450142Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:04:06.6481674Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-06-26T07:04:06.6493801Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:04:06.6525003Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-06-26T07:04:06.6538293Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:04:06.6571203Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-06-26T07:04:06.6583083Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:04:06.6616248Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-06-26T07:04:06.6631530Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:04:06.6663037Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-06-26T07:04:06.6676830Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:04:06.6707411Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-06-26T07:04:06.6736212Z Entering 'third_party/pocketfft' 2024-06-26T07:04:06.6770898Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-06-26T07:04:06.6784750Z Entering 'third_party/protobuf' 2024-06-26T07:04:06.6818628Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-06-26T07:04:06.6835549Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:04:06.6866305Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:04:06.6879303Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:04:06.6912272Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:04:06.6928149Z Entering 'third_party/psimd' 2024-06-26T07:04:06.6961482Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-06-26T07:04:06.6974851Z Entering 'third_party/pthreadpool' 2024-06-26T07:04:06.7008167Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-06-26T07:04:06.7020462Z Entering 'third_party/pybind11' 2024-06-26T07:04:06.7053228Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:04:06.7067277Z Entering 'third_party/python-peachpy' 2024-06-26T07:04:06.7097768Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-06-26T07:04:06.7110526Z Entering 'third_party/sleef' 2024-06-26T07:04:06.7142091Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-06-26T07:04:06.7154206Z Entering 'third_party/tensorpipe' 2024-06-26T07:04:06.7185784Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-06-26T07:04:06.7198576Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:04:06.7230220Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:04:06.7242460Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:04:06.7274029Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-06-26T07:04:06.7285953Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:04:06.7317050Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-06-26T07:04:06.7329831Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:04:06.7359957Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:04:06.7370187Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:04:06.7401836Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-06-26T07:04:06.8700242Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-06-26T07:04:06.8906998Z Entering 'android/libs/fbjni' 2024-06-26T07:04:06.8935803Z Entering 'third_party/FP16' 2024-06-26T07:04:06.8965329Z Entering 'third_party/FXdiv' 2024-06-26T07:04:06.8995052Z Entering 'third_party/NNPACK' 2024-06-26T07:04:06.9022910Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:04:06.9050639Z Entering 'third_party/XNNPACK' 2024-06-26T07:04:06.9089376Z Entering 'third_party/benchmark' 2024-06-26T07:04:06.9118534Z Entering 'third_party/cpp-httplib' 2024-06-26T07:04:06.9148436Z Entering 'third_party/cpuinfo' 2024-06-26T07:04:06.9177909Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:04:06.9206356Z Entering 'third_party/cutlass' 2024-06-26T07:04:06.9238691Z Entering 'third_party/eigen' 2024-06-26T07:04:06.9269103Z Entering 'third_party/fbgemm' 2024-06-26T07:04:06.9297731Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:04:06.9326095Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:04:06.9353256Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:04:06.9384446Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:04:06.9412200Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:04:06.9441212Z Entering 'third_party/flatbuffers' 2024-06-26T07:04:06.9472064Z Entering 'third_party/fmt' 2024-06-26T07:04:06.9499144Z Entering 'third_party/foxi' 2024-06-26T07:04:06.9528163Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:04:06.9554358Z Entering 'third_party/gloo' 2024-06-26T07:04:06.9581392Z Entering 'third_party/googletest' 2024-06-26T07:04:06.9611506Z Entering 'third_party/ideep' 2024-06-26T07:04:06.9639342Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:04:06.9670809Z Entering 'third_party/ittapi' 2024-06-26T07:04:06.9699324Z Entering 'third_party/kineto' 2024-06-26T07:04:06.9728265Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:04:06.9754623Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:04:06.9783712Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:04:06.9811519Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:04:06.9839799Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:04:06.9865959Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:04:06.9894938Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:04:06.9921885Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:04:06.9949864Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:04:06.9978568Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:04:07.0009255Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:04:07.0037953Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:04:07.0066368Z Entering 'third_party/mimalloc' 2024-06-26T07:04:07.0096417Z Entering 'third_party/nccl/nccl' 2024-06-26T07:04:07.0124628Z Entering 'third_party/nlohmann' 2024-06-26T07:04:07.0152935Z Entering 'third_party/onnx' 2024-06-26T07:04:07.0192211Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:04:07.0220721Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:04:07.0291335Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:04:07.0319026Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:04:07.0343995Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:04:07.0368933Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:04:07.0397003Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:04:07.0424808Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:04:07.0450929Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:04:07.0480252Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:04:07.0507357Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:04:07.0534766Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:04:07.0563816Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:04:07.0604889Z Entering 'third_party/pocketfft' 2024-06-26T07:04:07.0633253Z Entering 'third_party/protobuf' 2024-06-26T07:04:07.0661566Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:04:07.0689289Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:04:07.0717081Z Entering 'third_party/psimd' 2024-06-26T07:04:07.0743638Z Entering 'third_party/pthreadpool' 2024-06-26T07:04:07.0773073Z Entering 'third_party/pybind11' 2024-06-26T07:04:07.0802214Z Entering 'third_party/python-peachpy' 2024-06-26T07:04:07.0828206Z Entering 'third_party/sleef' 2024-06-26T07:04:07.0856249Z Entering 'third_party/tensorpipe' 2024-06-26T07:04:07.0883298Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:04:07.0910076Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:04:07.0935797Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:04:07.0962858Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:04:07.0987854Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:04:07.1028581Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-06-26T07:04:07.1237205Z Entering 'android/libs/fbjni' 2024-06-26T07:04:07.1263935Z Entering 'third_party/FP16' 2024-06-26T07:04:07.1294022Z Entering 'third_party/FXdiv' 2024-06-26T07:04:07.1321449Z Entering 'third_party/NNPACK' 2024-06-26T07:04:07.1347509Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:04:07.1376496Z Entering 'third_party/XNNPACK' 2024-06-26T07:04:07.1413919Z Entering 'third_party/benchmark' 2024-06-26T07:04:07.1441396Z Entering 'third_party/cpp-httplib' 2024-06-26T07:04:07.1468098Z Entering 'third_party/cpuinfo' 2024-06-26T07:04:07.1496833Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:04:07.1525936Z Entering 'third_party/cutlass' 2024-06-26T07:04:07.1557788Z Entering 'third_party/eigen' 2024-06-26T07:04:07.1585889Z Entering 'third_party/fbgemm' 2024-06-26T07:04:07.1615187Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:04:07.1641587Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:04:07.1667083Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:04:07.1698643Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:04:07.1726971Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:04:07.1754051Z Entering 'third_party/flatbuffers' 2024-06-26T07:04:07.1784175Z Entering 'third_party/fmt' 2024-06-26T07:04:07.1813583Z Entering 'third_party/foxi' 2024-06-26T07:04:07.1841726Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:04:07.1868237Z Entering 'third_party/gloo' 2024-06-26T07:04:07.1895491Z Entering 'third_party/googletest' 2024-06-26T07:04:07.1924070Z Entering 'third_party/ideep' 2024-06-26T07:04:07.1952050Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:04:07.1985068Z Entering 'third_party/ittapi' 2024-06-26T07:04:07.2014446Z Entering 'third_party/kineto' 2024-06-26T07:04:07.2042961Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:04:07.2069559Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:04:07.2097914Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:04:07.2124931Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:04:07.2154278Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:04:07.2180383Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:04:07.2210314Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:04:07.2239131Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:04:07.2268271Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:04:07.2296508Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:04:07.2325816Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:04:07.2354217Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:04:07.2382411Z Entering 'third_party/mimalloc' 2024-06-26T07:04:07.2412451Z Entering 'third_party/nccl/nccl' 2024-06-26T07:04:07.2439107Z Entering 'third_party/nlohmann' 2024-06-26T07:04:07.2469865Z Entering 'third_party/onnx' 2024-06-26T07:04:07.2509691Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:04:07.2535786Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:04:07.2565798Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:04:07.2595216Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:04:07.2622724Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:04:07.2650914Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:04:07.2679018Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:04:07.2707951Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:04:07.2733606Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:04:07.2759435Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:04:07.2786462Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:04:07.2815799Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:04:07.2843264Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:04:07.2884954Z Entering 'third_party/pocketfft' 2024-06-26T07:04:07.2913057Z Entering 'third_party/protobuf' 2024-06-26T07:04:07.2942630Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:04:07.2971111Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:04:07.3001675Z Entering 'third_party/psimd' 2024-06-26T07:04:07.3030127Z Entering 'third_party/pthreadpool' 2024-06-26T07:04:07.3058504Z Entering 'third_party/pybind11' 2024-06-26T07:04:07.3084690Z Entering 'third_party/python-peachpy' 2024-06-26T07:04:07.3109896Z Entering 'third_party/sleef' 2024-06-26T07:04:07.3141376Z Entering 'third_party/tensorpipe' 2024-06-26T07:04:07.3170316Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:04:07.3201168Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:04:07.3227459Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:04:07.3253811Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:04:07.3279435Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:04:07.3317268Z ##[endgroup] 2024-06-26T07:04:07.3344196Z [command]/usr/bin/git log -1 --format='%H' 2024-06-26T07:04:07.3364800Z 'b8c4c54d347aa776934c60784e35936878ef18dc' 2024-06-26T07:04:07.3541009Z Prepare all required actions 2024-06-26T07:04:07.3541557Z Getting action download info 2024-06-26T07:04:07.4948723Z ##[group]Run ./.github/actions/setup-linux 2024-06-26T07:04:07.4949150Z env: 2024-06-26T07:04:07.4949426Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:07.4949775Z ##[endgroup] 2024-06-26T07:04:07.5044807Z ##[group]Run set -euo pipefail 2024-06-26T07:04:07.5045242Z set -euo pipefail 2024-06-26T07:04:07.5045639Z function get_ec2_metadata() { 2024-06-26T07:04:07.5046177Z  # Pulled from instance metadata endpoint for EC2 2024-06-26T07:04:07.5047082Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-06-26T07:04:07.5047860Z  category=$1 2024-06-26T07:04:07.5048403Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-06-26T07:04:07.5049049Z  runner_name_str=i-083295e4fef5bcb20 2024-06-26T07:04:07.5049548Z  if [[ -f /.inarc ]]; then 2024-06-26T07:04:07.5050049Z  echo "ARC Runner, no info on ec2 metadata" 2024-06-26T07:04:07.5050634Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-06-26T07:04:07.5051334Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-06-26T07:04:07.5051945Z  else 2024-06-26T07:04:07.5052434Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-06-26T07:04:07.5053026Z  fi 2024-06-26T07:04:07.5053322Z } 2024-06-26T07:04:07.5053695Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-06-26T07:04:07.5054288Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-06-26T07:04:07.5054946Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-06-26T07:04:07.5055525Z echo "system info $(uname -a)" 2024-06-26T07:04:07.5063315Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:07.5063820Z env: 2024-06-26T07:04:07.5064090Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:07.5064439Z ##[endgroup] 2024-06-26T07:04:07.5134735Z ami-id: ami-0ce0c36d7a00b20e2 2024-06-26T07:04:07.5174630Z instance-id: i-083295e4fef5bcb20 2024-06-26T07:04:07.5216264Z instance-type: g5.4xlarge 2024-06-26T07:04:07.5221903Z system info Linux ip-10-0-72-207.ec2.internal 4.14.336-257.562.amzn2.x86_64 #1 SMP Sat Feb 24 09:50:35 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-06-26T07:04:07.5241845Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:04:07.5242769Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:04:07.5250332Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:07.5250947Z env: 2024-06-26T07:04:07.5251233Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:07.5251598Z ##[endgroup] 2024-06-26T07:04:07.5353158Z ##[group]Run if systemctl is-active --quiet docker; then 2024-06-26T07:04:07.5353780Z if systemctl is-active --quiet docker; then 2024-06-26T07:04:07.5354341Z  echo "Docker daemon is running..."; 2024-06-26T07:04:07.5354807Z else 2024-06-26T07:04:07.5355318Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-06-26T07:04:07.5355915Z fi 2024-06-26T07:04:07.5362801Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:07.5363301Z env: 2024-06-26T07:04:07.5363589Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:07.5363947Z ##[endgroup] 2024-06-26T07:04:07.5398284Z Docker daemon is running... 2024-06-26T07:04:07.5530940Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:04:07.5531503Z with: 2024-06-26T07:04:07.5531773Z shell: bash 2024-06-26T07:04:07.5532079Z timeout_minutes: 5 2024-06-26T07:04:07.5532406Z max_attempts: 3 2024-06-26T07:04:07.5532729Z retry_wait_seconds: 30 2024-06-26T07:04:07.5534219Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-06-26T07:04:07.5535642Z polling_interval_seconds: 1 2024-06-26T07:04:07.5536022Z warning_on_retry: true 2024-06-26T07:04:07.5536380Z continue_on_error: false 2024-06-26T07:04:07.5536720Z env: 2024-06-26T07:04:07.5536985Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:07.5537345Z AWS_RETRY_MODE: standard 2024-06-26T07:04:07.5537708Z AWS_MAX_ATTEMPTS: 5 2024-06-26T07:04:07.5538045Z AWS_DEFAULT_REGION: us-east-1 2024-06-26T07:04:07.5538423Z ##[endgroup] 2024-06-26T07:04:08.4069857Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:04:08.4070869Z Configure a credential helper to remove this warning. See 2024-06-26T07:04:08.4071810Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:04:08.4072812Z 2024-06-26T07:04:08.4073258Z Login Succeeded 2024-06-26T07:04:08.6024816Z Command completed after 1 attempt(s). 2024-06-26T07:04:08.6065653Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:04:08.6066379Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:04:08.6067462Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:04:08.6075511Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:08.6076050Z env: 2024-06-26T07:04:08.6076349Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:08.6076702Z ##[endgroup] 2024-06-26T07:04:08.6137106Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:04:08.6137861Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:04:08.6138456Z # shellcheck disable=SC2046 2024-06-26T07:04:08.6138916Z docker stop $(docker ps -q) || true 2024-06-26T07:04:08.6139403Z # Prune all of the docker images 2024-06-26T07:04:08.6140056Z docker system prune -af 2024-06-26T07:04:08.6147253Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:08.6147762Z env: 2024-06-26T07:04:08.6148045Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:08.6148394Z ##[endgroup] 2024-06-26T07:04:08.6396027Z "docker stop" requires at least 1 argument. 2024-06-26T07:04:08.6396719Z See 'docker stop --help'. 2024-06-26T07:04:08.6396960Z 2024-06-26T07:04:08.6397187Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-06-26T07:04:08.6397575Z 2024-06-26T07:04:08.6397726Z Stop one or more running containers 2024-06-26T07:04:08.6556523Z Total reclaimed space: 0B 2024-06-26T07:04:08.6584792Z ##[group]Run set +e 2024-06-26T07:04:08.6585143Z set +e 2024-06-26T07:04:08.6585563Z set -x 2024-06-26T07:04:08.6585859Z  2024-06-26T07:04:08.6586168Z PT_DOMAIN=download.pytorch.org 2024-06-26T07:04:08.6587177Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-06-26T07:04:08.6588260Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-06-26T07:04:08.6589009Z # one is returned at random 2024-06-26T07:04:08.6589551Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-06-26T07:04:08.6590055Z  2024-06-26T07:04:08.6590366Z if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:04:08.6590981Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-06-26T07:04:08.6591739Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-06-26T07:04:08.6592287Z  2024-06-26T07:04:08.6592609Z  if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:04:08.6593168Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-06-26T07:04:08.6593673Z  exit 1 2024-06-26T07:04:08.6593975Z  fi 2024-06-26T07:04:08.6594257Z fi 2024-06-26T07:04:08.6594642Z  2024-06-26T07:04:08.6594998Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-06-26T07:04:08.6595510Z  # Clean up any old records first 2024-06-26T07:04:08.6596013Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-06-26T07:04:08.6596469Z fi 2024-06-26T07:04:08.6596738Z  2024-06-26T07:04:08.6597342Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-06-26T07:04:08.6597894Z cat /etc/hosts 2024-06-26T07:04:08.6605585Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:08.6606077Z env: 2024-06-26T07:04:08.6606355Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:08.6606707Z ##[endgroup] 2024-06-26T07:04:08.6626697Z + PT_DOMAIN=download.pytorch.org 2024-06-26T07:04:08.6630329Z ++ dig -4 +short download.pytorch.org 2024-06-26T07:04:08.6630779Z ++ tail -n1 2024-06-26T07:04:08.6687726Z + RESOLVED_IP=18.160.10.22 2024-06-26T07:04:08.6688339Z + '[' -z 18.160.10.22 ']' 2024-06-26T07:04:08.6688952Z + grep -r download.pytorch.org /etc/hosts 2024-06-26T07:04:08.6694619Z 18.160.10.76 download.pytorch.org 2024-06-26T07:04:08.6695664Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-06-26T07:04:08.6793452Z + echo '18.160.10.22 download.pytorch.org' 2024-06-26T07:04:08.6793983Z + sudo tee -a /etc/hosts 2024-06-26T07:04:08.7100255Z 18.160.10.22 download.pytorch.org 2024-06-26T07:04:08.7113903Z + cat /etc/hosts 2024-06-26T07:04:08.7117990Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-06-26T07:04:08.7130313Z ::1 localhost6 localhost6.localdomain6 2024-06-26T07:04:08.7130908Z 18.160.10.22 download.pytorch.org 2024-06-26T07:04:08.7345610Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-06-26T07:04:08.7346311Z with: 2024-06-26T07:04:08.7347723Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:08.7349013Z docker-build-dir: .ci/docker 2024-06-26T07:04:08.7349473Z working-directory: . 2024-06-26T07:04:08.7350030Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:04:08.7350634Z force-push: false 2024-06-26T07:04:08.7351003Z env: 2024-06-26T07:04:08.7351351Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:08.7351765Z ##[endgroup] 2024-06-26T07:04:08.7371192Z ##[group]Run set -ex 2024-06-26T07:04:08.7371642Z set -ex 2024-06-26T07:04:08.7372018Z  2024-06-26T07:04:08.7372671Z # If the docker build directory or the build script doesn't exist, the action will 2024-06-26T07:04:08.7373744Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-06-26T07:04:08.7374788Z # job could then download the pre-built image as usual 2024-06-26T07:04:08.7375631Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-06-26T07:04:08.7376401Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:04:08.7377129Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:04:08.7377778Z  2024-06-26T07:04:08.7378372Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-06-26T07:04:08.7379096Z  exit 0 2024-06-26T07:04:08.7379485Z else 2024-06-26T07:04:08.7379943Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-06-26T07:04:08.7380474Z fi 2024-06-26T07:04:08.7380836Z  2024-06-26T07:04:08.7381396Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-06-26T07:04:08.7382347Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-06-26T07:04:08.7383229Z  # use it as it is, but first let's extract the tag 2024-06-26T07:04:08.7384028Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-06-26T07:04:08.7384952Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:04:08.7385743Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:04:08.7386380Z else 2024-06-26T07:04:08.7387113Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-06-26T07:04:08.7387870Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:04:08.7388841Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:04:08.7389659Z fi 2024-06-26T07:04:08.7397527Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:08.7398109Z env: 2024-06-26T07:04:08.7398475Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:08.7398907Z REPO_NAME: pytorch 2024-06-26T07:04:08.7400132Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:08.7401423Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:04:08.7402010Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:04:08.7402607Z ##[endgroup] 2024-06-26T07:04:08.7424282Z + [[ ! -d .ci/docker ]] 2024-06-26T07:04:08.7424854Z + [[ ! -f .ci/docker/build.sh ]] 2024-06-26T07:04:08.7425330Z + echo skip=false 2024-06-26T07:04:08.7427547Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-06-26T07:04:08.7430272Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:08.7431599Z ++ awk -F '[:,]' '{print $2}' 2024-06-26T07:04:08.7436680Z + DOCKER_TAG=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:08.7437418Z + echo docker-tag=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:08.7439212Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:08.7469321Z ##[group]Run set +e 2024-06-26T07:04:08.7469785Z set +e 2024-06-26T07:04:08.7470177Z set -x 2024-06-26T07:04:08.7470554Z  2024-06-26T07:04:08.7470921Z login() { 2024-06-26T07:04:08.7471692Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:04:08.7472506Z } 2024-06-26T07:04:08.7472858Z  2024-06-26T07:04:08.7473325Z retry () { 2024-06-26T07:04:08.7473806Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:04:08.7474347Z } 2024-06-26T07:04:08.7474703Z  2024-06-26T07:04:08.7475106Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:04:08.7475613Z  2024-06-26T07:04:08.7476166Z # Check if image already exists, if it does then skip building it 2024-06-26T07:04:08.7476958Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-06-26T07:04:08.7477556Z  exit 0 2024-06-26T07:04:08.7477941Z fi 2024-06-26T07:04:08.7478302Z  2024-06-26T07:04:08.7478898Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-06-26T07:04:08.7479812Z # be empty. The default action would be to continue rebuild the image 2024-06-26T07:04:08.7480643Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-06-26T07:04:08.7481397Z  # if we're on the base branch then use the parent commit 2024-06-26T07:04:08.7482085Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-06-26T07:04:08.7482601Z else 2024-06-26T07:04:08.7483161Z  # otherwise we're on a PR, so use the most recent base commit 2024-06-26T07:04:08.7483934Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-06-26T07:04:08.7484526Z fi 2024-06-26T07:04:08.7484889Z  2024-06-26T07:04:08.7485298Z if [[ -z "${MERGE_BASE}" ]]; then 2024-06-26T07:04:08.7485891Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:04:08.7486433Z  2024-06-26T07:04:08.7487180Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-06-26T07:04:08.7488028Z  exit 0 2024-06-26T07:04:08.7488406Z fi 2024-06-26T07:04:08.7488767Z  2024-06-26T07:04:08.7489301Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-06-26T07:04:08.7490378Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-06-26T07:04:08.7491258Z  exit 1 2024-06-26T07:04:08.7491650Z fi 2024-06-26T07:04:08.7492017Z  2024-06-26T07:04:08.7492602Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-06-26T07:04:08.7493641Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-06-26T07:04:08.7494595Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-06-26T07:04:08.7495648Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-06-26T07:04:08.7496839Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-06-26T07:04:08.7497669Z fi 2024-06-26T07:04:08.7498031Z  2024-06-26T07:04:08.7498483Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:04:08.7506638Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:08.7507233Z env: 2024-06-26T07:04:08.7507590Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:08.7508052Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:04:08.7508616Z BASE_REVISION: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:04:08.7509991Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:08.7511343Z DOCKER_TAG: 91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:08.7512105Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:04:08.7512770Z ##[endgroup] 2024-06-26T07:04:08.7533994Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:04:08.7534775Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:04:08.7536145Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:04:08.7537373Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:04:09.1647788Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:04:09.1648597Z Configure a credential helper to remove this warning. See 2024-06-26T07:04:09.1649427Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:04:09.1649957Z 2024-06-26T07:04:09.1650083Z Login Succeeded 2024-06-26T07:04:09.1660551Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:09.3818961Z { 2024-06-26T07:04:09.3819356Z "schemaVersion": 2, 2024-06-26T07:04:09.3820132Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-06-26T07:04:09.3820940Z "config": { 2024-06-26T07:04:09.3821586Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-06-26T07:04:09.3822485Z "size": 48681, 2024-06-26T07:04:09.3823387Z "digest": "sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd" 2024-06-26T07:04:09.3824290Z }, 2024-06-26T07:04:09.3824771Z "layers": [ 2024-06-26T07:04:09.3825193Z { 2024-06-26T07:04:09.3825848Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3826981Z "size": 28580681, 2024-06-26T07:04:09.3827789Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-06-26T07:04:09.3828744Z }, 2024-06-26T07:04:09.3829100Z { 2024-06-26T07:04:09.3829829Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3830783Z "size": 7943451, 2024-06-26T07:04:09.3831772Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-06-26T07:04:09.3832846Z }, 2024-06-26T07:04:09.3833244Z { 2024-06-26T07:04:09.3833829Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3834631Z "size": 55728572, 2024-06-26T07:04:09.3835111Z + exit 0 2024-06-26T07:04:09.3835851Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-06-26T07:04:09.3836666Z }, 2024-06-26T07:04:09.3837024Z { 2024-06-26T07:04:09.3837575Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3838147Z "size": 186, 2024-06-26T07:04:09.3838700Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-06-26T07:04:09.3839326Z }, 2024-06-26T07:04:09.3839575Z { 2024-06-26T07:04:09.3840014Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3840583Z "size": 6886, 2024-06-26T07:04:09.3841127Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-06-26T07:04:09.3841962Z }, 2024-06-26T07:04:09.3842217Z { 2024-06-26T07:04:09.3842659Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3843219Z "size": 1329001756, 2024-06-26T07:04:09.3843860Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-06-26T07:04:09.3844718Z }, 2024-06-26T07:04:09.3845043Z { 2024-06-26T07:04:09.3845622Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3846193Z "size": 62501, 2024-06-26T07:04:09.3846741Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-06-26T07:04:09.3847462Z }, 2024-06-26T07:04:09.3847783Z { 2024-06-26T07:04:09.3848352Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3849074Z "size": 1684, 2024-06-26T07:04:09.3849745Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-06-26T07:04:09.3850533Z }, 2024-06-26T07:04:09.3850862Z { 2024-06-26T07:04:09.3851424Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3852243Z "size": 1523, 2024-06-26T07:04:09.3852796Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-06-26T07:04:09.3853423Z }, 2024-06-26T07:04:09.3853668Z { 2024-06-26T07:04:09.3854091Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3854643Z "size": 2528295403, 2024-06-26T07:04:09.3855205Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-06-26T07:04:09.3855819Z }, 2024-06-26T07:04:09.3856059Z { 2024-06-26T07:04:09.3856483Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3857030Z "size": 86016, 2024-06-26T07:04:09.3857567Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-06-26T07:04:09.3858178Z }, 2024-06-26T07:04:09.3858422Z { 2024-06-26T07:04:09.3858851Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3859388Z "size": 1823, 2024-06-26T07:04:09.3859914Z "digest": "sha256:309b872da805fcdf74137a887464876fa0615dc8b24199307119eb65fc2bfb49" 2024-06-26T07:04:09.3860511Z }, 2024-06-26T07:04:09.3860757Z { 2024-06-26T07:04:09.3861174Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3861727Z "size": 246737542, 2024-06-26T07:04:09.3862284Z "digest": "sha256:e82932cb002c06a782077b8e5965f8733eb8563406a23cab260d9d8c4a994f7d" 2024-06-26T07:04:09.3863030Z }, 2024-06-26T07:04:09.3863268Z { 2024-06-26T07:04:09.3863703Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3864248Z "size": 545, 2024-06-26T07:04:09.3864887Z "digest": "sha256:ca9dd73e3e9378225ebdcfd12199ef44649e34fe55003cfbc8afe2e8a4527c5b" 2024-06-26T07:04:09.3865515Z }, 2024-06-26T07:04:09.3865757Z { 2024-06-26T07:04:09.3866191Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3866956Z "size": 1283, 2024-06-26T07:04:09.3867502Z "digest": "sha256:16d76a79d5d6616e228537785ec83a869f68a402520bbc6e826201621ca6cb98" 2024-06-26T07:04:09.3868110Z }, 2024-06-26T07:04:09.3868355Z { 2024-06-26T07:04:09.3868776Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3869326Z "size": 485, 2024-06-26T07:04:09.3869868Z "digest": "sha256:4bdf41c9ecbe1651e0986ccbea4663a0556af27b7145a679cfa45c35dc96b891" 2024-06-26T07:04:09.3870491Z }, 2024-06-26T07:04:09.3870727Z { 2024-06-26T07:04:09.3871159Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3871713Z "size": 91710937, 2024-06-26T07:04:09.3872275Z "digest": "sha256:f4863801e30cdbcd1d4e6cb1cbbcde7dbcaf6ae317ca80429295741ce60ac31d" 2024-06-26T07:04:09.3872911Z }, 2024-06-26T07:04:09.3873158Z { 2024-06-26T07:04:09.3873584Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3874242Z "size": 3233, 2024-06-26T07:04:09.3874794Z "digest": "sha256:d37dd9ed8948a0d02e1198bbedadbca36477a0980dd553196385aab0c025581d" 2024-06-26T07:04:09.3875426Z }, 2024-06-26T07:04:09.3875674Z { 2024-06-26T07:04:09.3876099Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3876656Z "size": 1905, 2024-06-26T07:04:09.3877204Z "digest": "sha256:cf2f34f225dbed17c1a9a8b9ea4ad030aba797774a716dc0a82fe212324aa751" 2024-06-26T07:04:09.3877836Z }, 2024-06-26T07:04:09.3878071Z { 2024-06-26T07:04:09.3878503Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3879050Z "size": 702, 2024-06-26T07:04:09.3879581Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:04:09.3880201Z }, 2024-06-26T07:04:09.3880449Z { 2024-06-26T07:04:09.3880881Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3881427Z "size": 2784789002, 2024-06-26T07:04:09.3881993Z "digest": "sha256:3d914950f8addfc4d657a2365c76e1143eb80b710cd3f6e9ae05640217fc6347" 2024-06-26T07:04:09.3882698Z }, 2024-06-26T07:04:09.3882942Z { 2024-06-26T07:04:09.3883366Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3883911Z "size": 381, 2024-06-26T07:04:09.3884444Z "digest": "sha256:96d0e2032184c57e62a652aaffcf0241f0d0c7a23e894ca0ad192c4a64a36583" 2024-06-26T07:04:09.3885056Z }, 2024-06-26T07:04:09.3885303Z { 2024-06-26T07:04:09.3885735Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3886287Z "size": 11879, 2024-06-26T07:04:09.3886868Z "digest": "sha256:06e12ce59790cb8b091e5d619c2a62a28329008ee184a8b9a562e3db12471f30" 2024-06-26T07:04:09.3887474Z }, 2024-06-26T07:04:09.3887723Z { 2024-06-26T07:04:09.3888151Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3888690Z "size": 803, 2024-06-26T07:04:09.3889231Z "digest": "sha256:ed712f63cde7070bdec2a6179c4c35e752c6991ddbc69fe9f2724a9903250ceb" 2024-06-26T07:04:09.3889854Z }, 2024-06-26T07:04:09.3890090Z { 2024-06-26T07:04:09.3890522Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3891075Z "size": 106, 2024-06-26T07:04:09.3891600Z "digest": "sha256:a97a71ef8a4e66234dfc2098834f8395ac5630240e60164d1a251fdb0115ba85" 2024-06-26T07:04:09.3892206Z }, 2024-06-26T07:04:09.3892451Z { 2024-06-26T07:04:09.3892878Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3893440Z "size": 503, 2024-06-26T07:04:09.3893972Z "digest": "sha256:25cd0c482ad612f936b35b4f220c580e16be982226aac8801a6a849a6bfdc221" 2024-06-26T07:04:09.3894588Z }, 2024-06-26T07:04:09.3894838Z { 2024-06-26T07:04:09.3895267Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3895816Z "size": 121477177, 2024-06-26T07:04:09.3896381Z "digest": "sha256:9bf94bc456584365fda21d0ed83b7cc690b08a71727b71c4c1e667e15028a386" 2024-06-26T07:04:09.3897044Z }, 2024-06-26T07:04:09.3897293Z { 2024-06-26T07:04:09.3897720Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3898270Z "size": 109, 2024-06-26T07:04:09.3898804Z "digest": "sha256:de45a14c9b227e4b0c9d45d42223b50a71c03522ddd9e2ca0d9517835206c823" 2024-06-26T07:04:09.3899411Z }, 2024-06-26T07:04:09.3899657Z { 2024-06-26T07:04:09.3900084Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3900636Z "size": 491, 2024-06-26T07:04:09.3901162Z "digest": "sha256:e99241e77209b0b35a0fb59a61c0821fc161190178af7e4af45e6eeaa043375e" 2024-06-26T07:04:09.3901772Z }, 2024-06-26T07:04:09.3902019Z { 2024-06-26T07:04:09.3902438Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3902986Z "size": 291, 2024-06-26T07:04:09.3903535Z "digest": "sha256:35e8f4e2a42f9df0c969dedd2199cf71719cf5ecfa676dfa014bbdeb4fdb295f" 2024-06-26T07:04:09.3904226Z }, 2024-06-26T07:04:09.3904467Z { 2024-06-26T07:04:09.3904985Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3905549Z "size": 101, 2024-06-26T07:04:09.3906092Z "digest": "sha256:a823ffa4c541394b52953391dc0a6867e26c37fe4a71fa022a2cc3f07f0ba70d" 2024-06-26T07:04:09.3906895Z }, 2024-06-26T07:04:09.3907144Z { 2024-06-26T07:04:09.3907575Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3908118Z "size": 1473, 2024-06-26T07:04:09.3908630Z "digest": "sha256:44ba272eae2064603605a6238b3f35cc6578859d660068e702a047a9cd8ab2fd" 2024-06-26T07:04:09.3909235Z }, 2024-06-26T07:04:09.3909478Z { 2024-06-26T07:04:09.3909899Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3910452Z "size": 446710781, 2024-06-26T07:04:09.3911007Z "digest": "sha256:da02d278e5240c285bc855ce4e42cd6121d5b903787f5c14f03e575ad1268650" 2024-06-26T07:04:09.3911611Z }, 2024-06-26T07:04:09.3911855Z { 2024-06-26T07:04:09.3912276Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3912915Z "size": 161, 2024-06-26T07:04:09.3913431Z "digest": "sha256:73a0ce681c295135456e190578733273276b38b4dd5babbb39298f384d43b1c6" 2024-06-26T07:04:09.3914023Z }, 2024-06-26T07:04:09.3914268Z { 2024-06-26T07:04:09.3914690Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3915235Z "size": 564, 2024-06-26T07:04:09.3915759Z "digest": "sha256:49737bd5ae709dbf7956e21fc8b05ba009e9e6c9b130b0126a73bd3d3f5efc78" 2024-06-26T07:04:09.3916374Z }, 2024-06-26T07:04:09.3916643Z { 2024-06-26T07:04:09.3917081Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3917629Z "size": 35874600, 2024-06-26T07:04:09.3918182Z "digest": "sha256:9cd4c745f8e4f9d5c9bb8050923575afad21f5b7852975a4a1981bc9eff282c5" 2024-06-26T07:04:09.3918794Z }, 2024-06-26T07:04:09.3919032Z { 2024-06-26T07:04:09.3919459Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3920008Z "size": 104, 2024-06-26T07:04:09.3920523Z "digest": "sha256:b2950a0365345d4c17a48627c905fd8930b3b3154c727f0526eb9ca547102bb7" 2024-06-26T07:04:09.3921114Z }, 2024-06-26T07:04:09.3921363Z { 2024-06-26T07:04:09.3921783Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3922321Z "size": 425, 2024-06-26T07:04:09.3922838Z "digest": "sha256:25e19d81d9fc081962ff4cbc09e098f4035419822712c19a410f5282300dfb9e" 2024-06-26T07:04:09.3923446Z }, 2024-06-26T07:04:09.3923691Z { 2024-06-26T07:04:09.3924104Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3924666Z "size": 20262084, 2024-06-26T07:04:09.3925225Z "digest": "sha256:60156b40fcbec2c25f0170b0a82fa8cd9303e0daae8bb1811fd0fc6562e88e8e" 2024-06-26T07:04:09.3925844Z }, 2024-06-26T07:04:09.3926081Z { 2024-06-26T07:04:09.3926547Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3927098Z "size": 439, 2024-06-26T07:04:09.3927623Z "digest": "sha256:7a868239227fd19b30a19927bbc8f058d247403fe5b343c55f4b4b8987918911" 2024-06-26T07:04:09.3928219Z }, 2024-06-26T07:04:09.3928461Z { 2024-06-26T07:04:09.3928885Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3929420Z "size": 702, 2024-06-26T07:04:09.3929951Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:04:09.3930567Z }, 2024-06-26T07:04:09.3930813Z { 2024-06-26T07:04:09.3931229Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3931772Z "size": 143, 2024-06-26T07:04:09.3932294Z "digest": "sha256:83911d7c7ecf11b5932bb7449975c52e7f737a5b867691935e1ccb85ceb25b52" 2024-06-26T07:04:09.3932901Z }, 2024-06-26T07:04:09.3933139Z { 2024-06-26T07:04:09.3933561Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3934199Z "size": 135, 2024-06-26T07:04:09.3934731Z "digest": "sha256:b3c5d66c679f58c71febdd502889bd8cd3d36e5001754cdf1e30eabe21135cce" 2024-06-26T07:04:09.3935360Z }, 2024-06-26T07:04:09.3935602Z { 2024-06-26T07:04:09.3936028Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3936578Z "size": 4911495247, 2024-06-26T07:04:09.3937169Z "digest": "sha256:84b2c7a899820f6acdb22148c5e8346d8b26c33269fd01842c9573515251e1e1" 2024-06-26T07:04:09.3937771Z }, 2024-06-26T07:04:09.3938014Z { 2024-06-26T07:04:09.3938433Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3938976Z "size": 190, 2024-06-26T07:04:09.3939523Z "digest": "sha256:c1068d03a811d8dead5da9e82fdc8869205fc4ee9cb92d4f7a4acfbb5ba83950" 2024-06-26T07:04:09.3940139Z }, 2024-06-26T07:04:09.3940375Z { 2024-06-26T07:04:09.3940801Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3941345Z "size": 565, 2024-06-26T07:04:09.3941873Z "digest": "sha256:671b4a301d81074fca17d36851f81801552c4ecb2f90ff3412aa9ab6a1a91371" 2024-06-26T07:04:09.3942536Z }, 2024-06-26T07:04:09.3942778Z { 2024-06-26T07:04:09.3943203Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3943743Z "size": 43163095, 2024-06-26T07:04:09.3944297Z "digest": "sha256:67760cf5ec46ab58d3ca98637fd9b26cb47a67446bfd0db176fe5e023606aa0d" 2024-06-26T07:04:09.3945009Z }, 2024-06-26T07:04:09.3945253Z { 2024-06-26T07:04:09.3945667Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3946209Z "size": 106, 2024-06-26T07:04:09.3946958Z "digest": "sha256:ddc9740bcea2b301e99b50525059aea8d9c7121a1c8eab566561a8968438a8c2" 2024-06-26T07:04:09.3947571Z }, 2024-06-26T07:04:09.3947817Z { 2024-06-26T07:04:09.3948236Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3948780Z "size": 1213, 2024-06-26T07:04:09.3949309Z "digest": "sha256:fb2e977685609fcc01de170ba7f50a573e73e6520e47253717e92a28952c5ee8" 2024-06-26T07:04:09.3949926Z }, 2024-06-26T07:04:09.3950167Z { 2024-06-26T07:04:09.3950584Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3951140Z "size": 702, 2024-06-26T07:04:09.3951680Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:04:09.3952288Z }, 2024-06-26T07:04:09.3952530Z { 2024-06-26T07:04:09.3952943Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3953484Z "size": 138, 2024-06-26T07:04:09.3954015Z "digest": "sha256:b599c5a8c7cb461cbd7b3c98b99960b8eb04506e05e79db1cc0cfadb14a47e91" 2024-06-26T07:04:09.3954616Z }, 2024-06-26T07:04:09.3954859Z { 2024-06-26T07:04:09.3955277Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3955813Z "size": 120, 2024-06-26T07:04:09.3956357Z "digest": "sha256:e08e88c1366827455ee7ddbf92b6ae615ceb3cbd3fb11cd03ddd31bd3fa94a9c" 2024-06-26T07:04:09.3956999Z }, 2024-06-26T07:04:09.3957243Z { 2024-06-26T07:04:09.3957668Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3958200Z "size": 1904157495, 2024-06-26T07:04:09.3958764Z "digest": "sha256:12ba7431ff38de8cf6af3052a2a52e79f4e712b8c99f5bafde3c20e63b1f570f" 2024-06-26T07:04:09.3959375Z }, 2024-06-26T07:04:09.3959612Z { 2024-06-26T07:04:09.3960037Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3960579Z "size": 172, 2024-06-26T07:04:09.3961108Z "digest": "sha256:cf7c2cb896cba6a594028cc0fd246e127a2f4481d48905a3aa12c5f5872332d4" 2024-06-26T07:04:09.3961707Z }, 2024-06-26T07:04:09.3961951Z { 2024-06-26T07:04:09.3962369Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3962906Z "size": 908, 2024-06-26T07:04:09.3963431Z "digest": "sha256:3a778ddfac2af5b3a11f49ddf2f3f1ad1731f74ba691f434ab69966d1e2a189f" 2024-06-26T07:04:09.3964152Z }, 2024-06-26T07:04:09.3964401Z { 2024-06-26T07:04:09.3964820Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3965357Z "size": 702, 2024-06-26T07:04:09.3965883Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:04:09.3966492Z }, 2024-06-26T07:04:09.3966727Z { 2024-06-26T07:04:09.3967147Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3967685Z "size": 134, 2024-06-26T07:04:09.3968205Z "digest": "sha256:236c8f0245f4b40616a4322eaf1e1b3e178d5990db1fcaba39622ea4e79c06a1" 2024-06-26T07:04:09.3968800Z }, 2024-06-26T07:04:09.3969045Z { 2024-06-26T07:04:09.3969461Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3970004Z "size": 32, 2024-06-26T07:04:09.3970527Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:04:09.3971144Z }, 2024-06-26T07:04:09.3971390Z { 2024-06-26T07:04:09.3971802Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3972412Z "size": 156, 2024-06-26T07:04:09.3972945Z "digest": "sha256:dda1ce302b9a9df0c8f1145d2e3eaa03c96909bb45a792451f1fcb555b06738d" 2024-06-26T07:04:09.3973556Z }, 2024-06-26T07:04:09.3973789Z { 2024-06-26T07:04:09.3974206Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3974744Z "size": 1841, 2024-06-26T07:04:09.3975278Z "digest": "sha256:e797fe86caef19319e03f8dca92cf96838ab3bb3644f4447c6dfc04230118afa" 2024-06-26T07:04:09.3975872Z }, 2024-06-26T07:04:09.3976115Z { 2024-06-26T07:04:09.3976535Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3977127Z "size": 7529784, 2024-06-26T07:04:09.3977660Z "digest": "sha256:967190076fc0ab2bb2fc0d74e58edd3d662a537d702ff3f8b6e876230154a16f" 2024-06-26T07:04:09.3978260Z }, 2024-06-26T07:04:09.3978504Z { 2024-06-26T07:04:09.3978921Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3979465Z "size": 165, 2024-06-26T07:04:09.3979987Z "digest": "sha256:4fb0a83f06855534ef82b80dcf7c685303edb2bcbd446196066dc2ad2dd71281" 2024-06-26T07:04:09.3980586Z }, 2024-06-26T07:04:09.3980820Z { 2024-06-26T07:04:09.3981245Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3981786Z "size": 7944, 2024-06-26T07:04:09.3982323Z "digest": "sha256:68af2ec4e30a33f26947ece9f3a31b7d3b04c2bca5865d4b48618457832a1da2" 2024-06-26T07:04:09.3982917Z }, 2024-06-26T07:04:09.3983161Z { 2024-06-26T07:04:09.3983586Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3984125Z "size": 8070, 2024-06-26T07:04:09.3984724Z "digest": "sha256:cc91933e985a044c6df53caa15488d2285896f5e991f38ec068c253ceec53f14" 2024-06-26T07:04:09.3985334Z }, 2024-06-26T07:04:09.3985573Z { 2024-06-26T07:04:09.3986007Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3986761Z "size": 303, 2024-06-26T07:04:09.3987315Z "digest": "sha256:9f3187916e7dcad33cc572f8f0e949d2f507b887f99cfd6acefd14826b44960d" 2024-06-26T07:04:09.3987934Z }, 2024-06-26T07:04:09.3988168Z { 2024-06-26T07:04:09.3988592Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3989140Z "size": 7629071, 2024-06-26T07:04:09.3989662Z "digest": "sha256:52608399485738d10d779254946c9e1d536010dd45593c69bf1eb9f62f85dc24" 2024-06-26T07:04:09.3990246Z }, 2024-06-26T07:04:09.3990494Z { 2024-06-26T07:04:09.3990916Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3991459Z "size": 108, 2024-06-26T07:04:09.3991998Z "digest": "sha256:05a6f6dcc1c00bac6660923a22adc7d7c34416718f8c89daf9b3351d9365f694" 2024-06-26T07:04:09.3992621Z }, 2024-06-26T07:04:09.3992869Z { 2024-06-26T07:04:09.3993283Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3993944Z "size": 54145775, 2024-06-26T07:04:09.3994506Z "digest": "sha256:6bc9dc8315b09b27c976bfba9700767ba412d72fd0528b18bbe8f8e62005ec22" 2024-06-26T07:04:09.3995125Z }, 2024-06-26T07:04:09.3995360Z { 2024-06-26T07:04:09.3995784Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3996326Z "size": 474, 2024-06-26T07:04:09.3996860Z "digest": "sha256:1b0e27c35f07b3fbf0aeb548d1949bf1d60a420136da21789d94d196c891d981" 2024-06-26T07:04:09.3997465Z }, 2024-06-26T07:04:09.3997706Z { 2024-06-26T07:04:09.3998132Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.3998673Z "size": 1374858747, 2024-06-26T07:04:09.3999226Z "digest": "sha256:95237e7311a1c5ce6937db7196c866632f44e8788514a5c0eb4a3f04da4017e6" 2024-06-26T07:04:09.3999822Z }, 2024-06-26T07:04:09.4000062Z { 2024-06-26T07:04:09.4000475Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.4001026Z "size": 106, 2024-06-26T07:04:09.4001577Z "digest": "sha256:789eff6c51aada70bf75eb1acafed63c0d839b768638001500616ff6e53b854a" 2024-06-26T07:04:09.4002306Z }, 2024-06-26T07:04:09.4002543Z { 2024-06-26T07:04:09.4002966Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.4003507Z "size": 558, 2024-06-26T07:04:09.4004040Z "digest": "sha256:de0f8a8c45c6077b3c5ba3b84d7f07de98353fd618dad9a1e2f368ffe00493c7" 2024-06-26T07:04:09.4004657Z }, 2024-06-26T07:04:09.4004904Z { 2024-06-26T07:04:09.4005322Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.4005862Z "size": 46248513, 2024-06-26T07:04:09.4006405Z "digest": "sha256:2060028f74888d218bebdd10d723f66c33ee1c708935c4ffd4654d34912ac4dc" 2024-06-26T07:04:09.4007058Z }, 2024-06-26T07:04:09.4007300Z { 2024-06-26T07:04:09.4007714Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.4008257Z "size": 111, 2024-06-26T07:04:09.4008800Z "digest": "sha256:e412c7f3b2ded440992955bd92512345cbece8a8894cd3592f714ebc758ad1a0" 2024-06-26T07:04:09.4009418Z }, 2024-06-26T07:04:09.4009655Z { 2024-06-26T07:04:09.4010081Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.4010625Z "size": 32, 2024-06-26T07:04:09.4011154Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:04:09.4011770Z }, 2024-06-26T07:04:09.4012014Z { 2024-06-26T07:04:09.4012437Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.4012973Z "size": 32, 2024-06-26T07:04:09.4013506Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:04:09.4014124Z }, 2024-06-26T07:04:09.4014372Z { 2024-06-26T07:04:09.4014790Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.4015338Z "size": 32, 2024-06-26T07:04:09.4015874Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:04:09.4016496Z }, 2024-06-26T07:04:09.4016779Z { 2024-06-26T07:04:09.4017212Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:04:09.4017763Z "size": 32, 2024-06-26T07:04:09.4018291Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:04:09.4018902Z } 2024-06-26T07:04:09.4032136Z ] 2024-06-26T07:04:09.4032414Z } 2024-06-26T07:04:09.4132147Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:04:09.4132638Z tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:04:09.4133191Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-06-26T07:04:09.4141549Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:09.4142046Z env: 2024-06-26T07:04:09.4142330Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:09.4143475Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:09.4144722Z ##[endgroup] 2024-06-26T07:04:09.4168280Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:09.4217540Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-06-26T07:04:09.4218121Z with: 2024-06-26T07:04:09.4219187Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:09.4220497Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:04:09.4221015Z env: 2024-06-26T07:04:09.4221292Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:09.4221646Z ##[endgroup] 2024-06-26T07:04:09.4237002Z ##[group]Run set -x 2024-06-26T07:04:09.4237344Z set -x 2024-06-26T07:04:09.4237648Z set +e 2024-06-26T07:04:09.4237944Z  2024-06-26T07:04:09.4238214Z login() { 2024-06-26T07:04:09.4238879Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:04:09.4239731Z } 2024-06-26T07:04:09.4240018Z  2024-06-26T07:04:09.4240331Z retry () { 2024-06-26T07:04:09.4240734Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:04:09.4241183Z } 2024-06-26T07:04:09.4241463Z  2024-06-26T07:04:09.4241780Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:04:09.4242199Z  2024-06-26T07:04:09.4242480Z set -e 2024-06-26T07:04:09.4242963Z # ignore output since only exit code is used for conditional 2024-06-26T07:04:09.4243671Z # only pull docker image if it's not available locally 2024-06-26T07:04:09.4244436Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-06-26T07:04:09.4245137Z  retry docker pull "${DOCKER_IMAGE}" 2024-06-26T07:04:09.4245584Z fi 2024-06-26T07:04:09.4253059Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:04:09.4253556Z env: 2024-06-26T07:04:09.4253831Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:04:09.4254978Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:09.4256285Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:04:09.4256798Z ##[endgroup] 2024-06-26T07:04:09.4276490Z + set +e 2024-06-26T07:04:09.4277110Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:04:09.4277759Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:04:09.4278597Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:04:09.4279514Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:04:09.8366211Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:04:09.8367082Z Configure a credential helper to remove this warning. See 2024-06-26T07:04:09.8367934Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:04:09.8368471Z 2024-06-26T07:04:09.8368591Z Login Succeeded 2024-06-26T07:04:09.8380226Z + set -e 2024-06-26T07:04:09.8381807Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:09.8508293Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:09.8510471Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:04:10.0650588Z 91382da70d5719cd7007b6b80b71d2f48398f6b7: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-06-26T07:04:10.0651959Z 7a2c55901189: Pulling fs layer 2024-06-26T07:04:10.0652480Z 224fe954d725: Pulling fs layer 2024-06-26T07:04:10.0653222Z 75722010b82e: Pulling fs layer 2024-06-26T07:04:10.0653748Z d527cbbb87e3: Pulling fs layer 2024-06-26T07:04:10.0654255Z b57676e46aee: Pulling fs layer 2024-06-26T07:04:10.0654752Z a8c1e85b5e14: Pulling fs layer 2024-06-26T07:04:10.0655248Z a41a8d1c11c8: Pulling fs layer 2024-06-26T07:04:10.0655635Z 0c1227890755: Pulling fs layer 2024-06-26T07:04:10.0656015Z d8d1234baab3: Pulling fs layer 2024-06-26T07:04:10.0656394Z 7ed32bc8e469: Pulling fs layer 2024-06-26T07:04:10.0656789Z ec1e7978c1fe: Pulling fs layer 2024-06-26T07:04:10.0657171Z 309b872da805: Pulling fs layer 2024-06-26T07:04:10.0657551Z e82932cb002c: Pulling fs layer 2024-06-26T07:04:10.0657928Z ca9dd73e3e93: Pulling fs layer 2024-06-26T07:04:10.0658312Z 16d76a79d5d6: Pulling fs layer 2024-06-26T07:04:10.0658701Z 4bdf41c9ecbe: Pulling fs layer 2024-06-26T07:04:10.0659078Z f4863801e30c: Pulling fs layer 2024-06-26T07:04:10.0659572Z d37dd9ed8948: Pulling fs layer 2024-06-26T07:04:10.0659958Z cf2f34f225db: Pulling fs layer 2024-06-26T07:04:10.0660351Z df546d7b3e34: Pulling fs layer 2024-06-26T07:04:10.0660720Z 3d914950f8ad: Pulling fs layer 2024-06-26T07:04:10.0661093Z 96d0e2032184: Pulling fs layer 2024-06-26T07:04:10.0661465Z 06e12ce59790: Pulling fs layer 2024-06-26T07:04:10.0661838Z ed712f63cde7: Pulling fs layer 2024-06-26T07:04:10.0662209Z a97a71ef8a4e: Pulling fs layer 2024-06-26T07:04:10.0662650Z 25cd0c482ad6: Pulling fs layer 2024-06-26T07:04:10.0663158Z b57676e46aee: Waiting 2024-06-26T07:04:10.0663651Z 9bf94bc45658: Pulling fs layer 2024-06-26T07:04:10.0664183Z a8c1e85b5e14: Waiting 2024-06-26T07:04:10.0664696Z de45a14c9b22: Pulling fs layer 2024-06-26T07:04:10.0665077Z e99241e77209: Pulling fs layer 2024-06-26T07:04:10.0665435Z a41a8d1c11c8: Waiting 2024-06-26T07:04:10.0665759Z 0c1227890755: Waiting 2024-06-26T07:04:10.0666076Z d8d1234baab3: Waiting 2024-06-26T07:04:10.0666387Z 7ed32bc8e469: Waiting 2024-06-26T07:04:10.0666993Z 309b872da805: Waiting 2024-06-26T07:04:10.0667335Z e82932cb002c: Waiting 2024-06-26T07:04:10.0667718Z 35e8f4e2a42f: Pulling fs layer 2024-06-26T07:04:10.0668088Z d527cbbb87e3: Waiting 2024-06-26T07:04:10.0668400Z 25cd0c482ad6: Waiting 2024-06-26T07:04:10.0668733Z a823ffa4c541: Pulling fs layer 2024-06-26T07:04:10.0669114Z 44ba272eae20: Pulling fs layer 2024-06-26T07:04:10.0669491Z da02d278e524: Pulling fs layer 2024-06-26T07:04:10.0669865Z 73a0ce681c29: Pulling fs layer 2024-06-26T07:04:10.0670226Z 96d0e2032184: Waiting 2024-06-26T07:04:10.0670545Z 06e12ce59790: Waiting 2024-06-26T07:04:10.0670853Z 9bf94bc45658: Waiting 2024-06-26T07:04:10.0671168Z 16d76a79d5d6: Waiting 2024-06-26T07:04:10.0671486Z a97a71ef8a4e: Waiting 2024-06-26T07:04:10.0671870Z de45a14c9b22: Waiting 2024-06-26T07:04:10.0672209Z 4bdf41c9ecbe: Waiting 2024-06-26T07:04:10.0672654Z e99241e77209: Waiting 2024-06-26T07:04:10.0673101Z d37dd9ed8948: Waiting 2024-06-26T07:04:10.0673540Z 35e8f4e2a42f: Waiting 2024-06-26T07:04:10.0673898Z a823ffa4c541: Waiting 2024-06-26T07:04:10.0674218Z ec1e7978c1fe: Waiting 2024-06-26T07:04:10.0674531Z f4863801e30c: Waiting 2024-06-26T07:04:10.0674845Z 3d914950f8ad: Waiting 2024-06-26T07:04:10.0675166Z df546d7b3e34: Waiting 2024-06-26T07:04:10.0675502Z 49737bd5ae70: Pulling fs layer 2024-06-26T07:04:10.0675867Z da02d278e524: Waiting 2024-06-26T07:04:10.0676194Z 9cd4c745f8e4: Pulling fs layer 2024-06-26T07:04:10.0676579Z b2950a036534: Pulling fs layer 2024-06-26T07:04:10.0676938Z ed712f63cde7: Waiting 2024-06-26T07:04:10.0677272Z 25e19d81d9fc: Pulling fs layer 2024-06-26T07:04:10.0677656Z 60156b40fcbe: Pulling fs layer 2024-06-26T07:04:10.0678019Z 9cd4c745f8e4: Waiting 2024-06-26T07:04:10.0678347Z 7a868239227f: Pulling fs layer 2024-06-26T07:04:10.0678695Z b2950a036534: Waiting 2024-06-26T07:04:10.0679017Z cf2f34f225db: Waiting 2024-06-26T07:04:10.0679334Z 49737bd5ae70: Waiting 2024-06-26T07:04:10.0679646Z 60156b40fcbe: Waiting 2024-06-26T07:04:10.0679967Z 83911d7c7ecf: Pulling fs layer 2024-06-26T07:04:10.0680351Z b3c5d66c679f: Pulling fs layer 2024-06-26T07:04:10.0680727Z 84b2c7a89982: Pulling fs layer 2024-06-26T07:04:10.0681235Z 83911d7c7ecf: Waiting 2024-06-26T07:04:10.0681569Z ca9dd73e3e93: Waiting 2024-06-26T07:04:10.0681898Z c1068d03a811: Pulling fs layer 2024-06-26T07:04:10.0682346Z 671b4a301d81: Pulling fs layer 2024-06-26T07:04:10.0682717Z 67760cf5ec46: Pulling fs layer 2024-06-26T07:04:10.0683079Z 671b4a301d81: Waiting 2024-06-26T07:04:10.0683398Z b3c5d66c679f: Waiting 2024-06-26T07:04:10.0683758Z ddc9740bcea2: Pulling fs layer 2024-06-26T07:04:10.0684141Z 84b2c7a89982: Waiting 2024-06-26T07:04:10.0684470Z fb2e97768560: Pulling fs layer 2024-06-26T07:04:10.0684830Z 67760cf5ec46: Waiting 2024-06-26T07:04:10.0685198Z b599c5a8c7cb: Pulling fs layer 2024-06-26T07:04:10.0685568Z ddc9740bcea2: Waiting 2024-06-26T07:04:10.0685965Z e08e88c13668: Pulling fs layer 2024-06-26T07:04:10.0686349Z 12ba7431ff38: Pulling fs layer 2024-06-26T07:04:10.0686725Z cf7c2cb896cb: Pulling fs layer 2024-06-26T07:04:10.0687216Z 3a778ddfac2a: Pulling fs layer 2024-06-26T07:04:10.0687586Z b599c5a8c7cb: Waiting 2024-06-26T07:04:10.0687914Z 12ba7431ff38: Waiting 2024-06-26T07:04:10.0688239Z 25e19d81d9fc: Waiting 2024-06-26T07:04:10.0688563Z cf7c2cb896cb: Waiting 2024-06-26T07:04:10.0688881Z 3a778ddfac2a: Waiting 2024-06-26T07:04:10.0689190Z e08e88c13668: Waiting 2024-06-26T07:04:10.0689521Z 236c8f0245f4: Pulling fs layer 2024-06-26T07:04:10.0689890Z c1068d03a811: Waiting 2024-06-26T07:04:10.0690225Z 4f4fb700ef54: Pulling fs layer 2024-06-26T07:04:10.0690597Z dda1ce302b9a: Pulling fs layer 2024-06-26T07:04:10.0690976Z e797fe86caef: Pulling fs layer 2024-06-26T07:04:10.0691352Z 967190076fc0: Pulling fs layer 2024-06-26T07:04:10.0691712Z 4fb0a83f0685: Pulling fs layer 2024-06-26T07:04:10.0692086Z 68af2ec4e30a: Pulling fs layer 2024-06-26T07:04:10.0692460Z cc91933e985a: Pulling fs layer 2024-06-26T07:04:10.0692833Z 9f3187916e7d: Pulling fs layer 2024-06-26T07:04:10.0693220Z 526083994857: Pulling fs layer 2024-06-26T07:04:10.0693634Z 05a6f6dcc1c0: Pulling fs layer 2024-06-26T07:04:10.0694050Z 6bc9dc8315b0: Pulling fs layer 2024-06-26T07:04:10.0694434Z 1b0e27c35f07: Pulling fs layer 2024-06-26T07:04:10.0694810Z 95237e7311a1: Pulling fs layer 2024-06-26T07:04:10.0695181Z 789eff6c51aa: Pulling fs layer 2024-06-26T07:04:10.0695559Z de0f8a8c45c6: Pulling fs layer 2024-06-26T07:04:10.0695923Z 2060028f7488: Pulling fs layer 2024-06-26T07:04:10.0696294Z 6bc9dc8315b0: Waiting 2024-06-26T07:04:10.0696635Z e412c7f3b2de: Pulling fs layer 2024-06-26T07:04:10.0696995Z 9f3187916e7d: Waiting 2024-06-26T07:04:10.0697301Z 236c8f0245f4: Waiting 2024-06-26T07:04:10.0697617Z 1b0e27c35f07: Waiting 2024-06-26T07:04:10.0697930Z 526083994857: Waiting 2024-06-26T07:04:10.0698250Z 4fb0a83f0685: Waiting 2024-06-26T07:04:10.0698564Z 4f4fb700ef54: Waiting 2024-06-26T07:04:10.0698886Z 05a6f6dcc1c0: Waiting 2024-06-26T07:04:10.0699201Z 95237e7311a1: Waiting 2024-06-26T07:04:10.0699516Z dda1ce302b9a: Waiting 2024-06-26T07:04:10.0699833Z 789eff6c51aa: Waiting 2024-06-26T07:04:10.0700163Z 2060028f7488: Waiting 2024-06-26T07:04:10.0700481Z cc91933e985a: Waiting 2024-06-26T07:04:10.0700789Z 967190076fc0: Waiting 2024-06-26T07:04:10.2054532Z 224fe954d725: Verifying Checksum 2024-06-26T07:04:10.2054976Z 224fe954d725: Download complete 2024-06-26T07:04:10.2739964Z d527cbbb87e3: Verifying Checksum 2024-06-26T07:04:10.2740488Z d527cbbb87e3: Download complete 2024-06-26T07:04:10.3459816Z b57676e46aee: Verifying Checksum 2024-06-26T07:04:10.3460309Z b57676e46aee: Download complete 2024-06-26T07:04:10.4096896Z 7a2c55901189: Verifying Checksum 2024-06-26T07:04:10.4097428Z 7a2c55901189: Download complete 2024-06-26T07:04:10.4888720Z a41a8d1c11c8: Download complete 2024-06-26T07:04:10.5672609Z 0c1227890755: Download complete 2024-06-26T07:04:10.6489578Z d8d1234baab3: Verifying Checksum 2024-06-26T07:04:10.6490102Z d8d1234baab3: Download complete 2024-06-26T07:04:10.6850561Z 75722010b82e: Verifying Checksum 2024-06-26T07:04:10.6851124Z 75722010b82e: Download complete 2024-06-26T07:04:10.7958345Z ec1e7978c1fe: Download complete 2024-06-26T07:04:10.8795132Z 309b872da805: Download complete 2024-06-26T07:04:11.4539162Z 7a2c55901189: Pull complete 2024-06-26T07:04:11.7617898Z 224fe954d725: Pull complete 2024-06-26T07:04:12.7206192Z 75722010b82e: Pull complete 2024-06-26T07:04:12.7590792Z d527cbbb87e3: Pull complete 2024-06-26T07:04:12.8424486Z b57676e46aee: Pull complete 2024-06-26T07:04:13.4086884Z e82932cb002c: Verifying Checksum 2024-06-26T07:04:13.4087452Z e82932cb002c: Download complete 2024-06-26T07:04:13.4888923Z ca9dd73e3e93: Download complete 2024-06-26T07:04:13.5635420Z 16d76a79d5d6: Verifying Checksum 2024-06-26T07:04:13.5635988Z 16d76a79d5d6: Download complete 2024-06-26T07:04:13.6419863Z 4bdf41c9ecbe: Download complete 2024-06-26T07:04:14.6191060Z f4863801e30c: Verifying Checksum 2024-06-26T07:04:14.6191713Z f4863801e30c: Download complete 2024-06-26T07:04:14.6851488Z d37dd9ed8948: Verifying Checksum 2024-06-26T07:04:14.6851921Z d37dd9ed8948: Download complete 2024-06-26T07:04:14.7548161Z cf2f34f225db: Download complete 2024-06-26T07:04:14.8486151Z df546d7b3e34: Verifying Checksum 2024-06-26T07:04:14.8486598Z df546d7b3e34: Download complete 2024-06-26T07:04:23.6871005Z a8c1e85b5e14: Verifying Checksum 2024-06-26T07:04:23.6871506Z a8c1e85b5e14: Download complete 2024-06-26T07:04:23.7488795Z 96d0e2032184: Verifying Checksum 2024-06-26T07:04:23.7489274Z 96d0e2032184: Download complete 2024-06-26T07:04:23.8273772Z 06e12ce59790: Verifying Checksum 2024-06-26T07:04:23.8274256Z 06e12ce59790: Download complete 2024-06-26T07:04:23.9313829Z ed712f63cde7: Verifying Checksum 2024-06-26T07:04:23.9314497Z ed712f63cde7: Download complete 2024-06-26T07:04:23.9996591Z a97a71ef8a4e: Verifying Checksum 2024-06-26T07:04:24.0711223Z 25cd0c482ad6: Verifying Checksum 2024-06-26T07:04:24.0711727Z 25cd0c482ad6: Download complete 2024-06-26T07:04:25.3532123Z 9bf94bc45658: Verifying Checksum 2024-06-26T07:04:25.3532638Z 9bf94bc45658: Download complete 2024-06-26T07:04:25.4210423Z de45a14c9b22: Download complete 2024-06-26T07:04:25.5002724Z e99241e77209: Verifying Checksum 2024-06-26T07:04:25.5003383Z e99241e77209: Download complete 2024-06-26T07:04:25.5762598Z 35e8f4e2a42f: Verifying Checksum 2024-06-26T07:04:25.5763069Z 35e8f4e2a42f: Download complete 2024-06-26T07:04:25.6584104Z a823ffa4c541: Verifying Checksum 2024-06-26T07:04:25.6584572Z a823ffa4c541: Download complete 2024-06-26T07:04:25.7352390Z 44ba272eae20: Verifying Checksum 2024-06-26T07:04:25.7352853Z 44ba272eae20: Download complete 2024-06-26T07:04:30.2516039Z da02d278e524: Verifying Checksum 2024-06-26T07:04:30.2516516Z da02d278e524: Download complete 2024-06-26T07:04:30.3599109Z 73a0ce681c29: Verifying Checksum 2024-06-26T07:04:30.3599757Z 73a0ce681c29: Download complete 2024-06-26T07:04:30.4314953Z 49737bd5ae70: Download complete 2024-06-26T07:04:30.8412378Z 9cd4c745f8e4: Verifying Checksum 2024-06-26T07:04:30.8412855Z 9cd4c745f8e4: Download complete 2024-06-26T07:04:30.9170477Z b2950a036534: Verifying Checksum 2024-06-26T07:04:30.9171149Z b2950a036534: Download complete 2024-06-26T07:04:31.0078152Z 25e19d81d9fc: Download complete 2024-06-26T07:04:31.2560995Z 60156b40fcbe: Verifying Checksum 2024-06-26T07:04:31.2561550Z 60156b40fcbe: Download complete 2024-06-26T07:04:31.3387436Z 7a868239227f: Verifying Checksum 2024-06-26T07:04:31.3388010Z 7a868239227f: Download complete 2024-06-26T07:04:31.4097491Z 83911d7c7ecf: Verifying Checksum 2024-06-26T07:04:31.4098056Z 83911d7c7ecf: Download complete 2024-06-26T07:04:31.4946198Z b3c5d66c679f: Download complete 2024-06-26T07:04:35.9869529Z 7ed32bc8e469: Verifying Checksum 2024-06-26T07:04:35.9870146Z 7ed32bc8e469: Download complete 2024-06-26T07:04:36.0657487Z c1068d03a811: Download complete 2024-06-26T07:04:36.1445329Z 671b4a301d81: Download complete 2024-06-26T07:04:36.5634099Z a8c1e85b5e14: Pull complete 2024-06-26T07:04:36.6416774Z 67760cf5ec46: Verifying Checksum 2024-06-26T07:04:36.6417380Z 67760cf5ec46: Download complete 2024-06-26T07:04:36.7110301Z ddc9740bcea2: Verifying Checksum 2024-06-26T07:04:36.7110894Z ddc9740bcea2: Download complete 2024-06-26T07:04:36.7847150Z fb2e97768560: Verifying Checksum 2024-06-26T07:04:36.8394260Z a41a8d1c11c8: Pull complete 2024-06-26T07:04:36.8581925Z b599c5a8c7cb: Verifying Checksum 2024-06-26T07:04:36.8582463Z b599c5a8c7cb: Download complete 2024-06-26T07:04:36.9385438Z e08e88c13668: Verifying Checksum 2024-06-26T07:04:36.9386016Z e08e88c13668: Download complete 2024-06-26T07:04:37.1250779Z 0c1227890755: Pull complete 2024-06-26T07:04:37.4081972Z d8d1234baab3: Pull complete 2024-06-26T07:04:42.7488016Z 3d914950f8ad: Verifying Checksum 2024-06-26T07:04:42.7488755Z 3d914950f8ad: Download complete 2024-06-26T07:04:42.8223072Z cf7c2cb896cb: Verifying Checksum 2024-06-26T07:04:42.8223524Z cf7c2cb896cb: Download complete 2024-06-26T07:04:42.8974767Z 3a778ddfac2a: Verifying Checksum 2024-06-26T07:04:42.8975301Z 3a778ddfac2a: Download complete 2024-06-26T07:04:42.9803096Z 236c8f0245f4: Download complete 2024-06-26T07:04:42.9884977Z 4f4fb700ef54: Download complete 2024-06-26T07:04:43.0665537Z dda1ce302b9a: Download complete 2024-06-26T07:04:43.1532325Z e797fe86caef: Download complete 2024-06-26T07:04:43.2905578Z 967190076fc0: Verifying Checksum 2024-06-26T07:04:43.2906020Z 967190076fc0: Download complete 2024-06-26T07:04:43.3796827Z 4fb0a83f0685: Verifying Checksum 2024-06-26T07:04:43.3797304Z 4fb0a83f0685: Download complete 2024-06-26T07:04:43.4656193Z 68af2ec4e30a: Verifying Checksum 2024-06-26T07:04:43.4656700Z 68af2ec4e30a: Download complete 2024-06-26T07:04:43.5431817Z cc91933e985a: Download complete 2024-06-26T07:04:43.6018110Z 9f3187916e7d: Verifying Checksum 2024-06-26T07:04:43.6018695Z 9f3187916e7d: Download complete 2024-06-26T07:04:43.7453744Z 526083994857: Verifying Checksum 2024-06-26T07:04:43.7454284Z 526083994857: Download complete 2024-06-26T07:04:43.8097070Z 05a6f6dcc1c0: Download complete 2024-06-26T07:04:44.4102986Z 6bc9dc8315b0: Verifying Checksum 2024-06-26T07:04:44.4103516Z 6bc9dc8315b0: Download complete 2024-06-26T07:04:44.4892422Z 1b0e27c35f07: Download complete 2024-06-26T07:05:11.3463056Z 12ba7431ff38: Verifying Checksum 2024-06-26T07:05:11.3463671Z 12ba7431ff38: Download complete 2024-06-26T07:05:11.4123760Z 789eff6c51aa: Verifying Checksum 2024-06-26T07:05:11.4124206Z 789eff6c51aa: Download complete 2024-06-26T07:05:11.4781530Z de0f8a8c45c6: Download complete 2024-06-26T07:05:12.8743186Z 2060028f7488: Verifying Checksum 2024-06-26T07:05:12.8743647Z 2060028f7488: Download complete 2024-06-26T07:05:12.9506876Z e412c7f3b2de: Verifying Checksum 2024-06-26T07:05:12.9507751Z e412c7f3b2de: Download complete 2024-06-26T07:05:19.7563065Z 95237e7311a1: Verifying Checksum 2024-06-26T07:05:19.7563697Z 95237e7311a1: Download complete 2024-06-26T07:06:05.8354916Z 84b2c7a89982: Verifying Checksum 2024-06-26T07:06:05.8356087Z 84b2c7a89982: Download complete 2024-06-26T07:06:28.6335961Z 7ed32bc8e469: Pull complete 2024-06-26T07:06:28.9300220Z ec1e7978c1fe: Pull complete 2024-06-26T07:06:29.2248943Z 309b872da805: Pull complete 2024-06-26T07:06:36.6981565Z e82932cb002c: Pull complete 2024-06-26T07:06:37.1697087Z ca9dd73e3e93: Pull complete 2024-06-26T07:06:37.6677492Z 16d76a79d5d6: Pull complete 2024-06-26T07:06:38.1207508Z 4bdf41c9ecbe: Pull complete 2024-06-26T07:06:40.6924846Z f4863801e30c: Pull complete 2024-06-26T07:06:41.0029133Z d37dd9ed8948: Pull complete 2024-06-26T07:06:41.2867974Z cf2f34f225db: Pull complete 2024-06-26T07:06:41.5509694Z df546d7b3e34: Pull complete 2024-06-26T07:07:42.7933989Z 3d914950f8ad: Pull complete 2024-06-26T07:07:43.0292949Z 96d0e2032184: Pull complete 2024-06-26T07:07:43.1933134Z 06e12ce59790: Pull complete 2024-06-26T07:07:43.3753336Z ed712f63cde7: Pull complete 2024-06-26T07:07:43.5742149Z a97a71ef8a4e: Pull complete 2024-06-26T07:07:43.7727138Z 25cd0c482ad6: Pull complete 2024-06-26T07:07:47.1921780Z 9bf94bc45658: Pull complete 2024-06-26T07:07:47.2987436Z de45a14c9b22: Pull complete 2024-06-26T07:07:47.4107963Z e99241e77209: Pull complete 2024-06-26T07:07:47.5735695Z 35e8f4e2a42f: Pull complete 2024-06-26T07:07:47.8316873Z a823ffa4c541: Pull complete 2024-06-26T07:07:48.2231772Z 44ba272eae20: Pull complete 2024-06-26T07:07:56.5739242Z da02d278e524: Pull complete 2024-06-26T07:07:56.7446483Z 73a0ce681c29: Pull complete 2024-06-26T07:07:56.9291542Z 49737bd5ae70: Pull complete 2024-06-26T07:07:57.8502818Z 9cd4c745f8e4: Pull complete 2024-06-26T07:07:58.1109158Z b2950a036534: Pull complete 2024-06-26T07:07:58.3893568Z 25e19d81d9fc: Pull complete 2024-06-26T07:07:58.8287172Z 60156b40fcbe: Pull complete 2024-06-26T07:07:58.9791281Z 7a868239227f: Pull complete 2024-06-26T07:07:59.3363292Z 83911d7c7ecf: Pull complete 2024-06-26T07:07:59.4989053Z b3c5d66c679f: Pull complete 2024-06-26T07:08:38.5061605Z 84b2c7a89982: Pull complete 2024-06-26T07:08:38.7189607Z c1068d03a811: Pull complete 2024-06-26T07:08:38.9374218Z 671b4a301d81: Pull complete 2024-06-26T07:08:41.0075701Z 67760cf5ec46: Pull complete 2024-06-26T07:08:41.1230215Z ddc9740bcea2: Pull complete 2024-06-26T07:08:41.2253591Z fb2e97768560: Pull complete 2024-06-26T07:08:41.5269736Z b599c5a8c7cb: Pull complete 2024-06-26T07:08:41.6278517Z e08e88c13668: Pull complete 2024-06-26T07:09:16.0738752Z 12ba7431ff38: Pull complete 2024-06-26T07:09:16.2969882Z cf7c2cb896cb: Pull complete 2024-06-26T07:09:16.5208706Z 3a778ddfac2a: Pull complete 2024-06-26T07:09:16.9724259Z 236c8f0245f4: Pull complete 2024-06-26T07:09:17.1984733Z 4f4fb700ef54: Pull complete 2024-06-26T07:09:17.4171913Z dda1ce302b9a: Pull complete 2024-06-26T07:09:17.6398941Z e797fe86caef: Pull complete 2024-06-26T07:09:18.0221204Z 967190076fc0: Pull complete 2024-06-26T07:09:18.2356420Z 4fb0a83f0685: Pull complete 2024-06-26T07:09:18.4546415Z 68af2ec4e30a: Pull complete 2024-06-26T07:09:18.6801345Z cc91933e985a: Pull complete 2024-06-26T07:09:18.8914672Z 9f3187916e7d: Pull complete 2024-06-26T07:09:19.9507338Z 526083994857: Pull complete 2024-06-26T07:09:20.1154960Z 05a6f6dcc1c0: Pull complete 2024-06-26T07:09:22.1579655Z 6bc9dc8315b0: Pull complete 2024-06-26T07:09:22.3818466Z 1b0e27c35f07: Pull complete 2024-06-26T07:09:35.9644193Z 95237e7311a1: Pull complete 2024-06-26T07:09:36.1829500Z 789eff6c51aa: Pull complete 2024-06-26T07:09:36.4056207Z de0f8a8c45c6: Pull complete 2024-06-26T07:09:37.2764020Z 2060028f7488: Pull complete 2024-06-26T07:09:37.5060001Z e412c7f3b2de: Pull complete 2024-06-26T07:09:38.5745458Z Digest: sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb 2024-06-26T07:09:38.6156879Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:09:38.6351041Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:09:38.6394562Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:09:38.6395617Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:09:38.6404045Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:09:38.6404663Z env: 2024-06-26T07:09:38.6405030Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:09:38.6405469Z ##[endgroup] 2024-06-26T07:09:38.6565726Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-06-26T07:09:38.6566274Z with: 2024-06-26T07:09:38.6566563Z driver-version: 550.54.15 2024-06-26T07:09:38.6566916Z env: 2024-06-26T07:09:38.6567187Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:09:38.6567541Z ##[endgroup] 2024-06-26T07:09:38.6660512Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:09:38.6661068Z with: 2024-06-26T07:09:38.6661344Z timeout_minutes: 10 2024-06-26T07:09:38.6661661Z max_attempts: 3 2024-06-26T07:09:38.6693207Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" 2024-06-26T07:09:38.6725103Z retry_wait_seconds: 10 2024-06-26T07:09:38.6725478Z polling_interval_seconds: 1 2024-06-26T07:09:38.6725975Z warning_on_retry: true 2024-06-26T07:09:38.6726346Z continue_on_error: false 2024-06-26T07:09:38.6726702Z env: 2024-06-26T07:09:38.6726981Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:09:38.6727352Z DRIVER_VERSION: 550.54.15 2024-06-26T07:09:38.6727716Z ##[endgroup] 2024-06-26T07:09:38.7250183Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-06-26T07:09:38.7251133Z + pre_install_nvidia_driver_amzn2 2024-06-26T07:09:38.7251771Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-06-26T07:09:39.0379178Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:09:39.0777410Z No Match for argument: nvidia-driver-latest-dkms 2024-06-26T07:09:39.1018855Z No Packages marked for removal 2024-06-26T07:09:39.1137153Z + install_nvidia_driver_common 2024-06-26T07:09:39.1139436Z + echo 'Before installing NVIDIA driver' 2024-06-26T07:09:39.1140028Z + lspci 2024-06-26T07:09:39.1140408Z Before installing NVIDIA driver 2024-06-26T07:09:39.1218425Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-06-26T07:09:39.1219450Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T07:09:39.1220489Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-06-26T07:09:39.1221367Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-06-26T07:09:39.1222203Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 2024-06-26T07:09:39.1223035Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T07:09:39.1223834Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1) 2024-06-26T07:09:39.1224795Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-06-26T07:09:39.1225511Z + lsmod 2024-06-26T07:09:39.1233093Z Module Size Used by 2024-06-26T07:09:39.1233598Z xt_nat 16384 0 2024-06-26T07:09:39.1234056Z ib_core 266240 0 2024-06-26T07:09:39.1234523Z nvidia_modeset 1339392 0 2024-06-26T07:09:39.1235042Z veth 16384 0 2024-06-26T07:09:39.1235831Z nvidia_uvm 4599808 0 2024-06-26T07:09:39.1236540Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:09:39.1237177Z drm 421888 1 nvidia 2024-06-26T07:09:39.1237703Z i2c_core 77824 2 nvidia,drm 2024-06-26T07:09:39.1238264Z backlight 16384 1 nvidia_modeset 2024-06-26T07:09:39.1238796Z xt_conntrack 16384 1 2024-06-26T07:09:39.1239339Z ipt_MASQUERADE 16384 1 2024-06-26T07:09:39.1239858Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T07:09:39.1240416Z nf_conntrack_netlink 49152 0 2024-06-26T07:09:39.1240993Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T07:09:39.1241543Z xfrm_user 45056 1 2024-06-26T07:09:39.1242014Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:09:39.1242513Z iptable_nat 16384 1 2024-06-26T07:09:39.1242996Z nf_conntrack_ipv4 16384 3 2024-06-26T07:09:39.1243519Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T07:09:39.1244227Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T07:09:39.1244897Z nf_nat 32768 3 nf_nat_masquerade_ipv4,nf_nat_ipv4,xt_nat 2024-06-26T07:09:39.1246223Z nf_conntrack 155648 8 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,xt_nat,nf_conntrack_netlink 2024-06-26T07:09:39.1247184Z xt_addrtype 16384 2 2024-06-26T07:09:39.1247648Z iptable_filter 16384 1 2024-06-26T07:09:39.1248108Z br_netfilter 24576 0 2024-06-26T07:09:39.1248597Z bridge 172032 1 br_netfilter 2024-06-26T07:09:39.1249169Z stp 16384 1 bridge 2024-06-26T07:09:39.1249711Z llc 16384 2 bridge,stp 2024-06-26T07:09:39.1250211Z overlay 86016 0 2024-06-26T07:09:39.1259132Z sunrpc 393216 1 2024-06-26T07:09:39.1259928Z dm_mirror 28672 0 2024-06-26T07:09:39.1260490Z dm_region_hash 20480 1 dm_mirror 2024-06-26T07:09:39.1261266Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T07:09:39.1261983Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T07:09:39.1262533Z dax 69632 1 dm_mod 2024-06-26T07:09:39.1263031Z crc32_pclmul 16384 0 2024-06-26T07:09:39.1263498Z ghash_clmulni_intel 16384 0 2024-06-26T07:09:39.1264078Z pcbc 16384 0 2024-06-26T07:09:39.1264548Z aesni_intel 188416 0 2024-06-26T07:09:39.1265038Z aes_x86_64 20480 1 aesni_intel 2024-06-26T07:09:39.1265612Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:09:39.1266161Z glue_helper 16384 1 aesni_intel 2024-06-26T07:09:39.1267121Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T07:09:39.1267796Z mousedev 24576 0 2024-06-26T07:09:39.1268336Z psmouse 32768 0 2024-06-26T07:09:39.1268860Z evdev 20480 3 2024-06-26T07:09:39.1269377Z button 16384 0 2024-06-26T07:09:39.1269826Z ena 139264 0 2024-06-26T07:09:39.1270348Z ptp 20480 1 ena 2024-06-26T07:09:39.1270835Z pps_core 20480 1 ptp 2024-06-26T07:09:39.1271323Z crc32c_intel 24576 0 2024-06-26T07:09:39.1271819Z autofs4 49152 2 2024-06-26T07:09:39.1272291Z + modinfo nvidia 2024-06-26T07:09:39.1273142Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T07:09:39.1273942Z alias: char-major-195-* 2024-06-26T07:09:39.1274416Z version: 550.54.15 2024-06-26T07:09:39.1274845Z supported: external 2024-06-26T07:09:39.1275269Z license: NVIDIA 2024-06-26T07:09:39.1275732Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:09:39.1276298Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:09:39.1276849Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T07:09:39.1277409Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:09:39.1277998Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:09:39.1278782Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:09:39.1279375Z depends: i2c-core,drm 2024-06-26T07:09:39.1279826Z retpoline: Y 2024-06-26T07:09:39.1280221Z name: nvidia 2024-06-26T07:09:39.1280853Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T07:09:39.1281612Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:09:39.1282347Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:09:39.1283029Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:09:39.1283595Z parm: NVreg_RmLogonRC:int 2024-06-26T07:09:39.1284120Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:09:39.1284665Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:09:39.1285209Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:09:39.1285732Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:09:39.1286353Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:09:39.1287076Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:09:39.1287800Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:09:39.1310801Z parm: NVreg_EnableMSI:int 2024-06-26T07:09:39.1311327Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:09:39.1311889Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:09:39.1312503Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:09:39.1313173Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:09:39.1313821Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:09:39.1314515Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:09:39.1315242Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:09:39.1315941Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:09:39.1316893Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:09:39.1317520Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:09:39.1318142Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:09:39.1318769Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:09:39.1319343Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:09:39.1319890Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:09:39.1320490Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:09:39.1321042Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:09:39.1321585Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:09:39.1322204Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:09:39.1322803Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:09:39.1323366Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:09:39.1323973Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:09:39.1324557Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:09:39.1325109Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:09:39.1325723Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:09:39.1326279Z parm: NVreg_RmMsg:charp 2024-06-26T07:09:39.1326761Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:09:39.1327342Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:09:39.1327899Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:09:39.1328427Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:09:39.1328983Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:09:39.1329580Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:09:39.1330152Z parm: rm_firmware_active:charp 2024-06-26T07:09:39.1330716Z + HAS_NVIDIA_DRIVER=0 2024-06-26T07:09:39.1331280Z ++ command -v nvidia-smi 2024-06-26T07:09:39.1331797Z + '[' -x /usr/bin/nvidia-smi ']' 2024-06-26T07:09:39.1332281Z + set +e 2024-06-26T07:09:39.1332946Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-06-26T07:09:41.1100422Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-06-26T07:09:41.1101247Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:09:41.1102353Z + '[' 0 -ne 0 ']' 2024-06-26T07:09:41.1103026Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-06-26T07:09:41.1103653Z + HAS_NVIDIA_DRIVER=1 2024-06-26T07:09:41.1104567Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-06-26T07:09:41.1105367Z + set -e 2024-06-26T07:09:41.1105755Z + '[' 1 -eq 0 ']' 2024-06-26T07:09:41.1106431Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-06-26T07:09:41.1107517Z + post_install_nvidia_driver_common 2024-06-26T07:09:41.1108016Z + sudo modprobe nvidia 2024-06-26T07:09:41.1202122Z + echo 'After installing NVIDIA driver' 2024-06-26T07:09:41.1202817Z + lspci 2024-06-26T07:09:41.1203186Z After installing NVIDIA driver 2024-06-26T07:09:41.1279371Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-06-26T07:09:41.1280461Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T07:09:41.1281500Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-06-26T07:09:41.1282384Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-06-26T07:09:41.1283227Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 2024-06-26T07:09:41.1284074Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T07:09:41.1284889Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1) 2024-06-26T07:09:41.1285758Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-06-26T07:09:41.1286435Z + lsmod 2024-06-26T07:09:41.1292211Z Module Size Used by 2024-06-26T07:09:41.1292899Z xt_nat 16384 0 2024-06-26T07:09:41.1293497Z ib_core 266240 0 2024-06-26T07:09:41.1294436Z nvidia_modeset 1339392 0 2024-06-26T07:09:41.1295033Z veth 16384 0 2024-06-26T07:09:41.1295488Z nvidia_uvm 4599808 0 2024-06-26T07:09:41.1296043Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:09:41.1296619Z drm 421888 1 nvidia 2024-06-26T07:09:41.1297136Z i2c_core 77824 2 nvidia,drm 2024-06-26T07:09:41.1297706Z backlight 16384 1 nvidia_modeset 2024-06-26T07:09:41.1298229Z xt_conntrack 16384 1 2024-06-26T07:09:41.1298698Z ipt_MASQUERADE 16384 1 2024-06-26T07:09:41.1299272Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T07:09:41.1300020Z nf_conntrack_netlink 49152 0 2024-06-26T07:09:41.1300768Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T07:09:41.1301484Z xfrm_user 45056 1 2024-06-26T07:09:41.1302082Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:09:41.1302587Z iptable_nat 16384 1 2024-06-26T07:09:41.1303061Z nf_conntrack_ipv4 16384 3 2024-06-26T07:09:41.1303573Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T07:09:41.1304232Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T07:09:41.1304885Z nf_nat 32768 3 nf_nat_masquerade_ipv4,nf_nat_ipv4,xt_nat 2024-06-26T07:09:41.1305981Z nf_conntrack 155648 8 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,xt_nat,nf_conntrack_netlink 2024-06-26T07:09:41.1307172Z xt_addrtype 16384 2 2024-06-26T07:09:41.1307727Z iptable_filter 16384 1 2024-06-26T07:09:41.1308197Z br_netfilter 24576 0 2024-06-26T07:09:41.1308696Z bridge 172032 1 br_netfilter 2024-06-26T07:09:41.1309224Z stp 16384 1 bridge 2024-06-26T07:09:41.1309734Z llc 16384 2 bridge,stp 2024-06-26T07:09:41.1310235Z overlay 86016 0 2024-06-26T07:09:41.1310693Z sunrpc 393216 1 2024-06-26T07:09:41.1311160Z dm_mirror 28672 0 2024-06-26T07:09:41.1311637Z dm_region_hash 20480 1 dm_mirror 2024-06-26T07:09:41.1312213Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T07:09:41.1312958Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T07:09:41.1313500Z dax 69632 1 dm_mod 2024-06-26T07:09:41.1313986Z crc32_pclmul 16384 0 2024-06-26T07:09:41.1314455Z ghash_clmulni_intel 16384 0 2024-06-26T07:09:41.1314925Z pcbc 16384 0 2024-06-26T07:09:41.1315387Z aesni_intel 188416 0 2024-06-26T07:09:41.1315863Z aes_x86_64 20480 1 aesni_intel 2024-06-26T07:09:41.1316405Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:09:41.1316953Z glue_helper 16384 1 aesni_intel 2024-06-26T07:09:41.1317604Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T07:09:41.1318237Z mousedev 24576 0 2024-06-26T07:09:41.1318705Z psmouse 32768 0 2024-06-26T07:09:41.1319173Z evdev 20480 3 2024-06-26T07:09:41.1319628Z button 16384 0 2024-06-26T07:09:41.1320084Z ena 139264 0 2024-06-26T07:09:41.1320546Z ptp 20480 1 ena 2024-06-26T07:09:41.1321033Z pps_core 20480 1 ptp 2024-06-26T07:09:41.1321513Z crc32c_intel 24576 0 2024-06-26T07:09:41.1321972Z autofs4 49152 2 2024-06-26T07:09:41.1322420Z + modinfo nvidia 2024-06-26T07:09:41.1323168Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T07:09:41.1323954Z alias: char-major-195-* 2024-06-26T07:09:41.1324431Z version: 550.54.15 2024-06-26T07:09:41.1324862Z supported: external 2024-06-26T07:09:41.1325292Z license: NVIDIA 2024-06-26T07:09:41.1325754Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:09:41.1326331Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:09:41.1326877Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T07:09:41.1327516Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:09:41.1328105Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:09:41.1328697Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:09:41.1329271Z depends: i2c-core,drm 2024-06-26T07:09:41.1329706Z retpoline: Y 2024-06-26T07:09:41.1330097Z name: nvidia 2024-06-26T07:09:41.1330740Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T07:09:41.1331494Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:09:41.1332231Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:09:41.1332916Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:09:41.1333451Z parm: NVreg_RmLogonRC:int 2024-06-26T07:09:41.1333978Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:09:41.1334521Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:09:41.1335049Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:09:41.1335588Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:09:41.1336208Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:09:41.1336860Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:09:41.1337430Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:09:41.1337950Z parm: NVreg_EnableMSI:int 2024-06-26T07:09:41.1338449Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:09:41.1339002Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:09:41.1339622Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:09:41.1340294Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:09:41.1340943Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:09:41.1341639Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:09:41.1342327Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:09:41.1343030Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:09:41.1343725Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:09:41.1344412Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:09:41.1345127Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:09:41.1345771Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:09:41.1346370Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:09:41.1347163Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:09:41.1347733Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:09:41.1348292Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:09:41.1348829Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:09:41.1349421Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:09:41.1350029Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:09:41.1350599Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:09:41.1351178Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:09:41.1351764Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:09:41.1352340Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:09:41.1352935Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:09:41.1353508Z parm: NVreg_RmMsg:charp 2024-06-26T07:09:41.1354010Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:09:41.1369838Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:09:41.1370417Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:09:41.1370957Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:09:41.1371536Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:09:41.1372111Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:09:41.1372649Z parm: rm_firmware_active:charp 2024-06-26T07:09:41.1373137Z + set +e 2024-06-26T07:09:41.1373535Z + nvidia-smi 2024-06-26T07:09:42.7050426Z Wed Jun 26 07:09:42 2024 2024-06-26T07:09:42.7051452Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:09:42.7052740Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-06-26T07:09:42.7053643Z |-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:09:42.7054535Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-06-26T07:09:42.7055526Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-06-26T07:09:42.7056318Z | | | MIG M. | 2024-06-26T07:09:42.7056966Z |=========================================+========================+======================| 2024-06-26T07:09:42.7176585Z | 0 NVIDIA A10G Off | 00000000:00:1E.0 Off | 0 | 2024-06-26T07:09:42.7177477Z | 0% 32C P0 57W / 300W | 0MiB / 23028MiB | 0% Default | 2024-06-26T07:09:42.7178882Z | | | N/A | 2024-06-26T07:09:42.7180525Z +-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:09:42.7182012Z 2024-06-26T07:09:42.7183521Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:09:42.7185141Z | Processes: | 2024-06-26T07:09:42.7187342Z | GPU GI CI PID Type Process name GPU Memory | 2024-06-26T07:09:42.7188275Z | ID ID Usage | 2024-06-26T07:09:42.7188939Z |=========================================================================================| 2024-06-26T07:09:42.7189686Z | No running processes found | 2024-06-26T07:09:42.7190556Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:09:43.3304880Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-06-26T07:09:44.8959950Z NVIDIA A10G 2024-06-26T07:09:45.3339499Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:09:45.3340134Z + '[' 0 -eq 0 ']' 2024-06-26T07:09:45.3340552Z + echo 'INFO: Ignoring allowed status 0' 2024-06-26T07:09:45.3340993Z + set -e 2024-06-26T07:09:45.3341293Z INFO: Ignoring allowed status 0 2024-06-26T07:09:45.3343199Z == Installing nvidia container toolkit for amzn2 == 2024-06-26T07:09:45.3347698Z + sudo yum install -y yum-utils 2024-06-26T07:09:45.6413918Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:09:47.2564329Z Package yum-utils-1.1.31-46.amzn2.0.1.noarch already installed and latest version 2024-06-26T07:09:47.2565969Z Nothing to do 2024-06-26T07:09:47.4058322Z + [[ amzn2 == \a\m\z\n\2\0\2\3 ]] 2024-06-26T07:09:47.4059348Z + YUM_REPO_URL=https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:09:47.4060500Z + sudo yum-config-manager --add-repo https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:09:47.7374740Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:09:47.7655808Z adding repo from: https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:09:47.7657120Z grabbing file https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T07:09:47.7658181Z repo saved to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T07:09:47.7772693Z + sudo yum install -y nvidia-docker2 2024-06-26T07:09:48.0723998Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:09:49.5327317Z Package nvidia-docker2-2.13.0-1.noarch already installed and latest version 2024-06-26T07:09:49.5328090Z Nothing to do 2024-06-26T07:09:49.6811394Z + sudo systemctl restart docker 2024-06-26T07:10:34.7864409Z Command completed after 1 attempt(s). 2024-06-26T07:10:34.7921774Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:10:34.7922544Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:10:34.7923236Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-06-26T07:10:34.7923902Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:10:34.7931750Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:10:34.7932263Z env: 2024-06-26T07:10:34.7932541Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:34.7933002Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:34.7933484Z ##[endgroup] 2024-06-26T07:10:35.0135025Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T07:10:35.0295516Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.7/site-packages (5.9.1) 2024-06-26T07:10:35.0397321Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.7/site-packages (11.525.84) 2024-06-26T07:10:35.1327925Z Prepare all required actions 2024-06-26T07:10:35.1328421Z Getting action download info 2024-06-26T07:10:35.2755565Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-06-26T07:10:35.4823515Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-06-26T07:10:35.6312336Z ##[group]Run ./.github/actions/download-build-artifacts 2024-06-26T07:10:35.6312815Z with: 2024-06-26T07:10:35.6313154Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:10:35.6313619Z s3-bucket: gha-artifacts 2024-06-26T07:10:35.6313956Z env: 2024-06-26T07:10:35.6314227Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:35.6314677Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:35.6315169Z ##[endgroup] 2024-06-26T07:10:35.6346801Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:10:35.6347248Z with: 2024-06-26T07:10:35.6347623Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:10:35.6348084Z s3-bucket: gha-artifacts 2024-06-26T07:10:35.6348437Z region: us-east-1 2024-06-26T07:10:35.6348743Z env: 2024-06-26T07:10:35.6349022Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:35.6349478Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:35.6349979Z ##[endgroup] 2024-06-26T07:10:36.0498676Z (node:31765) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:10:36.0499411Z 2024-06-26T07:10:36.0499670Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:10:36.0500410Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:10:36.0501283Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:10:36.1489023Z Found 1 objects with prefix pytorch/pytorch/9673646808/linux-focal-cuda12.1-py3.10-gcc9-sm86/ 2024-06-26T07:10:36.1490166Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:10:43.6354283Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:10:43.6360617Z Artifact download has finished successfully 2024-06-26T07:10:43.6526305Z ##[group]Run unzip -o artifacts.zip 2024-06-26T07:10:43.6526826Z unzip -o artifacts.zip 2024-06-26T07:10:43.6535075Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:10:43.6535661Z env: 2024-06-26T07:10:43.6536009Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:43.6536550Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:43.6537120Z ##[endgroup] 2024-06-26T07:10:43.6565714Z Archive: artifacts.zip 2024-06-26T07:10:43.6567471Z creating: dist/ 2024-06-26T07:10:45.6393811Z inflating: dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:10:45.6394837Z creating: build/custom_test_artifacts/ 2024-06-26T07:10:45.6395522Z creating: build/custom_test_artifacts/custom-op-build/ 2024-06-26T07:10:45.6396339Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-06-26T07:10:45.6397272Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:10:45.6401335Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:10:45.6402377Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-06-26T07:10:45.6403413Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:10:45.6404502Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:10:45.6405571Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:10:45.6406770Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:10:45.6408125Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:10:45.6409277Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:10:45.6410379Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:10:45.6411629Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:10:45.6412899Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:10:45.6414145Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:10:45.6415373Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:10:45.6416643Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:10:45.6417930Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:10:45.6419067Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:10:45.6420171Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:10:45.6456004Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:10:45.6493831Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:10:45.6495394Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:10:45.6538096Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:10:45.6539687Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:10:45.6541233Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:10:45.6542817Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:10:45.6544420Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:10:45.6545903Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:10:45.6547689Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:10:45.6549184Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:10:45.6550759Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:10:45.6552144Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:10:45.6553484Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:10:45.6554800Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:10:45.6556127Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:10:45.6557425Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:10:45.6558738Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:10:45.6607227Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:10:45.6665067Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:10:45.6666361Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:10:45.6667566Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:10:45.6668529Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:10:45.6669533Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-06-26T07:10:45.6670544Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-06-26T07:10:45.6671651Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-06-26T07:10:45.6672918Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-06-26T07:10:45.6674122Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-06-26T07:10:45.6675250Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-06-26T07:10:45.6676419Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-06-26T07:10:45.6677575Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-06-26T07:10:45.6678749Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-06-26T07:10:45.6679913Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-06-26T07:10:45.6681069Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-06-26T07:10:45.6691836Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-06-26T07:10:45.6819134Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-06-26T07:10:45.6820233Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-06-26T07:10:45.6821424Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-06-26T07:10:45.6822714Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-06-26T07:10:45.6824013Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-06-26T07:10:45.6825205Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-06-26T07:10:45.6826426Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-06-26T07:10:45.6827990Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-06-26T07:10:45.6829251Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-06-26T07:10:45.6830510Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-06-26T07:10:45.6831731Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-06-26T07:10:45.6844326Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-06-26T07:10:45.6923411Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-06-26T07:10:45.6924688Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:10:45.6925842Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:10:45.6927018Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-06-26T07:10:45.6928020Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-06-26T07:10:45.6929003Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-06-26T07:10:45.6929994Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-06-26T07:10:45.6930923Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-06-26T07:10:45.6931793Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-06-26T07:10:45.6932690Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-06-26T07:10:45.7038575Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-06-26T07:10:45.7099174Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-06-26T07:10:45.7100020Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-06-26T07:10:45.7100817Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-06-26T07:10:45.7101730Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:10:45.7106573Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:10:45.7107596Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-06-26T07:10:45.7108623Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:10:45.7109696Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:10:45.7110738Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:10:45.7111908Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:10:45.7113091Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:10:45.7114161Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:10:45.7115226Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:10:45.7116444Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:10:45.7117675Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:10:45.7118894Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:10:45.7120117Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:10:45.7121354Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:10:45.7122725Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:10:45.7123836Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:10:45.7124906Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:10:45.7158541Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:10:45.7196376Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:10:45.7197891Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:10:45.7240582Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:10:45.7242069Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:10:45.7243680Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:10:45.7245275Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:10:45.7246932Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:10:45.7248383Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:10:45.7249903Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:10:45.7271545Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:10:45.7273056Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:10:45.7274441Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:10:45.7275814Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:10:45.7277122Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:10:45.7278445Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:10:45.7279793Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:10:45.7281110Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:10:45.7310319Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:10:45.7368115Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:10:45.7369613Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:10:45.7371634Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:10:45.7373359Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:10:45.7375149Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-06-26T07:10:45.7377056Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-06-26T07:10:45.7379157Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-06-26T07:10:45.7380685Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-06-26T07:10:45.7382100Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-06-26T07:10:45.7383249Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-06-26T07:10:45.7384526Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-06-26T07:10:45.7385730Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-06-26T07:10:45.7387091Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-06-26T07:10:45.7388293Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-06-26T07:10:45.7389474Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-06-26T07:10:45.7395444Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-06-26T07:10:45.7458233Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-06-26T07:10:45.7460025Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:10:45.7461188Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:10:45.7462241Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-06-26T07:10:45.7463246Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-06-26T07:10:45.7464282Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-06-26T07:10:45.7465270Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-06-26T07:10:45.7466202Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-06-26T07:10:45.7467210Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-06-26T07:10:45.7468106Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-06-26T07:10:45.7511137Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-06-26T07:10:45.7512214Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-06-26T07:10:45.7513158Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-06-26T07:10:45.7514137Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:10:45.7518673Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:10:45.7519810Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-06-26T07:10:45.7520885Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:10:45.7522046Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:10:45.7523262Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:10:45.7524504Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:10:45.7525755Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:10:45.7526890Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:10:45.7528026Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:10:45.7529375Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:10:45.7530756Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:10:45.7532189Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:10:45.7533473Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:10:45.7534789Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:10:45.7536097Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:10:45.7537298Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:10:45.7538442Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:10:45.7571022Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:10:45.7607663Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:10:45.7609457Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:10:45.7651854Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:10:45.7653447Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:10:45.7655035Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:10:45.7656667Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:10:45.7658231Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:10:45.7692153Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:10:45.7695226Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:10:45.7698257Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:10:45.7700396Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:10:45.7701812Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:10:45.7703212Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:10:45.7704637Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:10:45.7706033Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:10:45.7707504Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:10:45.7708873Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:10:45.7758559Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:10:45.7816942Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:10:45.7818580Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:10:45.7820041Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:10:45.7821366Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:10:45.7822493Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-06-26T07:10:45.7823754Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-06-26T07:10:45.7825140Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-06-26T07:10:45.7826684Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-06-26T07:10:45.7828001Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-06-26T07:10:45.7829244Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-06-26T07:10:45.7830608Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-06-26T07:10:45.7832025Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-06-26T07:10:45.7833331Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-06-26T07:10:45.7834611Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-06-26T07:10:45.7835893Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-06-26T07:10:45.7837236Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-06-26T07:10:45.7951485Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-06-26T07:10:45.7952783Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-06-26T07:10:45.7954079Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-06-26T07:10:45.7955479Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-06-26T07:10:45.7956821Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-06-26T07:10:45.7958113Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-06-26T07:10:45.7959486Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-06-26T07:10:45.7960808Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-06-26T07:10:45.7962133Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-06-26T07:10:45.7963462Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-06-26T07:10:45.7964770Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-06-26T07:10:45.7977245Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-06-26T07:10:45.8030457Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-06-26T07:10:45.8031871Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:10:45.8033103Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:10:45.8034216Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-06-26T07:10:45.8035272Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-06-26T07:10:45.8036468Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-06-26T07:10:45.8037524Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-06-26T07:10:45.8038534Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-06-26T07:10:45.8039466Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-06-26T07:10:45.8040426Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-06-26T07:10:45.8138249Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-06-26T07:10:45.8179927Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-06-26T07:10:45.8180664Z creating: build/lib/ 2024-06-26T07:10:45.8265237Z inflating: build/lib/libprotobuf-lite.a 2024-06-26T07:10:45.8704703Z inflating: build/lib/libprotobuf.a 2024-06-26T07:10:45.8714451Z inflating: build/lib/libpthreadpool.a 2024-06-26T07:10:45.8722479Z inflating: build/lib/libcpuinfo.a 2024-06-26T07:10:45.8730338Z inflating: build/lib/libcpuinfo_internals.a 2024-06-26T07:10:45.8731072Z inflating: build/lib/libclog.a 2024-06-26T07:10:45.8748901Z inflating: build/lib/libnnpack.a 2024-06-26T07:10:45.8751070Z inflating: build/lib/libnnpack_reference_layers.a 2024-06-26T07:10:45.8813693Z inflating: build/lib/libgtest.a 2024-06-26T07:10:45.8886053Z inflating: build/lib/libbenchmark.a 2024-06-26T07:10:45.8960558Z inflating: build/lib/libasmjit.a 2024-06-26T07:10:45.8969167Z inflating: build/lib/libittnotify.a 2024-06-26T07:10:45.9002009Z inflating: build/lib/libtensorpipe_uv.a 2024-06-26T07:10:45.9132343Z inflating: build/lib/libgloo.a 2024-06-26T07:10:45.9153736Z inflating: build/lib/libfmt.a 2024-06-26T07:10:45.9242744Z inflating: build/lib/libc10.so 2024-06-26T07:10:45.9244220Z inflating: build/lib/libcaffe2_nvrtc.so 2024-06-26T07:10:45.9244797Z inflating: build/lib/libfoxi_loader.a 2024-06-26T07:10:45.9245763Z inflating: build/lib/libtorch_global_deps.so 2024-06-26T07:10:45.9264652Z inflating: build/lib/libpytorch_qnnpack.a 2024-06-26T07:10:45.9281887Z inflating: build/lib/libgmock.a 2024-06-26T07:10:45.9282423Z inflating: build/lib/libgtest_main.a 2024-06-26T07:10:45.9282993Z inflating: build/lib/libbenchmark_main.a 2024-06-26T07:10:45.9769861Z inflating: build/lib/libprotoc.a 2024-06-26T07:10:46.0138922Z inflating: build/lib/libgloo_cuda.a 2024-06-26T07:10:46.9895829Z inflating: build/lib/libdnnl.a 2024-06-26T07:10:47.0445227Z inflating: build/lib/libtensorpipe.a 2024-06-26T07:10:47.0498324Z inflating: build/lib/libc10_cuda.so 2024-06-26T07:10:47.0499413Z inflating: build/lib/libgmock_main.a 2024-06-26T07:10:47.1702193Z inflating: build/lib/libfbgemm.a 2024-06-26T07:10:47.1946185Z inflating: build/lib/libtensorpipe_cuda.a 2024-06-26T07:10:47.2443815Z inflating: build/lib/libkineto.a 2024-06-26T07:10:47.2621075Z inflating: build/lib/libXNNPACK.a 2024-06-26T07:10:47.2661907Z inflating: build/lib/libonnx_proto.a 2024-06-26T07:10:47.3325428Z inflating: build/lib/libonnx.a 2024-06-26T07:10:49.6652130Z inflating: build/lib/libtorch_cpu.so 2024-06-26T07:10:49.6656249Z inflating: build/lib/libunbox_lib.a 2024-06-26T07:10:49.6660244Z inflating: build/lib/libshm.so 2024-06-26T07:10:51.6609837Z inflating: build/lib/libtorch_cuda.so 2024-06-26T07:10:51.6610603Z inflating: build/lib/libtorch.so 2024-06-26T07:10:51.6612766Z inflating: build/lib/libc10d_cuda_test.so 2024-06-26T07:10:52.4617082Z inflating: build/lib/libtorch_cuda_linalg.so 2024-06-26T07:10:52.6513021Z inflating: build/lib/libtorch_python.so 2024-06-26T07:10:52.6578968Z inflating: build/lib/libtorchbind_test.so 2024-06-26T07:10:52.6599067Z inflating: build/lib/libjitbackend_test.so 2024-06-26T07:10:52.6623358Z inflating: build/lib/libbackend_with_compiler.so 2024-06-26T07:10:52.6656830Z inflating: build/lib/libnnapi_backend.so 2024-06-26T07:10:52.6657392Z creating: build/bin/ 2024-06-26T07:10:52.6705839Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-06-26T07:10:52.6755857Z inflating: build/bin/c10_DeviceGuard_test 2024-06-26T07:10:52.6804848Z inflating: build/bin/c10_Device_test 2024-06-26T07:10:52.6861951Z inflating: build/bin/c10_DispatchKeySet_test 2024-06-26T07:10:52.6913992Z inflating: build/bin/c10_Scalar_test 2024-06-26T07:10:52.6961398Z inflating: build/bin/c10_StreamGuard_test 2024-06-26T07:10:52.7011445Z inflating: build/bin/c10_SymInt_test 2024-06-26T07:10:52.7063712Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-06-26T07:10:52.7117803Z inflating: build/bin/c10_InlineStreamGuard_test 2024-06-26T07:10:52.7172523Z inflating: build/bin/c10_SizesAndStrides_test 2024-06-26T07:10:52.7241217Z inflating: build/bin/c10_cow_test 2024-06-26T07:10:52.7292704Z inflating: build/bin/c10_Bitset_test 2024-06-26T07:10:52.7340700Z inflating: build/bin/c10_ConstexprCrc_test 2024-06-26T07:10:52.7389284Z inflating: build/bin/c10_DeadlockDetection_test 2024-06-26T07:10:52.7437771Z inflating: build/bin/c10_Half_test 2024-06-26T07:10:52.7492709Z inflating: build/bin/c10_LeftRight_test 2024-06-26T07:10:52.7546723Z inflating: build/bin/c10_Metaprogramming_test 2024-06-26T07:10:52.7595183Z inflating: build/bin/c10_Synchronized_test 2024-06-26T07:10:52.7649035Z inflating: build/bin/c10_ThreadLocal_test 2024-06-26T07:10:52.7699535Z inflating: build/bin/c10_TypeIndex_test 2024-06-26T07:10:52.7748555Z inflating: build/bin/c10_TypeList_test 2024-06-26T07:10:52.7796173Z inflating: build/bin/c10_TypeTraits_test 2024-06-26T07:10:52.7846832Z inflating: build/bin/c10_accumulate_test 2024-06-26T07:10:52.7900033Z inflating: build/bin/c10_bfloat16_test 2024-06-26T07:10:52.7949424Z inflating: build/bin/c10_bit_cast_test 2024-06-26T07:10:52.8003891Z inflating: build/bin/c10_complex_math_test 2024-06-26T07:10:52.8058136Z inflating: build/bin/c10_complex_test 2024-06-26T07:10:52.8108972Z inflating: build/bin/c10_exception_test 2024-06-26T07:10:52.8156984Z inflating: build/bin/c10_flags_test 2024-06-26T07:10:52.8206132Z inflating: build/bin/c10_generic_math_test 2024-06-26T07:10:52.8364805Z inflating: build/bin/c10_intrusive_ptr_test 2024-06-26T07:10:52.8413838Z inflating: build/bin/c10_irange_test 2024-06-26T07:10:52.8465643Z inflating: build/bin/c10_lazy_test 2024-06-26T07:10:52.8521064Z inflating: build/bin/c10_logging_test 2024-06-26T07:10:52.8593364Z inflating: build/bin/c10_optional_test 2024-06-26T07:10:52.8653783Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-06-26T07:10:52.8706294Z inflating: build/bin/c10_registry_test 2024-06-26T07:10:52.8851450Z inflating: build/bin/c10_small_vector_test 2024-06-26T07:10:52.8901417Z inflating: build/bin/c10_ssize_test 2024-06-26T07:10:52.8952178Z inflating: build/bin/c10_string_util_test 2024-06-26T07:10:52.9008150Z inflating: build/bin/c10_string_view_test 2024-06-26T07:10:52.9057371Z inflating: build/bin/c10_tempfile_test 2024-06-26T07:10:52.9104258Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-06-26T07:10:52.9158220Z inflating: build/bin/c10_typeid_test 2024-06-26T07:10:52.9590312Z inflating: build/bin/protoc-3.13.0.0 2024-06-26T07:10:53.0022218Z inflating: build/bin/protoc 2024-06-26T07:10:53.0074132Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-06-26T07:10:53.0130126Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-06-26T07:10:53.0192669Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-06-26T07:10:53.0254590Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-06-26T07:10:53.0306851Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-06-26T07:10:53.0358324Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-06-26T07:10:53.0406154Z inflating: build/bin/c10_cuda_CUDATest 2024-06-26T07:10:53.0458080Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-06-26T07:10:53.0778084Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-06-26T07:10:53.1111329Z inflating: build/bin/vec_test_all_types_AVX512 2024-06-26T07:10:53.1458679Z inflating: build/bin/vec_test_all_types_AVX2 2024-06-26T07:10:53.1510732Z inflating: build/bin/FileStoreTest 2024-06-26T07:10:53.1565605Z inflating: build/bin/TCPStoreTest 2024-06-26T07:10:53.1617981Z inflating: build/bin/HashStoreTest 2024-06-26T07:10:53.1631226Z inflating: build/bin/ProcessGroupMPITest 2024-06-26T07:10:53.1683942Z inflating: build/bin/test_edge_op_registration 2024-06-26T07:10:53.1687909Z inflating: build/bin/torch_shm_manager 2024-06-26T07:10:53.1690558Z inflating: build/bin/example_allreduce 2024-06-26T07:10:53.1743926Z inflating: build/bin/test_dist_autograd 2024-06-26T07:10:53.1812016Z inflating: build/bin/test_cpp_rpc 2024-06-26T07:10:53.1814201Z inflating: build/bin/parallel_benchmark 2024-06-26T07:10:53.1879751Z inflating: build/bin/test_mobile_nnc 2024-06-26T07:10:53.1888338Z inflating: build/bin/aot_model_compiler_test 2024-06-26T07:10:53.2221621Z inflating: build/bin/test_lazy 2024-06-26T07:10:53.3367792Z inflating: build/bin/test_api 2024-06-26T07:10:53.3438565Z inflating: build/bin/Dict_test 2024-06-26T07:10:53.3489289Z inflating: build/bin/Dimname_test 2024-06-26T07:10:53.3552369Z inflating: build/bin/MaybeOwned_test 2024-06-26T07:10:53.3607501Z inflating: build/bin/NamedTensor_test 2024-06-26T07:10:53.3664942Z inflating: build/bin/apply_utils_test 2024-06-26T07:10:53.3722098Z inflating: build/bin/atest 2024-06-26T07:10:53.3783168Z inflating: build/bin/basic 2024-06-26T07:10:53.3836618Z inflating: build/bin/broadcast_test 2024-06-26T07:10:53.3885787Z inflating: build/bin/cpu_allocator_test 2024-06-26T07:10:53.3942550Z inflating: build/bin/cpu_generator_test 2024-06-26T07:10:53.3993797Z inflating: build/bin/cpu_profiling_allocator_test 2024-06-26T07:10:53.4084813Z inflating: build/bin/cpu_rng_test 2024-06-26T07:10:53.4132860Z inflating: build/bin/dispatch_key_set_test 2024-06-26T07:10:53.4181924Z inflating: build/bin/dlconvertor_test 2024-06-26T07:10:53.4238011Z inflating: build/bin/extension_backend_test 2024-06-26T07:10:53.4291599Z inflating: build/bin/half_test 2024-06-26T07:10:53.4384593Z inflating: build/bin/ivalue_test 2024-06-26T07:10:53.4433682Z inflating: build/bin/lazy_tensor_test 2024-06-26T07:10:53.4486640Z inflating: build/bin/math_kernel_test 2024-06-26T07:10:53.4539414Z inflating: build/bin/memory_format_test 2024-06-26T07:10:53.4590621Z inflating: build/bin/memory_overlapping_test 2024-06-26T07:10:53.4642010Z inflating: build/bin/mobile_memory_cleanup 2024-06-26T07:10:53.4696371Z inflating: build/bin/native_test 2024-06-26T07:10:53.4745403Z inflating: build/bin/operator_name_test 2024-06-26T07:10:53.4795469Z inflating: build/bin/operators_test 2024-06-26T07:10:53.4846159Z inflating: build/bin/packedtensoraccessor_test 2024-06-26T07:10:53.4910756Z inflating: build/bin/pow_test 2024-06-26T07:10:53.4966876Z inflating: build/bin/quantized_test 2024-06-26T07:10:53.5014845Z inflating: build/bin/reduce_ops_test 2024-06-26T07:10:53.5064621Z inflating: build/bin/reportMemoryUsage_test 2024-06-26T07:10:53.5119439Z inflating: build/bin/scalar_tensor_test 2024-06-26T07:10:53.5176140Z inflating: build/bin/scalar_test 2024-06-26T07:10:53.5226438Z inflating: build/bin/StorageUtils_test 2024-06-26T07:10:53.5277767Z inflating: build/bin/stride_properties_test 2024-06-26T07:10:53.5353519Z inflating: build/bin/tensor_iterator_test 2024-06-26T07:10:53.5406710Z inflating: build/bin/test_parallel 2024-06-26T07:10:53.5408835Z inflating: build/bin/thread_init_test 2024-06-26T07:10:53.5462974Z inflating: build/bin/type_ptr_test 2024-06-26T07:10:53.5521267Z inflating: build/bin/type_test 2024-06-26T07:10:53.5571881Z inflating: build/bin/undefined_tensor_test 2024-06-26T07:10:53.5572957Z inflating: build/bin/verify_api_visibility 2024-06-26T07:10:53.5640746Z inflating: build/bin/legacy_vmap_test 2024-06-26T07:10:53.5691235Z inflating: build/bin/weakref_test 2024-06-26T07:10:53.5740890Z inflating: build/bin/wrapdim_test 2024-06-26T07:10:53.5791382Z inflating: build/bin/xla_tensor_test 2024-06-26T07:10:53.5849618Z inflating: build/bin/IListRef_test 2024-06-26T07:10:53.5952815Z inflating: build/bin/List_test 2024-06-26T07:10:53.6017290Z inflating: build/bin/KernelFunction_test 2024-06-26T07:10:53.6135115Z inflating: build/bin/kernel_function_legacy_test 2024-06-26T07:10:53.6228750Z inflating: build/bin/kernel_function_test 2024-06-26T07:10:53.6352411Z inflating: build/bin/kernel_lambda_legacy_test 2024-06-26T07:10:53.6453394Z inflating: build/bin/kernel_lambda_test 2024-06-26T07:10:53.6513057Z inflating: build/bin/kernel_stackbased_test 2024-06-26T07:10:53.6607575Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-06-26T07:10:53.6656593Z inflating: build/bin/CppSignature_test 2024-06-26T07:10:53.6710727Z inflating: build/bin/backend_fallback_test 2024-06-26T07:10:53.6757888Z inflating: build/bin/op_allowlist_test 2024-06-26T07:10:53.6819298Z inflating: build/bin/inline_container_test 2024-06-26T07:10:53.7110758Z inflating: build/bin/op_registration_test 2024-06-26T07:10:53.7161889Z inflating: build/bin/cuda_apply_test 2024-06-26T07:10:53.7211905Z inflating: build/bin/cuda_allocator_test 2024-06-26T07:10:53.7268853Z inflating: build/bin/cuda_atomic_ops_test 2024-06-26T07:10:53.7321475Z inflating: build/bin/cuda_caching_host_allocator_test 2024-06-26T07:10:53.7389269Z inflating: build/bin/cuda_complex_math_test 2024-06-26T07:10:53.7447041Z inflating: build/bin/cuda_complex_test 2024-06-26T07:10:53.7495130Z inflating: build/bin/cuda_device_test 2024-06-26T07:10:53.7550314Z inflating: build/bin/cuda_cub_test 2024-06-26T07:10:53.7599584Z inflating: build/bin/cuda_dlconvertor_test 2024-06-26T07:10:53.7662549Z inflating: build/bin/cuda_distributions_test 2024-06-26T07:10:53.7717159Z inflating: build/bin/cuda_generator_test 2024-06-26T07:10:53.7765687Z inflating: build/bin/cuda_half_test 2024-06-26T07:10:53.7814940Z inflating: build/bin/cuda_integer_divider_test 2024-06-26T07:10:53.7862188Z inflating: build/bin/cuda_optional_test 2024-06-26T07:10:53.7912340Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-06-26T07:10:53.7963601Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-06-26T07:10:53.8011968Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-06-26T07:10:53.8071028Z inflating: build/bin/cuda_stream_test 2024-06-26T07:10:53.8118606Z inflating: build/bin/cuda_cudnn_test 2024-06-26T07:10:53.8169031Z inflating: build/bin/cuda_vectorized_test 2024-06-26T07:10:53.8183338Z inflating: build/bin/tutorial_tensorexpr 2024-06-26T07:10:53.8247525Z inflating: build/bin/ProcessGroupGlooTest 2024-06-26T07:10:53.8302629Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-06-26T07:10:53.8364088Z inflating: build/bin/ProcessGroupNCCLTest 2024-06-26T07:10:53.8424156Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-06-26T07:10:53.9228674Z inflating: build/bin/test_tensorexpr 2024-06-26T07:10:53.9787275Z inflating: build/bin/test_jit 2024-06-26T07:10:53.9787811Z creating: .additional_ci_files/ 2024-06-26T07:10:53.9842152Z inflating: .additional_ci_files/test-times.json 2024-06-26T07:10:54.0052230Z inflating: .additional_ci_files/test-class-times.json 2024-06-26T07:10:54.0081378Z ##[group]Run rm artifacts.zip 2024-06-26T07:10:54.0081773Z rm artifacts.zip 2024-06-26T07:10:54.0091106Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:10:54.0091629Z env: 2024-06-26T07:10:54.0091934Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:54.0092390Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:54.0092890Z ##[endgroup] 2024-06-26T07:10:54.0801553Z ##[group]Run df -H 2024-06-26T07:10:54.0801872Z df -H 2024-06-26T07:10:54.0809480Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:10:54.0809996Z env: 2024-06-26T07:10:54.0810282Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:54.0810737Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:54.0811231Z ##[endgroup] 2024-06-26T07:10:54.0838112Z Filesystem Size Used Avail Use% Mounted on 2024-06-26T07:10:54.0838648Z devtmpfs 34G 0 34G 0% /dev 2024-06-26T07:10:54.0839126Z tmpfs 34G 0 34G 0% /dev/shm 2024-06-26T07:10:54.0839820Z tmpfs 34G 455k 34G 1% /run 2024-06-26T07:10:54.0840324Z tmpfs 34G 0 34G 0% /sys/fs/cgroup 2024-06-26T07:10:54.0840808Z /dev/nvme0n1p1 162G 44G 119G 27% / 2024-06-26T07:10:54.0879524Z Prepare all required actions 2024-06-26T07:10:54.0879968Z Getting action download info 2024-06-26T07:10:54.1943927Z ##[group]Run ./.github/actions/download-td-artifacts 2024-06-26T07:10:54.1944492Z with: 2024-06-26T07:10:54.1944748Z env: 2024-06-26T07:10:54.1945041Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:54.1945543Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:54.1946041Z ##[endgroup] 2024-06-26T07:10:54.1977555Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:10:54.1978030Z with: 2024-06-26T07:10:54.1978296Z name: td_results 2024-06-26T07:10:54.1978626Z s3-bucket: gha-artifacts 2024-06-26T07:10:54.1979001Z region: us-east-1 2024-06-26T07:10:54.1979306Z env: 2024-06-26T07:10:54.1979584Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:54.1980069Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:54.1980593Z ##[endgroup] 2024-06-26T07:10:54.6077545Z (node:31786) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:10:54.6078240Z 2024-06-26T07:10:54.6078509Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:10:54.6079239Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:10:54.6080067Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:10:54.6764152Z Found 0 objects with prefix pytorch/pytorch/9673646808/td_results/ 2024-06-26T07:10:54.6770922Z Artifact download has finished successfully 2024-06-26T07:10:54.6913963Z ##[group]Run mkdir -p .additional_ci_files 2024-06-26T07:10:54.6914525Z mkdir -p .additional_ci_files 2024-06-26T07:10:54.6915081Z mv td_results.json .additional_ci_files/td_results.json 2024-06-26T07:10:54.6922922Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:10:54.6923438Z env: 2024-06-26T07:10:54.6923705Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:54.6924167Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:54.6924644Z ##[endgroup] 2024-06-26T07:10:54.6959553Z mv: cannot stat 'td_results.json': No such file or directory 2024-06-26T07:10:54.6969994Z ##[error]Process completed with exit code 1. 2024-06-26T07:10:54.7002249Z ##[group]Run .github/scripts/parse_ref.py 2024-06-26T07:10:54.7002735Z .github/scripts/parse_ref.py 2024-06-26T07:10:54.7010218Z shell: /usr/bin/bash -e {0} 2024-06-26T07:10:54.7010580Z env: 2024-06-26T07:10:54.7010860Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:54.7011325Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:54.7011806Z ##[endgroup] 2024-06-26T07:10:54.7245175Z Prepare all required actions 2024-06-26T07:10:54.7282595Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-06-26T07:10:54.7283062Z with: 2024-06-26T07:10:54.7283618Z github-token: *** 2024-06-26T07:10:54.7284070Z env: 2024-06-26T07:10:54.7284346Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:54.7284815Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:54.7285308Z ##[endgroup] 2024-06-26T07:10:54.7300320Z ##[group]Run set -eux 2024-06-26T07:10:54.7300666Z set -eux 2024-06-26T07:10:54.7301267Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-06-26T07:10:54.7309164Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:10:54.7309671Z env: 2024-06-26T07:10:54.7309960Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:54.7310426Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:54.7311079Z GITHUB_TOKEN: *** 2024-06-26T07:10:54.7311393Z ##[endgroup] 2024-06-26T07:10:54.7330787Z + python3 .github/scripts/get_workflow_job_id.py 9673646808 i-083295e4fef5bcb20 2024-06-26T07:10:55.9711405Z setting job-id=26688912665 2024-06-26T07:10:55.9712491Z setting job-name=cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:10:55.9911612Z Prepare all required actions 2024-06-26T07:10:55.9912064Z Getting action download info 2024-06-26T07:10:56.1058024Z ##[group]Run ./.github/actions/filter-test-configs 2024-06-26T07:10:56.1058496Z with: 2024-06-26T07:10:56.1058947Z github-token: *** 2024-06-26T07:10:56.1067685Z test-matrix: {"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-06-26T07:10:56.1076481Z job-name: cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:10:56.1077239Z env: 2024-06-26T07:10:56.1077526Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:56.1077998Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:56.1078485Z ##[endgroup] 2024-06-26T07:10:56.1119118Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:10:56.1119687Z with: 2024-06-26T07:10:56.1119963Z shell: bash 2024-06-26T07:10:56.1120269Z timeout_minutes: 10 2024-06-26T07:10:56.1120610Z max_attempts: 5 2024-06-26T07:10:56.1121058Z retry_wait_seconds: 30 2024-06-26T07:10:56.1122215Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:10:56.1123431Z polling_interval_seconds: 1 2024-06-26T07:10:56.1123821Z warning_on_retry: true 2024-06-26T07:10:56.1124170Z continue_on_error: false 2024-06-26T07:10:56.1124522Z env: 2024-06-26T07:10:56.1124799Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:56.1125255Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:56.1125958Z GITHUB_TOKEN: *** 2024-06-26T07:10:56.1126277Z ##[endgroup] 2024-06-26T07:10:56.1620807Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:10:56.3787383Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T07:10:56.3941503Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.7/site-packages (2.27.1) 2024-06-26T07:10:56.4089792Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.7/site-packages (6.0.1) 2024-06-26T07:10:56.4098108Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2024.6.2) 2024-06-26T07:10:56.4106742Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (1.26.19) 2024-06-26T07:10:56.4312338Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2.0.12) 2024-06-26T07:10:56.4332498Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (3.7) 2024-06-26T07:10:57.1622834Z Command completed after 1 attempt(s). 2024-06-26T07:10:57.1661598Z ##[group]Run set -x 2024-06-26T07:10:57.1661942Z set -x 2024-06-26T07:10:57.1662263Z  2024-06-26T07:10:57.1662838Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:10:57.1663530Z # in runner workspace 2024-06-26T07:10:57.1664191Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-06-26T07:10:57.1672595Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:10:57.1673113Z env: 2024-06-26T07:10:57.1673392Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:57.1673860Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:57.1674358Z ##[endgroup] 2024-06-26T07:10:57.1698371Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-06-26T07:10:57.1914090Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:10:57.1914645Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:10:57.1915110Z echo "Job name: ${JOB_NAME}" 2024-06-26T07:10:57.1915516Z  2024-06-26T07:10:57.1916082Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:10:57.1916776Z # in runner workspace 2024-06-26T07:10:57.1917377Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-06-26T07:10:57.1918025Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-06-26T07:10:57.1918494Z  --job-name "${JOB_NAME}" \ 2024-06-26T07:10:57.1927023Z  --test-matrix "{"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-06-26T07:10:57.1935743Z  --selected-test-configs "" \ 2024-06-26T07:10:57.1936212Z  --pr-number "${PR_NUMBER}" \ 2024-06-26T07:10:57.1936647Z  --tag "${TAG}" \ 2024-06-26T07:10:57.1937096Z  --event-name "${EVENT_NAME}" \ 2024-06-26T07:10:57.1937541Z  --schedule "${SCHEDULE}" \ 2024-06-26T07:10:57.1937983Z  --branch "${HEAD_BRANCH}" 2024-06-26T07:10:57.1945612Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:10:57.1946118Z env: 2024-06-26T07:10:57.1946403Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:57.1947010Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:57.1947715Z GITHUB_TOKEN: *** 2024-06-26T07:10:57.1948400Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:10:57.1949156Z PR_NUMBER: 2024-06-26T07:10:57.1949479Z TAG: ciflow/inductor/129470 2024-06-26T07:10:57.1949855Z EVENT_NAME: push 2024-06-26T07:10:57.1950175Z SCHEDULE: 2024-06-26T07:10:57.1950464Z HEAD_BRANCH: 2024-06-26T07:10:57.1950775Z ##[endgroup] 2024-06-26T07:10:57.1969507Z Workflow: inductor 2024-06-26T07:10:57.1970415Z Job name: cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:10:57.4042389Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-06-26T07:10:57.5279296Z ##[group]Run echo "Filtered matrix:" 2024-06-26T07:10:57.5279760Z echo "Filtered matrix:" 2024-06-26T07:10:57.5288181Z echo "{"include": [{"config": "inductor", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_distributed", "shard": 1, "num_shards": 1, "runner": "linux.g5.12xlarge.nvidia.gpu"}, {"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_cpp_wrapper_abi_compatible", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" 2024-06-26T07:10:57.5297092Z  2024-06-26T07:10:57.5297375Z echo 2024-06-26T07:10:57.5297752Z echo "Is the current job unstable? False" 2024-06-26T07:10:57.5298213Z  2024-06-26T07:10:57.5298491Z echo 2024-06-26T07:10:57.5298852Z echo "Is keep-going label set? False" 2024-06-26T07:10:57.5299289Z  2024-06-26T07:10:57.5299559Z echo 2024-06-26T07:10:57.5299874Z echo "Renabled issues? " 2024-06-26T07:10:57.5307936Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:10:57.5308438Z env: 2024-06-26T07:10:57.5308721Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:57.5309175Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:57.5309659Z ##[endgroup] 2024-06-26T07:10:57.5328488Z Filtered matrix: 2024-06-26T07:10:57.5338750Z {include: [{config: inductor, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_distributed, shard: 1, num_shards: 1, runner: linux.g5.12xlarge.nvidia.gpu}, {config: inductor_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_inductor_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_inductor_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_cpp_wrapper_abi_compatible, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}]} 2024-06-26T07:10:57.5347192Z 2024-06-26T07:10:57.5347346Z Is the current job unstable? False 2024-06-26T07:10:57.5347638Z 2024-06-26T07:10:57.5347916Z Is keep-going label set? False 2024-06-26T07:10:57.5348179Z 2024-06-26T07:10:57.5348301Z Renabled issues? 2024-06-26T07:10:57.5386762Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:10:57.5387631Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:10:57.5394414Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:10:57.5394924Z env: 2024-06-26T07:10:57.5395216Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:57.5395689Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:57.5396191Z JOB_TIMEOUT: 240 2024-06-26T07:10:57.5396506Z ##[endgroup] 2024-06-26T07:10:57.5464253Z ##[group]Run set -x 2024-06-26T07:10:57.5464656Z set -x 2024-06-26T07:10:57.5464964Z  2024-06-26T07:10:57.5465326Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-06-26T07:10:57.5465879Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-06-26T07:10:57.5466463Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-06-26T07:10:57.5467260Z  TEST_COMMAND=.ci/onnx/test.sh 2024-06-26T07:10:57.5467682Z else 2024-06-26T07:10:57.5468028Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:10:57.5468488Z fi 2024-06-26T07:10:57.5468789Z  2024-06-26T07:10:57.5469281Z # detached container should get cleaned up by teardown_ec2_linux 2024-06-26T07:10:57.5470053Z # TODO: Stop building test binaries as part of the build phase 2024-06-26T07:10:57.5470752Z # Used for GPU_FLAG since that doesn't play nice 2024-06-26T07:10:57.5471360Z # shellcheck disable=SC2086,SC2090 2024-06-26T07:10:57.5471832Z container_name=$(docker run \ 2024-06-26T07:10:57.5472265Z  ${GPU_FLAG:-} \ 2024-06-26T07:10:57.5472654Z  -e BUILD_ENVIRONMENT \ 2024-06-26T07:10:57.5473074Z  -e PR_NUMBER \ 2024-06-26T07:10:57.5473444Z  -e GITHUB_ACTIONS \ 2024-06-26T07:10:57.5473853Z  -e GITHUB_REPOSITORY \ 2024-06-26T07:10:57.5474272Z  -e GITHUB_WORKFLOW \ 2024-06-26T07:10:57.5474676Z  -e GITHUB_JOB \ 2024-06-26T07:10:57.5475044Z  -e GITHUB_RUN_ID \ 2024-06-26T07:10:57.5475457Z  -e GITHUB_RUN_NUMBER \ 2024-06-26T07:10:57.5475885Z  -e GITHUB_RUN_ATTEMPT \ 2024-06-26T07:10:57.5476291Z  -e JOB_ID \ 2024-06-26T07:10:57.5476651Z  -e JOB_NAME \ 2024-06-26T07:10:57.5477065Z  -e BASE_SHA \ 2024-06-26T07:10:57.5477435Z  -e BRANCH \ 2024-06-26T07:10:57.5477776Z  -e SHA1 \ 2024-06-26T07:10:57.5478132Z  -e AWS_DEFAULT_REGION \ 2024-06-26T07:10:57.5478563Z  -e IN_WHEEL_TEST \ 2024-06-26T07:10:57.5478962Z  -e SHARD_NUMBER \ 2024-06-26T07:10:57.5479338Z  -e TEST_CONFIG \ 2024-06-26T07:10:57.5479729Z  -e NUM_TEST_SHARDS \ 2024-06-26T07:10:57.5480138Z  -e REENABLED_ISSUES \ 2024-06-26T07:10:57.5480566Z  -e CONTINUE_THROUGH_ERROR \ 2024-06-26T07:10:57.5481003Z  -e VERBOSE_TEST_LOGS \ 2024-06-26T07:10:57.5481422Z  -e NO_TEST_TIMEOUT \ 2024-06-26T07:10:57.5481815Z  -e NO_TD \ 2024-06-26T07:10:57.5482169Z  -e TD_DISTRIBUTED \ 2024-06-26T07:10:57.5482573Z  -e PR_LABELS \ 2024-06-26T07:10:57.5482996Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-06-26T07:10:57.5483476Z  -e SCCACHE_BUCKET \ 2024-06-26T07:10:57.5483882Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-06-26T07:10:57.5484306Z  -e XLA_CUDA \ 2024-06-26T07:10:57.5484730Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-06-26T07:10:57.5485260Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-06-26T07:10:57.5485803Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-06-26T07:10:57.5486341Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-06-26T07:10:57.5486824Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-06-26T07:10:57.5487263Z  -e DASHBOARD_TAG \ 2024-06-26T07:10:57.5487747Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-06-26T07:10:57.5488325Z  --security-opt seccomp=unconfined \ 2024-06-26T07:10:57.5488801Z  --cap-add=SYS_PTRACE \ 2024-06-26T07:10:57.5489337Z  --ipc=host \ 2024-06-26T07:10:57.5489713Z  --shm-size="${SHM_SIZE}" \ 2024-06-26T07:10:57.5490139Z  --tty \ 2024-06-26T07:10:57.5490473Z  --detach \ 2024-06-26T07:10:57.5490852Z  --name="${container_name}" \ 2024-06-26T07:10:57.5491278Z  --user jenkins \ 2024-06-26T07:10:57.5491862Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-06-26T07:10:57.5492452Z  -w /var/lib/jenkins/workspace \ 2024-06-26T07:10:57.5492909Z  "${DOCKER_IMAGE}" 2024-06-26T07:10:57.5493272Z ) 2024-06-26T07:10:57.5493696Z # Propagate download.pytorch.org IP to container 2024-06-26T07:10:57.5494649Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-06-26T07:10:57.5495641Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-06-26T07:10:57.5496572Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-06-26T07:10:57.5503821Z shell: /usr/bin/bash -e {0} 2024-06-26T07:10:57.5504267Z env: 2024-06-26T07:10:57.5504556Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:10:57.5505011Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:10:57.5505646Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:10:57.5506174Z PR_NUMBER: 2024-06-26T07:10:57.5506771Z GITHUB_REPOSITORY: pytorch/pytorch 2024-06-26T07:10:57.5507203Z GITHUB_WORKFLOW: inductor 2024-06-26T07:10:57.5507574Z GITHUB_JOB: test 2024-06-26T07:10:57.5507909Z GITHUB_RUN_ID: 9673646808 2024-06-26T07:10:57.5508278Z GITHUB_RUN_NUMBER: 76977 2024-06-26T07:10:57.5508631Z GITHUB_RUN_ATTEMPT: 1 2024-06-26T07:10:57.5508974Z JOB_ID: 26688912665 2024-06-26T07:10:57.5509655Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:10:57.5510411Z BRANCH: 2024-06-26T07:10:57.5510761Z SHA1: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:10:57.5511295Z BASE_SHA: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:10:57.5511795Z TEST_CONFIG: aot_inductor_torchbench 2024-06-26T07:10:57.5512217Z SHARD_NUMBER: 1 2024-06-26T07:10:57.5512531Z NUM_TEST_SHARDS: 2 2024-06-26T07:10:57.5512871Z REENABLED_ISSUES: 2024-06-26T07:10:57.5513232Z CONTINUE_THROUGH_ERROR: False 2024-06-26T07:10:57.5513626Z VERBOSE_TEST_LOGS: False 2024-06-26T07:10:57.5513995Z NO_TEST_TIMEOUT: False 2024-06-26T07:10:57.5514344Z NO_TD: False 2024-06-26T07:10:57.5514661Z TD_DISTRIBUTED: False 2024-06-26T07:10:57.5515083Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-06-26T07:10:57.5515597Z SCCACHE_S3_KEY_PREFIX: inductor 2024-06-26T07:10:57.5515992Z SHM_SIZE: 2g 2024-06-26T07:10:57.5517155Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:10:57.5518344Z XLA_CUDA: 2024-06-26T07:10:57.5518840Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:10:57.5519464Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-06-26T07:10:57.5519916Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-06-26T07:10:57.5520325Z DASHBOARD_TAG: 2024-06-26T07:10:57.5520853Z HUGGING_FACE_HUB_TOKEN: *** 2024-06-26T07:10:57.5521231Z ##[endgroup] 2024-06-26T07:10:57.5541978Z + [[ aot_inductor_torchbench == \m\u\l\t\i\g\p\u ]] 2024-06-26T07:10:57.5542697Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]] 2024-06-26T07:10:57.5543229Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:10:57.5551375Z +++ nproc --ignore=2 2024-06-26T07:10:57.5566403Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_9673646808 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:11:08.8013521Z + container_name=80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T07:11:08.8015363Z + grep download.pytorch.org /etc/hosts 2024-06-26T07:11:08.8016778Z + docker exec -i 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 sudo bash -c '/bin/cat >> /etc/hosts' 2024-06-26T07:11:08.8665279Z + echo DOCKER_CONTAINER_ID=80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T07:11:08.8669189Z ++ echo dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:11:08.8671758Z + docker exec -t 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 sh -c 'pip install dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-06-26T07:11:09.2650981Z Processing ./dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+gitb8c4c54) 2024-06-26T07:11:09.5822554Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.13.1) 2024-06-26T07:11:09.5828227Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (4.12.2) 2024-06-26T07:11:09.5830783Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.12.1) 2024-06-26T07:11:09.5832992Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.8.8) 2024-06-26T07:11:09.5835779Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.1.4) 2024-06-26T07:11:09.5838544Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2024.2.0) 2024-06-26T07:11:09.5864244Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.3.0) 2024-06-26T07:11:09.5922385Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.21.2) 2024-06-26T07:11:09.6354670Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.1.5) 2024-06-26T07:11:09.6513425Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.3.0) 2024-06-26T07:11:10.9840394Z Installing collected packages: torch 2024-06-26T07:11:19.7411781Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-06-26T07:11:19.7413431Z timm 0.9.7 requires torchvision, which is not installed. 2024-06-26T07:11:19.7414113Z Successfully installed torch-2.5.0a0+gitb8c4c54 2024-06-26T07:11:19.8252681Z ++ dirname .ci/pytorch/test.sh 2024-06-26T07:11:19.8257033Z + source .ci/pytorch/common.sh 2024-06-26T07:11:19.8259119Z +++ dirname .ci/pytorch/common.sh 2024-06-26T07:11:19.8263948Z ++ source .ci/pytorch/common_utils.sh 2024-06-26T07:11:19.8266194Z +++ declare -f -t trap_add 2024-06-26T07:11:19.8272213Z ++ set -ex 2024-06-26T07:11:19.8272875Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-06-26T07:11:19.8273406Z ++ BUILD_TEST_LIBTORCH=0 2024-06-26T07:11:19.8274082Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]] 2024-06-26T07:11:19.8275803Z ++ stat -c %u /var/lib/jenkins/workspace 2024-06-26T07:11:19.8285277Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-06-26T07:11:19.8286057Z + trap_add cleanup_workspace EXIT 2024-06-26T07:11:19.8286520Z + trap_add_cmd=cleanup_workspace 2024-06-26T07:11:19.8286911Z + shift 2024-06-26T07:11:19.8287194Z + for trap_add_name in "$@" 2024-06-26T07:11:19.8291414Z +++ trap -p EXIT 2024-06-26T07:11:19.8292668Z ++ eval 'extract_trap_cmd ' 2024-06-26T07:11:19.8293136Z +++ extract_trap_cmd 2024-06-26T07:11:19.8293547Z +++ printf '%s\n' '' 2024-06-26T07:11:19.8293932Z ++ printf '%s\n' cleanup_workspace 2024-06-26T07:11:19.8294646Z + trap -- ' 2024-06-26T07:11:19.8295126Z cleanup_workspace' EXIT 2024-06-26T07:11:19.8295677Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-06-26T07:11:20.4211223Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-06-26T07:11:20.4224493Z + echo 'Environment variables:' 2024-06-26T07:11:20.4224922Z Environment variables: 2024-06-26T07:11:20.4225252Z + env 2024-06-26T07:11:20.4232000Z INSTALLED_DB=yes 2024-06-26T07:11:20.4232535Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-06-26T07:11:20.4233007Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:11:20.4233400Z NV_NVML_DEV_VERSION=12.1.105-1 2024-06-26T07:11:20.4234001Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:11:20.4234598Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:11:20.4235214Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-06-26T07:11:20.4235940Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:11:20.4236564Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:11:20.4237065Z HOSTNAME=80c146ebd76a 2024-06-26T07:11:20.4237919Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_af372510-8b50-4e25-9e68-f7a325eec49a 2024-06-26T07:11:20.4238917Z GITHUB_ACTION=__self 2024-06-26T07:11:20.4239424Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:11:20.4243820Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-06-26T07:11:20.4247801Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-06-26T07:11:20.4248316Z NV_NVTX_VERSION=12.1.105-1 2024-06-26T07:11:20.4248668Z GITHUB_RUN_NUMBER=76977 2024-06-26T07:11:20.4249067Z TEST_CONFIG=aot_inductor_torchbench 2024-06-26T07:11:20.4249494Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:11:20.4249967Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:11:20.4250436Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-06-26T07:11:20.4250873Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-06-26T07:11:20.4251300Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-06-26T07:11:20.4251946Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:11:20.4252438Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:11:20.4252907Z GITHUB_REF_TYPE=tag 2024-06-26T07:11:20.4253233Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:11:20.4253622Z NCCL_VERSION=2.17.1-1 2024-06-26T07:11:20.4254012Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:11:20.4254571Z XLA_CUDA= 2024-06-26T07:11:20.4255072Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:11:20.4257020Z *** 2024-06-26T07:11:20.4257312Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:11:20.4257722Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:11:20.4258090Z GITHUB_ACTIONS=true 2024-06-26T07:11:20.4258428Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:20.4258933Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-06-26T07:11:20.4259472Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-06-26T07:11:20.4259943Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:11:20.4260450Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:11:20.4260939Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:11:20.4261774Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:11:20.4262528Z UCC_HOME=/usr 2024-06-26T07:11:20.4262869Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-06-26T07:11:20.4263263Z VERBOSE_TEST_LOGS=False 2024-06-26T07:11:20.4263615Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:11:20.4264168Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-06-26T07:11:20.4264667Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:11:20.4265141Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-06-26T07:11:20.4265527Z SHARD_NUMBER=1 2024-06-26T07:11:20.4265838Z GITHUB_REF_PROTECTED=false 2024-06-26T07:11:20.4266188Z HOME=/var/lib/jenkins 2024-06-26T07:11:20.4266686Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:11:20.4267139Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:11:20.4267623Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:11:20.4268109Z SCCACHE_S3_KEY_PREFIX=inductor 2024-06-26T07:11:20.4268484Z CUDA_VERSION=12.1.1 2024-06-26T07:11:20.4268905Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-06-26T07:11:20.4269346Z NUM_TEST_SHARDS=2 2024-06-26T07:11:20.4269645Z UCX_HOME=/usr 2024-06-26T07:11:20.4270153Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-06-26T07:11:20.4271226Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_af372510-8b50-4e25-9e68-f7a325eec49a 2024-06-26T07:11:20.4272526Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:11:20.4273743Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_af372510-8b50-4e25-9e68-f7a325eec49a 2024-06-26T07:11:20.4274854Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:11:20.4275510Z GITHUB_EVENT_NAME=push 2024-06-26T07:11:20.4275841Z DASHBOARD_TAG= 2024-06-26T07:11:20.4276145Z GITHUB_RUN_ID=9673646808 2024-06-26T07:11:20.4276602Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-06-26T07:11:20.4277138Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-06-26T07:11:20.4278115Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_af372510-8b50-4e25-9e68-f7a325eec49a 2024-06-26T07:11:20.4279021Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:11:20.4279430Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-06-26T07:11:20.4279810Z PR_NUMBER= 2024-06-26T07:11:20.4280088Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:11:20.4280425Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:11:20.4280876Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:11:20.4281338Z TERM=xterm 2024-06-26T07:11:20.4281665Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-06-26T07:11:20.4282081Z INSTALLED_VISION=yes 2024-06-26T07:11:20.4282396Z BRANCH= 2024-06-26T07:11:20.4282679Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:11:20.4283075Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:11:20.4283626Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:11:20.4284384Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:11:20.4285145Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:11:20.4285641Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:11:20.4286113Z REENABLED_ISSUES= 2024-06-26T07:11:20.4286485Z SHLVL=1 2024-06-26T07:11:20.4286747Z MAX_JOBS=14 2024-06-26T07:11:20.4287063Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-06-26T07:11:20.4287423Z NVARCH=x86_64 2024-06-26T07:11:20.4287728Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:11:20.4288180Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:11:20.4288714Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:11:20.4289198Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-06-26T07:11:20.4289853Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:11:20.4290427Z GITHUB_JOB=test 2024-06-26T07:11:20.4290840Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-06-26T07:11:20.4291402Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:11:20.4291969Z NO_TEST_TIMEOUT=False 2024-06-26T07:11:20.4292299Z TD_DISTRIBUTED=False 2024-06-26T07:11:20.4292680Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-06-26T07:11:20.4293121Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:11:20.4293542Z NV_NVPROF_VERSION=12.1.105-1 2024-06-26T07:11:20.4293914Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:11:20.4294268Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:11:20.4294618Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:11:20.4295675Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:11:20.4296751Z GITHUB_BASE_REF= 2024-06-26T07:11:20.4297081Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:11:20.4297452Z CI=true 2024-06-26T07:11:20.4297765Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:11:20.4298193Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:11:20.4298571Z JOB_ID=26688912665 2024-06-26T07:11:20.4298886Z INSTALLED_PROTOBUF=yes 2024-06-26T07:11:20.4299219Z GITHUB_HEAD_REF= 2024-06-26T07:11:20.4299531Z GITHUB_ACTION_REF= 2024-06-26T07:11:20.4299970Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:11:20.4300439Z GITHUB_WORKFLOW=inductor 2024-06-26T07:11:20.4300803Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:11:20.4301716Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_af372510-8b50-4e25-9e68-f7a325eec49a 2024-06-26T07:11:20.4302586Z NO_TD=False 2024-06-26T07:11:20.4302888Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:11:20.4303252Z _=/usr/bin/env 2024-06-26T07:11:20.4303725Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-06-26T07:11:20.4387571Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-06-26T07:11:20.4388910Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-06-26T07:11:20.4389989Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-06-26T07:11:20.4390867Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-06-26T07:11:20.4391478Z + BUILD_DIR=build 2024-06-26T07:11:20.4391832Z + BUILD_RENAMED_DIR=build_renamed 2024-06-26T07:11:20.4392246Z + BUILD_BIN_DIR=build/bin 2024-06-26T07:11:20.4392737Z + SHARD_NUMBER=1 2024-06-26T07:11:20.4393166Z + NUM_TEST_SHARDS=2 2024-06-26T07:11:20.4393494Z + export VALGRIND=ON 2024-06-26T07:11:20.4393815Z + VALGRIND=ON 2024-06-26T07:11:20.4394288Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]] 2024-06-26T07:11:20.4394778Z + [[ 0 == \1 ]] 2024-06-26T07:11:20.4395077Z + [[ False == \1 ]] 2024-06-26T07:11:20.4395545Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]] 2024-06-26T07:11:20.4396073Z ++ realpath build/custom_test_artifacts 2024-06-26T07:11:20.4398816Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-06-26T07:11:20.4399753Z + [[ -n '' ]] 2024-06-26T07:11:20.4400103Z + echo 'Environment variables' 2024-06-26T07:11:20.4400494Z Environment variables 2024-06-26T07:11:20.4400805Z + env 2024-06-26T07:11:20.4405150Z INSTALLED_DB=yes 2024-06-26T07:11:20.4405746Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-06-26T07:11:20.4406433Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:11:20.4406930Z NV_NVML_DEV_VERSION=12.1.105-1 2024-06-26T07:11:20.4407909Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:11:20.4408663Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:11:20.4409229Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-06-26T07:11:20.4409746Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:11:20.4410428Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-06-26T07:11:20.4411092Z HOSTNAME=80c146ebd76a 2024-06-26T07:11:20.4412198Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_af372510-8b50-4e25-9e68-f7a325eec49a 2024-06-26T07:11:20.4413291Z GITHUB_ACTION=__self 2024-06-26T07:11:20.4413764Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:11:20.4417937Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-06-26T07:11:20.4421771Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-06-26T07:11:20.4422284Z NV_NVTX_VERSION=12.1.105-1 2024-06-26T07:11:20.4422641Z GITHUB_RUN_NUMBER=76977 2024-06-26T07:11:20.4422992Z TEST_CONFIG=aot_inductor_torchbench 2024-06-26T07:11:20.4423410Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:11:20.4423886Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:11:20.4424462Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-06-26T07:11:20.4424902Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-06-26T07:11:20.4425329Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-06-26T07:11:20.4425764Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:11:20.4426252Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:11:20.4427020Z GITHUB_REF_TYPE=tag 2024-06-26T07:11:20.4427345Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:11:20.4427725Z NCCL_VERSION=2.17.1-1 2024-06-26T07:11:20.4428106Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:11:20.4428537Z XLA_CUDA= 2024-06-26T07:11:20.4428986Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:11:20.4429379Z *** 2024-06-26T07:11:20.4429663Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:11:20.4430060Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:11:20.4430430Z GITHUB_ACTIONS=true 2024-06-26T07:11:20.4430767Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:20.4431256Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-06-26T07:11:20.4431812Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-06-26T07:11:20.4432307Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:11:20.4432810Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:11:20.4433290Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:11:20.4434072Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:11:20.4434791Z UCC_HOME=/usr 2024-06-26T07:11:20.4435122Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-06-26T07:11:20.4447920Z VERBOSE_TEST_LOGS=False 2024-06-26T07:11:20.4448271Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:11:20.4448755Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-06-26T07:11:20.4449251Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:11:20.4449853Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-06-26T07:11:20.4450232Z SHARD_NUMBER=1 2024-06-26T07:11:20.4450542Z GITHUB_REF_PROTECTED=false 2024-06-26T07:11:20.4450895Z HOME=/var/lib/jenkins 2024-06-26T07:11:20.4451256Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:11:20.4451693Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:11:20.4452246Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:11:20.4452734Z SCCACHE_S3_KEY_PREFIX=inductor 2024-06-26T07:11:20.4453102Z CUDA_VERSION=12.1.1 2024-06-26T07:11:20.4453519Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-06-26T07:11:20.4453945Z NUM_TEST_SHARDS=2 2024-06-26T07:11:20.4454245Z UCX_HOME=/usr 2024-06-26T07:11:20.4454752Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-06-26T07:11:20.4455824Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_af372510-8b50-4e25-9e68-f7a325eec49a 2024-06-26T07:11:20.4457058Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86 / test (aot_inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:11:20.4458294Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_af372510-8b50-4e25-9e68-f7a325eec49a 2024-06-26T07:11:20.4459410Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:11:20.4460062Z GITHUB_EVENT_NAME=push 2024-06-26T07:11:20.4460392Z DASHBOARD_TAG= 2024-06-26T07:11:20.4460696Z GITHUB_RUN_ID=9673646808 2024-06-26T07:11:20.4461145Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-06-26T07:11:20.4461684Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-06-26T07:11:20.4462717Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_af372510-8b50-4e25-9e68-f7a325eec49a 2024-06-26T07:11:20.4463621Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:11:20.4464097Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-06-26T07:11:20.4464475Z PR_NUMBER= 2024-06-26T07:11:20.4464747Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:11:20.4465064Z VALGRIND=ON 2024-06-26T07:11:20.4465359Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:11:20.4465791Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:11:20.4466247Z TERM=xterm 2024-06-26T07:11:20.4466768Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-06-26T07:11:20.4467198Z INSTALLED_VISION=yes 2024-06-26T07:11:20.4467501Z BRANCH= 2024-06-26T07:11:20.4467788Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:11:20.4468185Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:11:20.4468595Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:11:20.4469347Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:11:20.4470088Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:11:20.4470568Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:11:20.4471031Z REENABLED_ISSUES= 2024-06-26T07:11:20.4471330Z SHLVL=1 2024-06-26T07:11:20.4471585Z MAX_JOBS=14 2024-06-26T07:11:20.4471893Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-06-26T07:11:20.4472277Z NVARCH=x86_64 2024-06-26T07:11:20.4472587Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:11:20.4473036Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:11:20.4473565Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:11:20.4474033Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-06-26T07:11:20.4474689Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:11:20.4475252Z GITHUB_JOB=test 2024-06-26T07:11:20.4475639Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-06-26T07:11:20.4476190Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:11:20.4476699Z NO_TEST_TIMEOUT=False 2024-06-26T07:11:20.4477024Z TD_DISTRIBUTED=False 2024-06-26T07:11:20.4477390Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-06-26T07:11:20.4477819Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:11:20.4478231Z NV_NVPROF_VERSION=12.1.105-1 2024-06-26T07:11:20.4478589Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:11:20.4478927Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:11:20.4479396Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:11:20.4480445Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:11:20.4481532Z GITHUB_BASE_REF= 2024-06-26T07:11:20.4481845Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:11:20.4482206Z CI=true 2024-06-26T07:11:20.4482582Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:11:20.4482991Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:11:20.4483367Z JOB_ID=26688912665 2024-06-26T07:11:20.4483677Z INSTALLED_PROTOBUF=yes 2024-06-26T07:11:20.4484008Z GITHUB_HEAD_REF= 2024-06-26T07:11:20.4484302Z GITHUB_ACTION_REF= 2024-06-26T07:11:20.4484725Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:11:20.4485192Z GITHUB_WORKFLOW=inductor 2024-06-26T07:11:20.4485547Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:11:20.4486438Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_af372510-8b50-4e25-9e68-f7a325eec49a 2024-06-26T07:11:20.4487262Z NO_TD=False 2024-06-26T07:11:20.4487558Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:11:20.4487910Z _=/usr/bin/env 2024-06-26T07:11:20.4488250Z + echo 'Testing pytorch' 2024-06-26T07:11:20.4488593Z Testing pytorch 2024-06-26T07:11:20.4488908Z + export LANG=C.UTF-8 2024-06-26T07:11:20.4489232Z + LANG=C.UTF-8 2024-06-26T07:11:20.4489540Z + PR_NUMBER= 2024-06-26T07:11:20.4489889Z + [[ aot_inductor_torchbench == \d\e\f\a\u\l\t ]] 2024-06-26T07:11:20.4490430Z + [[ aot_inductor_torchbench == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-06-26T07:11:20.4490940Z + [[ aot_inductor_torchbench == \s\l\o\w ]] 2024-06-26T07:11:20.4491580Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-06-26T07:11:20.4492324Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]] 2024-06-26T07:11:20.4492872Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:11:20.4493335Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:11:20.4493796Z + [[ aot_inductor_torchbench == *crossref* ]] 2024-06-26T07:11:20.4494390Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-06-26T07:11:20.4495016Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]] 2024-06-26T07:11:20.4495654Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:11:20.4496214Z + pip_install --user ninja==1.10.2 2024-06-26T07:11:20.4496748Z + pip install --progress-bar off --user ninja==1.10.2 2024-06-26T07:11:20.8132557Z Collecting ninja==1.10.2 2024-06-26T07:11:20.8302720Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-06-26T07:11:20.8443101Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-06-26T07:11:22.1645975Z Installing collected packages: ninja 2024-06-26T07:11:22.1713836Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-06-26T07:11:22.1715315Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-06-26T07:11:22.1760484Z Successfully installed ninja-1.10.2 2024-06-26T07:11:22.2648399Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:11:22.2668795Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:11:22.2670240Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-06-26T07:11:22.2670761Z + install_tlparse 2024-06-26T07:11:22.2671134Z + pip_install --user tlparse==0.3.7 2024-06-26T07:11:22.2671685Z + pip install --progress-bar off --user tlparse==0.3.7 2024-06-26T07:11:22.6239168Z Collecting tlparse==0.3.7 2024-06-26T07:11:22.6388269Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-06-26T07:11:22.6522751Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-06-26T07:11:23.9991621Z Installing collected packages: tlparse 2024-06-26T07:11:24.0355333Z Successfully installed tlparse-0.3.7 2024-06-26T07:11:24.1222619Z ++ python -m site --user-base 2024-06-26T07:11:24.1380600Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:11:24.1382434Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]] 2024-06-26T07:11:24.1383262Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]] 2024-06-26T07:11:24.1384136Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:11:24.1385394Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-06-26T07:11:24.1386857Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass 2024-06-26T07:11:24.1387585Z + cd test 2024-06-26T07:11:24.1388124Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-06-26T07:11:25.6229731Z + [[ aot_inductor_torchbench == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-06-26T07:11:25.6230514Z + [[ aot_inductor_torchbench == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-06-26T07:11:25.6231572Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:11:25.6232128Z + pushd test 2024-06-26T07:11:25.6232538Z ~/workspace/test ~/workspace 2024-06-26T07:11:25.6233080Z ++ python -c 'import torch; print(torch.version.cuda)' 2024-06-26T07:11:27.1021973Z + CUDA_VERSION=12.1 2024-06-26T07:11:27.1022518Z + '[' 12.1 == 12.4 ']' 2024-06-26T07:11:27.1022857Z + ISCUDA124= 2024-06-26T07:11:27.1023156Z + popd 2024-06-26T07:11:27.1023421Z ~/workspace 2024-06-26T07:11:27.1026625Z + DYNAMO_BENCHMARK_FLAGS=() 2024-06-26T07:11:27.1027262Z + [[ aot_inductor_torchbench == *dynamo_eager* ]] 2024-06-26T07:11:27.1027950Z + [[ aot_inductor_torchbench == *aot_eager* ]] 2024-06-26T07:11:27.1028454Z + [[ aot_inductor_torchbench == *aot_inductor* ]] 2024-06-26T07:11:27.1029050Z + DYNAMO_BENCHMARK_FLAGS+=(--export-aot-inductor) 2024-06-26T07:11:27.1029577Z + [[ aot_inductor_torchbench == *dynamic* ]] 2024-06-26T07:11:27.1030076Z + [[ aot_inductor_torchbench == *cpu_inductor* ]] 2024-06-26T07:11:27.1030608Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-06-26T07:11:27.1053459Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-06-26T07:11:27.1054299Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-06-26T07:11:27.1055745Z + cd test 2024-06-26T07:11:27.1056203Z + python -c 'import torch; print(torch.__config__.show())' 2024-06-26T07:11:28.4619211Z PyTorch built with: 2024-06-26T07:11:28.4619865Z - GCC 9.4 2024-06-26T07:11:28.4620245Z - C++ Version: 201703 2024-06-26T07:11:28.4621132Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:11:28.4622259Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:11:28.4622928Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:11:28.4623438Z - LAPACK is enabled (usually provided by MKL) 2024-06-26T07:11:28.4623926Z - NNPACK is enabled 2024-06-26T07:11:28.4624406Z - CPU capability usage: AVX2 2024-06-26T07:11:28.4624823Z - CUDA Runtime 12.1 2024-06-26T07:11:28.4625350Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-06-26T07:11:28.4625960Z - CuDNN 90.1 (built against CUDA 12.4) 2024-06-26T07:11:28.4626410Z - Magma 2.6.1 2024-06-26T07:11:28.4633878Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-06-26T07:11:28.4640623Z 2024-06-26T07:11:28.7344516Z + cd test 2024-06-26T07:11:28.7345214Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-06-26T07:11:29.9517451Z ATen/Parallel: 2024-06-26T07:11:29.9517936Z at::get_num_threads() : 8 2024-06-26T07:11:29.9518507Z at::get_num_interop_threads() : 16 2024-06-26T07:11:29.9519038Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:11:29.9519443Z omp_get_max_threads() : 8 2024-06-26T07:11:29.9520449Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:11:29.9521282Z mkl_get_max_threads() : 8 2024-06-26T07:11:29.9521908Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:11:29.9522552Z std::thread::hardware_concurrency() : 16 2024-06-26T07:11:29.9522986Z Environment variables: 2024-06-26T07:11:29.9523338Z OMP_NUM_THREADS : [not set] 2024-06-26T07:11:29.9523711Z MKL_NUM_THREADS : [not set] 2024-06-26T07:11:29.9524091Z ATen parallel backend: OpenMP 2024-06-26T07:11:29.9524358Z 2024-06-26T07:11:30.1945681Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-06-26T07:11:30.1946368Z + [[ aot_inductor_torchbench == *backward* ]] 2024-06-26T07:11:30.1947339Z + [[ aot_inductor_torchbench == *xla* ]] 2024-06-26T07:11:30.1947851Z + [[ aot_inductor_torchbench == *executorch* ]] 2024-06-26T07:11:30.1948441Z + [[ aot_inductor_torchbench == \j\i\t\_\l\e\g\a\c\y ]] 2024-06-26T07:11:30.1949190Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-06-26T07:11:30.1949752Z + [[ aot_inductor_torchbench == distributed ]] 2024-06-26T07:11:30.1950290Z + [[ aot_inductor_torchbench == *inductor_distributed* ]] 2024-06-26T07:11:30.1950898Z + [[ aot_inductor_torchbench == *inductor-halide* ]] 2024-06-26T07:11:30.1951538Z + [[ aot_inductor_torchbench == *inductor-micro-benchmark* ]] 2024-06-26T07:11:30.1952111Z + [[ aot_inductor_torchbench == *huggingface* ]] 2024-06-26T07:11:30.1952598Z + [[ aot_inductor_torchbench == *timm* ]] 2024-06-26T07:11:30.1953072Z + [[ aot_inductor_torchbench == *torchbench* ]] 2024-06-26T07:11:30.1953589Z + [[ aot_inductor_torchbench == *cpu_inductor* ]] 2024-06-26T07:11:30.1954041Z + install_torchaudio cuda 2024-06-26T07:11:30.1954388Z + local commit 2024-06-26T07:11:30.1954700Z ++ get_pinned_commit audio 2024-06-26T07:11:30.1955073Z ++ cat .github/ci_commit_pins/audio.txt 2024-06-26T07:11:30.1961306Z + commit=b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:11:30.1962008Z + [[ cuda == \c\u\d\a ]] 2024-06-26T07:11:30.1962574Z + TORCH_CUDA_ARCH_LIST='8.0;8.6' 2024-06-26T07:11:30.1963514Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:11:30.1964979Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:11:30.5119054Z Collecting git+https://github.com/pytorch/audio.git@b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:11:30.5122642Z Cloning https://github.com/pytorch/audio.git (to revision b829e936f7cc61b48149f5f957a451a38bf2a178) to /tmp/pip-req-build-i0vejmrw 2024-06-26T07:11:30.5138562Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/audio.git /tmp/pip-req-build-i0vejmrw 2024-06-26T07:11:31.1963934Z Running command git rev-parse -q --verify 'sha^b829e936f7cc61b48149f5f957a451a38bf2a178' 2024-06-26T07:11:31.1982570Z Running command git fetch -q https://github.com/pytorch/audio.git b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:11:31.5668367Z Running command git checkout -q b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:11:31.6858417Z Resolved https://github.com/pytorch/audio.git to commit b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:11:31.6859824Z Running command git submodule update --init --recursive -q 2024-06-26T07:11:33.8769595Z Preparing metadata (setup.py) ... [?25l- done 2024-06-26T07:11:33.8800201Z [?25hRequirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchaudio==2.4.0a0+b829e93) (2.5.0a0+gitb8c4c54) 2024-06-26T07:11:33.8867304Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b829e93) (3.13.1) 2024-06-26T07:11:33.8871724Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b829e93) (4.12.2) 2024-06-26T07:11:33.8874310Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b829e93) (1.12.1) 2024-06-26T07:11:33.8876234Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b829e93) (2.8.8) 2024-06-26T07:11:33.8879200Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b829e93) (3.1.4) 2024-06-26T07:11:33.8882261Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b829e93) (2024.2.0) 2024-06-26T07:11:33.9322830Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchaudio==2.4.0a0+b829e93) (2.1.5) 2024-06-26T07:11:33.9479935Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchaudio==2.4.0a0+b829e93) (1.3.0) 2024-06-26T07:11:33.9552735Z Building wheels for collected packages: torchaudio 2024-06-26T07:13:59.5956588Z Building wheel for torchaudio (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / done 2024-06-26T07:13:59.5982720Z [?25h Created wheel for torchaudio: filename=torchaudio-2.4.0a0+b829e93-cp310-cp310-linux_x86_64.whl size=2308811 sha256=6fbf75f896f4012cfbf489c9669f67feb3207095cde4e82212933cc6786ee436 2024-06-26T07:13:59.5984699Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/30/b3/9c/882f183f00af2e582825654a92a04400924519922db53ff522 2024-06-26T07:13:59.6011270Z Successfully built torchaudio 2024-06-26T07:14:00.8404047Z Installing collected packages: torchaudio 2024-06-26T07:14:01.0348157Z Successfully installed torchaudio-2.4.0a0+b829e93 2024-06-26T07:14:01.1694051Z + install_torchtext 2024-06-26T07:14:01.1694466Z + local data_commit 2024-06-26T07:14:01.1699390Z + local text_commit 2024-06-26T07:14:01.1700076Z ++ get_pinned_commit data 2024-06-26T07:14:01.1700594Z ++ cat .github/ci_commit_pins/data.txt 2024-06-26T07:14:01.1710540Z + data_commit=11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:14:01.1712333Z ++ get_pinned_commit text 2024-06-26T07:14:01.1713168Z ++ cat .github/ci_commit_pins/text.txt 2024-06-26T07:14:01.1723413Z + text_commit=b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:14:01.1724619Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:14:01.1726254Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:14:01.4892458Z Collecting git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:14:01.4898982Z Cloning https://github.com/pytorch/data.git (to revision 11bb5b847ea8b9e0d9bb82db3304daf383008d3f) to /tmp/pip-req-build-ifuyq2v5 2024-06-26T07:14:01.4912876Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/data.git /tmp/pip-req-build-ifuyq2v5 2024-06-26T07:14:01.9287997Z Running command git rev-parse -q --verify 'sha^11bb5b847ea8b9e0d9bb82db3304daf383008d3f' 2024-06-26T07:14:01.9307331Z Running command git fetch -q https://github.com/pytorch/data.git 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:14:02.1562568Z Running command git checkout -q 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:14:02.2908576Z Resolved https://github.com/pytorch/data.git to commit 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:14:02.2909759Z Running command git submodule update --init --recursive -q 2024-06-26T07:15:14.4771173Z Preparing metadata (setup.py) ... [?25l- done 2024-06-26T07:15:14.4822715Z [?25hRequirement already satisfied: urllib3>=1.25 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (1.26.19) 2024-06-26T07:15:14.4825274Z Requirement already satisfied: requests in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.32.3) 2024-06-26T07:15:14.4831279Z Requirement already satisfied: torch>2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.5.0a0+gitb8c4c54) 2024-06-26T07:15:14.4895179Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.13.1) 2024-06-26T07:15:14.4898696Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (4.12.2) 2024-06-26T07:15:14.4901059Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.12.1) 2024-06-26T07:15:14.4904078Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.8.8) 2024-06-26T07:15:14.4907448Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.1.4) 2024-06-26T07:15:14.4910577Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2024.2.0) 2024-06-26T07:15:14.5101252Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.3.2) 2024-06-26T07:15:14.5107560Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.7) 2024-06-26T07:15:14.5113985Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (2024.6.2) 2024-06-26T07:15:14.5586817Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.1.5) 2024-06-26T07:15:14.5750066Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.3.0) 2024-06-26T07:15:14.5832823Z Building wheels for collected packages: torchdata 2024-06-26T07:15:16.3019219Z Building wheel for torchdata (setup.py) ... [?25l- \ | done 2024-06-26T07:15:16.3026950Z [?25h Created wheel for torchdata: filename=torchdata-0.7.0a0+11bb5b8-py3-none-any.whl size=182956 sha256=0e5545937095198d656a36a964a763a2982ee18c477e75b39189874c3433dc5c 2024-06-26T07:15:16.3029150Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/ca/59/a3/c8250bfc8d3d4d639498d4beb2e0f0e70b9a508ac61fde85ce 2024-06-26T07:15:16.3055231Z Successfully built torchdata 2024-06-26T07:15:17.5357618Z Installing collected packages: torchdata 2024-06-26T07:15:17.6205129Z Successfully installed torchdata-0.7.0a0+11bb5b8 2024-06-26T07:15:20.1257534Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:15:20.1259142Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:15:20.4401477Z Collecting git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:15:20.4405261Z Cloning https://github.com/pytorch/text.git (to revision b0ebddc648d279826089db91775375221777a2db) to /tmp/pip-req-build-eu5z7fd4 2024-06-26T07:15:20.4422729Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/text.git /tmp/pip-req-build-eu5z7fd4 2024-06-26T07:15:21.2489903Z Running command git rev-parse -q --verify 'sha^b0ebddc648d279826089db91775375221777a2db' 2024-06-26T07:15:21.2508599Z Running command git fetch -q https://github.com/pytorch/text.git b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:15:21.7045423Z Running command git checkout -q b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:15:21.8567782Z Resolved https://github.com/pytorch/text.git to commit b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:15:21.8568885Z Running command git submodule update --init --recursive -q 2024-06-26T07:15:28.9327405Z Preparing metadata (setup.py) ... [?25l- done 2024-06-26T07:15:28.9380004Z [?25hRequirement already satisfied: tqdm in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (4.66.4) 2024-06-26T07:15:28.9382935Z Requirement already satisfied: requests in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.32.3) 2024-06-26T07:15:28.9386069Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.5.0a0+gitb8c4c54) 2024-06-26T07:15:28.9388422Z Requirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (1.21.2) 2024-06-26T07:15:28.9392367Z Requirement already satisfied: torchdata in /var/lib/jenkins/.local/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (0.7.0a0+11bb5b8) 2024-06-26T07:15:28.9453418Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.3.2) 2024-06-26T07:15:28.9458474Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.7) 2024-06-26T07:15:28.9465468Z Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (1.26.19) 2024-06-26T07:15:28.9471175Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (2024.6.2) 2024-06-26T07:15:28.9527069Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.13.1) 2024-06-26T07:15:28.9534204Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (4.12.2) 2024-06-26T07:15:28.9537077Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (1.12.1) 2024-06-26T07:15:28.9540304Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2.8.8) 2024-06-26T07:15:28.9543606Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.1.4) 2024-06-26T07:15:28.9546803Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2024.2.0) 2024-06-26T07:15:29.0239514Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchtext==0.16.0a0+b0ebddc) (2.1.5) 2024-06-26T07:15:29.0407845Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchtext==0.16.0a0+b0ebddc) (1.3.0) 2024-06-26T07:15:29.0491846Z Building wheels for collected packages: torchtext 2024-06-26T07:16:05.4740900Z Building wheel for torchtext (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / done 2024-06-26T07:16:05.4767993Z [?25h Created wheel for torchtext: filename=torchtext-0.16.0a0+b0ebddc-cp310-cp310-linux_x86_64.whl size=2045303 sha256=69bf297552e0c5edf43a3351f8a8d720c0898695cdf4267e592bb6a5656c9a89 2024-06-26T07:16:05.4770351Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/e5/3d/d7/faafad098ec9437ecdf4495c0f5e72b817fffbb06063e32cbf 2024-06-26T07:16:05.4794485Z Successfully built torchtext 2024-06-26T07:16:06.6980006Z Installing collected packages: torchtext 2024-06-26T07:16:06.7979331Z Successfully installed torchtext-0.16.0a0+b0ebddc 2024-06-26T07:16:06.9193288Z + install_torchvision 2024-06-26T07:16:06.9193699Z + local orig_preload 2024-06-26T07:16:06.9194063Z + local commit 2024-06-26T07:16:06.9196196Z ++ get_pinned_commit vision 2024-06-26T07:16:06.9196771Z ++ cat .github/ci_commit_pins/vision.txt 2024-06-26T07:16:06.9205246Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:16:06.9205893Z + orig_preload= 2024-06-26T07:16:06.9206350Z + '[' -n '' ']' 2024-06-26T07:16:06.9207199Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:16:06.9208624Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:16:07.2381381Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:16:07.2383797Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-73acjj29 2024-06-26T07:16:07.2401015Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-73acjj29 2024-06-26T07:16:08.7226084Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-06-26T07:16:08.7245385Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:16:09.9520357Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:16:10.1859671Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:16:12.5442403Z Preparing metadata (setup.py) ... [?25l- \ done 2024-06-26T07:16:12.5490918Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2) 2024-06-26T07:16:12.5493605Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+gitb8c4c54) 2024-06-26T07:16:12.5499031Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-06-26T07:16:12.5705659Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-06-26T07:16:12.5711608Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-06-26T07:16:12.5713943Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.12.1) 2024-06-26T07:16:12.5717024Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-06-26T07:16:12.5719414Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-06-26T07:16:12.5722458Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.2.0) 2024-06-26T07:16:12.6154124Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-06-26T07:16:12.6312199Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-06-26T07:16:12.6387592Z Building wheels for collected packages: torchvision 2024-06-26T07:17:26.1270260Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2024-06-26T07:17:26.1299997Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2113069 sha256=a4a93e9d7c813435d53828277f9188d15cd4d0a8c1cf47ecbcb8a8a1666da94b 2024-06-26T07:17:26.1303278Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-06-26T07:17:26.1335234Z Successfully built torchvision 2024-06-26T07:17:27.3473252Z Installing collected packages: torchvision 2024-06-26T07:17:27.7351656Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-06-26T07:17:27.8452675Z + '[' -n '' ']' 2024-06-26T07:17:27.8453383Z + id=0 2024-06-26T07:17:27.8454449Z + pip_install opencv-python==4.8.0.74 2024-06-26T07:17:27.8455272Z + pip install --progress-bar off opencv-python==4.8.0.74 2024-06-26T07:17:28.2816109Z Collecting opencv-python==4.8.0.74 2024-06-26T07:17:28.2997950Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB) 2024-06-26T07:17:28.3199214Z Requirement already satisfied: numpy>=1.21.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opencv-python==4.8.0.74) (1.21.2) 2024-06-26T07:17:28.3267168Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.7 MB) 2024-06-26T07:17:30.3082346Z Installing collected packages: opencv-python 2024-06-26T07:17:31.1348347Z Successfully installed opencv-python-4.8.0.74 2024-06-26T07:17:31.2295749Z + [[ aot_inductor_torchbench == *inductor_torchbench_smoketest_perf* ]] 2024-06-26T07:17:31.2296549Z + [[ aot_inductor_torchbench == *inductor_torchbench_cpu_smoketest_perf* ]] 2024-06-26T07:17:31.2297245Z + [[ aot_inductor_torchbench == *torchbench_gcp_smoketest* ]] 2024-06-26T07:17:31.2297772Z + checkout_install_torchbench 2024-06-26T07:17:31.2298141Z + local commit 2024-06-26T07:17:31.2298456Z ++ get_pinned_commit torchbench 2024-06-26T07:17:31.2298892Z ++ cat .github/ci_commit_pins/torchbench.txt 2024-06-26T07:17:31.2306901Z + commit=23512dbebd44a11eb84afbf53c3c071dd105297e 2024-06-26T07:17:31.2307569Z + git clone https://github.com/pytorch/benchmark torchbench 2024-06-26T07:17:31.2316200Z Cloning into 'torchbench'... 2024-06-26T07:17:31.3644266Z remote: Enumerating objects: 30187, done. 2024-06-26T07:17:31.3644842Z remote: Counting objects: 0% (1/546) 2024-06-26T07:17:31.3645697Z remote: Counting objects: 1% (6/546) 2024-06-26T07:17:31.3646204Z remote: Counting objects: 2% (11/546) 2024-06-26T07:17:31.3646712Z remote: Counting objects: 3% (17/546) 2024-06-26T07:17:31.3647256Z remote: Counting objects: 4% (22/546) 2024-06-26T07:17:31.3647752Z remote: Counting objects: 5% (28/546) 2024-06-26T07:17:31.3648335Z remote: Counting objects: 6% (33/546) 2024-06-26T07:17:31.3648832Z remote: Counting objects: 7% (39/546) 2024-06-26T07:17:31.3652745Z remote: Counting objects: 8% (44/546) 2024-06-26T07:17:31.3653305Z remote: Counting objects: 9% (50/546) 2024-06-26T07:17:31.3653935Z remote: Counting objects: 10% (55/546) 2024-06-26T07:17:31.3654441Z remote: Counting objects: 11% (61/546) 2024-06-26T07:17:31.3654957Z remote: Counting objects: 12% (66/546) 2024-06-26T07:17:31.3655523Z remote: Counting objects: 13% (71/546) 2024-06-26T07:17:31.3656141Z remote: Counting objects: 14% (77/546) 2024-06-26T07:17:31.3656651Z remote: Counting objects: 15% (82/546) 2024-06-26T07:17:31.3657200Z remote: Counting objects: 16% (88/546) 2024-06-26T07:17:31.3657701Z remote: Counting objects: 17% (93/546) 2024-06-26T07:17:31.3658202Z remote: Counting objects: 18% (99/546) 2024-06-26T07:17:31.3658705Z remote: Counting objects: 19% (104/546) 2024-06-26T07:17:31.3659231Z remote: Counting objects: 20% (110/546) 2024-06-26T07:17:31.3659751Z remote: Counting objects: 21% (115/546) 2024-06-26T07:17:31.3660263Z remote: Counting objects: 22% (121/546) 2024-06-26T07:17:31.3660773Z remote: Counting objects: 23% (126/546) 2024-06-26T07:17:31.3661299Z remote: Counting objects: 24% (132/546) 2024-06-26T07:17:31.3661811Z remote: Counting objects: 25% (137/546) 2024-06-26T07:17:31.3662351Z remote: Counting objects: 26% (142/546) 2024-06-26T07:17:31.3662862Z remote: Counting objects: 27% (148/546) 2024-06-26T07:17:31.3663363Z remote: Counting objects: 28% (153/546) 2024-06-26T07:17:31.3663949Z remote: Counting objects: 29% (159/546) 2024-06-26T07:17:31.3664455Z remote: Counting objects: 30% (164/546) 2024-06-26T07:17:31.3664953Z remote: Counting objects: 31% (170/546) 2024-06-26T07:17:31.3665454Z remote: Counting objects: 32% (175/546) 2024-06-26T07:17:31.3665956Z remote: Counting objects: 33% (181/546) 2024-06-26T07:17:31.3666457Z remote: Counting objects: 34% (186/546) 2024-06-26T07:17:31.3667559Z remote: Counting objects: 35% (192/546) 2024-06-26T07:17:31.3668100Z remote: Counting objects: 36% (197/546) 2024-06-26T07:17:31.3668634Z remote: Counting objects: 37% (203/546) 2024-06-26T07:17:31.3669154Z remote: Counting objects: 38% (208/546) 2024-06-26T07:17:31.3669657Z remote: Counting objects: 39% (213/546) 2024-06-26T07:17:31.3670173Z remote: Counting objects: 40% (219/546) 2024-06-26T07:17:31.3670682Z remote: Counting objects: 41% (224/546) 2024-06-26T07:17:31.3671189Z remote: Counting objects: 42% (230/546) 2024-06-26T07:17:31.3671696Z remote: Counting objects: 43% (235/546) 2024-06-26T07:17:31.3672216Z remote: Counting objects: 44% (241/546) 2024-06-26T07:17:31.3672727Z remote: Counting objects: 45% (246/546) 2024-06-26T07:17:31.3673231Z remote: Counting objects: 46% (252/546) 2024-06-26T07:17:31.3673729Z remote: Counting objects: 47% (257/546) 2024-06-26T07:17:31.3674240Z remote: Counting objects: 48% (263/546) 2024-06-26T07:17:31.3674746Z remote: Counting objects: 49% (268/546) 2024-06-26T07:17:31.3675245Z remote: Counting objects: 50% (273/546) 2024-06-26T07:17:31.3675752Z remote: Counting objects: 51% (279/546) 2024-06-26T07:17:31.3676429Z remote: Counting objects: 52% (284/546) 2024-06-26T07:17:31.3676974Z remote: Counting objects: 53% (290/546) 2024-06-26T07:17:31.3677483Z remote: Counting objects: 54% (295/546) 2024-06-26T07:17:31.3677985Z remote: Counting objects: 55% (301/546) 2024-06-26T07:17:31.3678497Z remote: Counting objects: 56% (306/546) 2024-06-26T07:17:31.3679113Z remote: Counting objects: 57% (312/546) 2024-06-26T07:17:31.3679622Z remote: Counting objects: 58% (317/546) 2024-06-26T07:17:31.3680139Z remote: Counting objects: 59% (323/546) 2024-06-26T07:17:31.3680650Z remote: Counting objects: 60% (328/546) 2024-06-26T07:17:31.3681161Z remote: Counting objects: 61% (334/546) 2024-06-26T07:17:31.3681661Z remote: Counting objects: 62% (339/546) 2024-06-26T07:17:31.3682183Z remote: Counting objects: 63% (344/546) 2024-06-26T07:17:31.3682699Z remote: Counting objects: 64% (350/546) 2024-06-26T07:17:31.3683203Z remote: Counting objects: 65% (355/546) 2024-06-26T07:17:31.3683722Z remote: Counting objects: 66% (361/546) 2024-06-26T07:17:31.3684229Z remote: Counting objects: 67% (366/546) 2024-06-26T07:17:31.3684743Z remote: Counting objects: 68% (372/546) 2024-06-26T07:17:31.3685248Z remote: Counting objects: 69% (377/546) 2024-06-26T07:17:31.3685780Z remote: Counting objects: 70% (383/546) 2024-06-26T07:17:31.3686306Z remote: Counting objects: 71% (388/546) 2024-06-26T07:17:31.3686843Z remote: Counting objects: 72% (394/546) 2024-06-26T07:17:31.3687354Z remote: Counting objects: 73% (399/546) 2024-06-26T07:17:31.3687878Z remote: Counting objects: 74% (405/546) 2024-06-26T07:17:31.3688402Z remote: Counting objects: 75% (410/546) 2024-06-26T07:17:31.3688921Z remote: Counting objects: 76% (415/546) 2024-06-26T07:17:31.3689439Z remote: Counting objects: 77% (421/546) 2024-06-26T07:17:31.3689964Z remote: Counting objects: 78% (426/546) 2024-06-26T07:17:31.3690484Z remote: Counting objects: 79% (432/546) 2024-06-26T07:17:31.3691010Z remote: Counting objects: 80% (437/546) 2024-06-26T07:17:31.3691513Z remote: Counting objects: 81% (443/546) 2024-06-26T07:17:31.3692031Z remote: Counting objects: 82% (448/546) 2024-06-26T07:17:31.3692548Z remote: Counting objects: 83% (454/546) 2024-06-26T07:17:31.3693060Z remote: Counting objects: 84% (459/546) 2024-06-26T07:17:31.3693576Z remote: Counting objects: 85% (465/546) 2024-06-26T07:17:31.3694090Z remote: Counting objects: 86% (470/546) 2024-06-26T07:17:31.3694603Z remote: Counting objects: 87% (476/546) 2024-06-26T07:17:31.3695103Z remote: Counting objects: 88% (481/546) 2024-06-26T07:17:31.3695671Z remote: Counting objects: 89% (486/546) 2024-06-26T07:17:31.3696179Z remote: Counting objects: 90% (492/546) 2024-06-26T07:17:31.3696688Z remote: Counting objects: 91% (497/546) 2024-06-26T07:17:31.3697232Z remote: Counting objects: 92% (503/546) 2024-06-26T07:17:31.3697733Z remote: Counting objects: 93% (508/546) 2024-06-26T07:17:31.3698237Z remote: Counting objects: 94% (514/546) 2024-06-26T07:17:31.3698740Z remote: Counting objects: 95% (519/546) 2024-06-26T07:17:31.3699233Z remote: Counting objects: 96% (525/546) 2024-06-26T07:17:31.3699736Z remote: Counting objects: 97% (530/546) 2024-06-26T07:17:31.3700246Z remote: Counting objects: 98% (536/546) 2024-06-26T07:17:31.3700756Z remote: Counting objects: 99% (541/546) 2024-06-26T07:17:31.3701264Z remote: Counting objects: 100% (546/546) 2024-06-26T07:17:31.3701804Z remote: Counting objects: 100% (546/546), done. 2024-06-26T07:17:31.3702414Z remote: Compressing objects: 0% (1/319) 2024-06-26T07:17:31.3702939Z remote: Compressing objects: 1% (4/319) 2024-06-26T07:17:31.3703471Z remote: Compressing objects: 2% (7/319) 2024-06-26T07:17:31.3704091Z remote: Compressing objects: 3% (10/319) 2024-06-26T07:17:31.3704625Z remote: Compressing objects: 4% (13/319) 2024-06-26T07:17:31.3705154Z remote: Compressing objects: 5% (16/319) 2024-06-26T07:17:31.3705751Z remote: Compressing objects: 6% (20/319) 2024-06-26T07:17:31.3706279Z remote: Compressing objects: 7% (23/319) 2024-06-26T07:17:31.3707022Z remote: Compressing objects: 8% (26/319) 2024-06-26T07:17:31.3707746Z remote: Compressing objects: 9% (29/319) 2024-06-26T07:17:31.3708290Z remote: Compressing objects: 10% (32/319) 2024-06-26T07:17:31.3708951Z remote: Compressing objects: 11% (36/319) 2024-06-26T07:17:31.3709506Z remote: Compressing objects: 12% (39/319) 2024-06-26T07:17:31.3710029Z remote: Compressing objects: 13% (42/319) 2024-06-26T07:17:31.3710826Z remote: Compressing objects: 14% (45/319) 2024-06-26T07:17:31.3711356Z remote: Compressing objects: 15% (48/319) 2024-06-26T07:17:31.3711929Z remote: Compressing objects: 16% (52/319) 2024-06-26T07:17:31.3712456Z remote: Compressing objects: 17% (55/319) 2024-06-26T07:17:31.3713133Z remote: Compressing objects: 18% (58/319) 2024-06-26T07:17:31.3713766Z remote: Compressing objects: 19% (61/319) 2024-06-26T07:17:31.3714312Z remote: Compressing objects: 20% (64/319) 2024-06-26T07:17:31.3714828Z remote: Compressing objects: 21% (67/319) 2024-06-26T07:17:31.3715396Z remote: Compressing objects: 22% (71/319) 2024-06-26T07:17:31.3715919Z remote: Compressing objects: 23% (74/319) 2024-06-26T07:17:31.3716490Z remote: Compressing objects: 24% (77/319) 2024-06-26T07:17:31.3717017Z remote: Compressing objects: 25% (80/319) 2024-06-26T07:17:31.3717579Z remote: Compressing objects: 26% (83/319) 2024-06-26T07:17:31.3718117Z remote: Compressing objects: 27% (87/319) 2024-06-26T07:17:31.3718764Z remote: Compressing objects: 28% (90/319) 2024-06-26T07:17:31.3719324Z remote: Compressing objects: 29% (93/319) 2024-06-26T07:17:31.3719916Z remote: Compressing objects: 30% (96/319) 2024-06-26T07:17:31.3720447Z remote: Compressing objects: 31% (99/319) 2024-06-26T07:17:31.3720985Z remote: Compressing objects: 32% (103/319) 2024-06-26T07:17:31.3721541Z remote: Compressing objects: 33% (106/319) 2024-06-26T07:17:31.3722087Z remote: Compressing objects: 34% (109/319) 2024-06-26T07:17:31.3722623Z remote: Compressing objects: 35% (112/319) 2024-06-26T07:17:31.3723208Z remote: Compressing objects: 36% (115/319) 2024-06-26T07:17:31.3723810Z remote: Compressing objects: 37% (119/319) 2024-06-26T07:17:31.3724354Z remote: Compressing objects: 38% (122/319) 2024-06-26T07:17:31.3724958Z remote: Compressing objects: 39% (125/319) 2024-06-26T07:17:31.3725568Z remote: Compressing objects: 40% (128/319) 2024-06-26T07:17:31.3726204Z remote: Compressing objects: 41% (131/319) 2024-06-26T07:17:31.3726962Z remote: Compressing objects: 42% (134/319) 2024-06-26T07:17:31.3727567Z remote: Compressing objects: 43% (138/319) 2024-06-26T07:17:31.3728108Z remote: Compressing objects: 44% (141/319) 2024-06-26T07:17:31.3728643Z remote: Compressing objects: 45% (144/319) 2024-06-26T07:17:31.3729236Z remote: Compressing objects: 46% (147/319) 2024-06-26T07:17:31.3729857Z remote: Compressing objects: 47% (150/319) 2024-06-26T07:17:31.3730401Z remote: Compressing objects: 48% (154/319) 2024-06-26T07:17:31.3730966Z remote: Compressing objects: 49% (157/319) 2024-06-26T07:17:31.3731527Z remote: Compressing objects: 50% (160/319) 2024-06-26T07:17:31.3732136Z remote: Compressing objects: 51% (163/319) 2024-06-26T07:17:31.3732669Z remote: Compressing objects: 52% (166/319) 2024-06-26T07:17:31.3733259Z remote: Compressing objects: 53% (170/319) 2024-06-26T07:17:31.3733791Z remote: Compressing objects: 54% (173/319) 2024-06-26T07:17:31.3734314Z remote: Compressing objects: 55% (176/319) 2024-06-26T07:17:31.3734890Z remote: Compressing objects: 56% (179/319) 2024-06-26T07:17:31.3735490Z remote: Compressing objects: 57% (182/319) 2024-06-26T07:17:31.3736109Z remote: Compressing objects: 58% (186/319) 2024-06-26T07:17:31.3736642Z remote: Compressing objects: 59% (189/319) 2024-06-26T07:17:31.3737366Z remote: Compressing objects: 60% (192/319) 2024-06-26T07:17:31.3737897Z remote: Compressing objects: 61% (195/319) 2024-06-26T07:17:31.3738450Z remote: Compressing objects: 62% (198/319) 2024-06-26T07:17:31.3739118Z remote: Compressing objects: 63% (201/319) 2024-06-26T07:17:31.3739654Z remote: Compressing objects: 64% (205/319) 2024-06-26T07:17:31.3740262Z remote: Compressing objects: 65% (208/319) 2024-06-26T07:17:31.3740814Z remote: Compressing objects: 66% (211/319) 2024-06-26T07:17:31.3741354Z remote: Compressing objects: 67% (214/319) 2024-06-26T07:17:31.3741879Z remote: Compressing objects: 68% (217/319) 2024-06-26T07:17:31.3742462Z remote: Compressing objects: 69% (221/319) 2024-06-26T07:17:31.3743063Z remote: Compressing objects: 70% (224/319) 2024-06-26T07:17:31.3743667Z remote: Compressing objects: 71% (227/319) 2024-06-26T07:17:31.3744351Z remote: Compressing objects: 72% (230/319) 2024-06-26T07:17:31.3744923Z remote: Compressing objects: 73% (233/319) 2024-06-26T07:17:31.3745461Z remote: Compressing objects: 74% (237/319) 2024-06-26T07:17:31.3746066Z remote: Compressing objects: 75% (240/319) 2024-06-26T07:17:31.3746774Z remote: Compressing objects: 76% (243/319) 2024-06-26T07:17:31.3747366Z remote: Compressing objects: 77% (246/319) 2024-06-26T07:17:31.3748013Z remote: Compressing objects: 78% (249/319) 2024-06-26T07:17:31.3748599Z remote: Compressing objects: 79% (253/319) 2024-06-26T07:17:31.3749211Z remote: Compressing objects: 80% (256/319) 2024-06-26T07:17:31.3749732Z remote: Compressing objects: 81% (259/319) 2024-06-26T07:17:31.3750372Z remote: Compressing objects: 82% (262/319) 2024-06-26T07:17:31.3750907Z remote: Compressing objects: 83% (265/319) 2024-06-26T07:17:31.3751465Z remote: Compressing objects: 84% (268/319) 2024-06-26T07:17:31.3751990Z remote: Compressing objects: 85% (272/319) 2024-06-26T07:17:31.3752538Z remote: Compressing objects: 86% (275/319) 2024-06-26T07:17:31.3753073Z remote: Compressing objects: 87% (278/319) 2024-06-26T07:17:31.3753593Z remote: Compressing objects: 88% (281/319) 2024-06-26T07:17:31.3754116Z remote: Compressing objects: 89% (284/319) 2024-06-26T07:17:31.3754643Z remote: Compressing objects: 90% (288/319) 2024-06-26T07:17:31.3755167Z remote: Compressing objects: 91% (291/319) 2024-06-26T07:17:31.3755691Z remote: Compressing objects: 92% (294/319) 2024-06-26T07:17:31.3756215Z remote: Compressing objects: 93% (297/319) 2024-06-26T07:17:31.3756738Z remote: Compressing objects: 94% (300/319) 2024-06-26T07:17:31.3757315Z remote: Compressing objects: 95% (304/319) 2024-06-26T07:17:31.3757860Z remote: Compressing objects: 96% (307/319) 2024-06-26T07:17:31.3758403Z remote: Compressing objects: 97% (310/319) 2024-06-26T07:17:31.3758932Z remote: Compressing objects: 98% (313/319) 2024-06-26T07:17:31.3759444Z remote: Compressing objects: 99% (316/319) 2024-06-26T07:17:31.3759965Z remote: Compressing objects: 100% (319/319) 2024-06-26T07:17:31.3760532Z remote: Compressing objects: 100% (319/319), done. 2024-06-26T07:17:31.3822081Z Receiving objects: 0% (1/30187) 2024-06-26T07:17:31.3870639Z Receiving objects: 1% (302/30187) 2024-06-26T07:17:31.3913154Z Receiving objects: 2% (604/30187) 2024-06-26T07:17:31.3948965Z Receiving objects: 3% (906/30187) 2024-06-26T07:17:31.3977076Z Receiving objects: 4% (1208/30187) 2024-06-26T07:17:31.4007915Z Receiving objects: 5% (1510/30187) 2024-06-26T07:17:31.4038441Z Receiving objects: 6% (1812/30187) 2024-06-26T07:17:31.4076704Z Receiving objects: 7% (2114/30187) 2024-06-26T07:17:31.4108705Z Receiving objects: 8% (2415/30187) 2024-06-26T07:17:31.4139097Z Receiving objects: 9% (2717/30187) 2024-06-26T07:17:31.4174855Z Receiving objects: 10% (3019/30187) 2024-06-26T07:17:31.4236032Z Receiving objects: 11% (3321/30187) 2024-06-26T07:17:31.4299630Z Receiving objects: 12% (3623/30187) 2024-06-26T07:17:31.4355820Z Receiving objects: 13% (3925/30187) 2024-06-26T07:17:31.4524318Z Receiving objects: 14% (4227/30187) 2024-06-26T07:17:31.4626737Z Receiving objects: 15% (4529/30187) 2024-06-26T07:17:31.4717665Z Receiving objects: 16% (4830/30187) 2024-06-26T07:17:31.4768670Z Receiving objects: 17% (5132/30187) 2024-06-26T07:17:31.4809890Z Receiving objects: 18% (5434/30187) 2024-06-26T07:17:31.4835046Z Receiving objects: 19% (5736/30187) 2024-06-26T07:17:31.4897486Z Receiving objects: 20% (6038/30187) 2024-06-26T07:17:31.4916600Z Receiving objects: 21% (6340/30187) 2024-06-26T07:17:31.4936111Z Receiving objects: 22% (6642/30187) 2024-06-26T07:17:31.5994115Z Receiving objects: 23% (6944/30187) 2024-06-26T07:17:32.3767834Z Receiving objects: 24% (7245/30187) 2024-06-26T07:17:32.5604509Z Receiving objects: 24% (7426/30187), 73.12 MiB | 73.11 MiB/s 2024-06-26T07:17:33.0543938Z Receiving objects: 25% (7547/30187), 73.12 MiB | 73.11 MiB/s 2024-06-26T07:17:33.1308317Z Receiving objects: 26% (7849/30187), 113.30 MiB | 75.58 MiB/s 2024-06-26T07:17:33.1977310Z Receiving objects: 27% (8151/30187), 113.30 MiB | 75.58 MiB/s 2024-06-26T07:17:33.2631285Z Receiving objects: 28% (8453/30187), 113.30 MiB | 75.58 MiB/s 2024-06-26T07:17:33.3287807Z Receiving objects: 29% (8755/30187), 113.30 MiB | 75.58 MiB/s 2024-06-26T07:17:33.3766764Z Receiving objects: 30% (9057/30187), 113.30 MiB | 75.58 MiB/s 2024-06-26T07:17:33.3944120Z Receiving objects: 30% (9279/30187), 152.43 MiB | 76.29 MiB/s 2024-06-26T07:17:33.4601180Z Receiving objects: 31% (9358/30187), 152.43 MiB | 76.29 MiB/s 2024-06-26T07:17:33.5264731Z Receiving objects: 32% (9660/30187), 152.43 MiB | 76.29 MiB/s 2024-06-26T07:17:33.7471257Z Receiving objects: 33% (9962/30187), 152.43 MiB | 76.29 MiB/s 2024-06-26T07:17:33.8192679Z Receiving objects: 34% (10264/30187), 152.43 MiB | 76.29 MiB/s 2024-06-26T07:17:33.8314314Z Receiving objects: 35% (10566/30187), 152.43 MiB | 76.29 MiB/s 2024-06-26T07:17:34.0484517Z Receiving objects: 36% (10868/30187), 152.43 MiB | 76.29 MiB/s 2024-06-26T07:17:34.2955554Z Receiving objects: 37% (11170/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3157799Z Receiving objects: 38% (11472/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3170913Z Receiving objects: 39% (11773/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3195973Z Receiving objects: 40% (12075/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3210095Z Receiving objects: 41% (12377/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3223088Z Receiving objects: 42% (12679/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3234302Z Receiving objects: 43% (12981/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3243787Z Receiving objects: 44% (13283/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3262667Z Receiving objects: 45% (13585/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3274641Z Receiving objects: 46% (13887/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3279384Z Receiving objects: 47% (14188/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3285856Z Receiving objects: 48% (14490/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3309368Z Receiving objects: 49% (14792/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3325662Z Receiving objects: 50% (15094/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3335596Z Receiving objects: 51% (15396/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3532042Z Receiving objects: 52% (15698/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3536807Z Receiving objects: 53% (16000/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3544424Z Receiving objects: 54% (16301/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3551259Z Receiving objects: 55% (16603/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3556711Z Receiving objects: 56% (16905/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3573742Z Receiving objects: 57% (17207/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3585835Z Receiving objects: 58% (17509/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3595100Z Receiving objects: 59% (17811/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3609182Z Receiving objects: 60% (18113/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3619050Z Receiving objects: 61% (18415/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3629472Z Receiving objects: 62% (18716/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3756723Z Receiving objects: 63% (19018/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3764166Z Receiving objects: 64% (19320/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3765022Z Receiving objects: 65% (19622/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3770949Z Receiving objects: 65% (19628/30187), 185.45 MiB | 74.24 MiB/s 2024-06-26T07:17:34.3777296Z Receiving objects: 66% (19924/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.3787020Z Receiving objects: 67% (20226/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.3795057Z Receiving objects: 68% (20528/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.4036717Z Receiving objects: 69% (20830/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.4422355Z Receiving objects: 70% (21131/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.4429606Z Receiving objects: 71% (21433/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.4442735Z Receiving objects: 72% (21735/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.4455346Z Receiving objects: 73% (22037/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.4468605Z Receiving objects: 74% (22339/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.4479581Z Receiving objects: 75% (22641/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.4487547Z Receiving objects: 76% (22943/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.4506806Z Receiving objects: 77% (23244/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.4513444Z Receiving objects: 78% (23546/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.4521994Z Receiving objects: 79% (23848/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.4543962Z Receiving objects: 80% (24150/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6477310Z Receiving objects: 81% (24452/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6686181Z Receiving objects: 82% (24754/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6719097Z Receiving objects: 83% (25056/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6730169Z Receiving objects: 84% (25358/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6742086Z Receiving objects: 85% (25659/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6772151Z Receiving objects: 86% (25961/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6801051Z Receiving objects: 87% (26263/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6928235Z Receiving objects: 88% (26565/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6934653Z Receiving objects: 89% (26867/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6943286Z Receiving objects: 90% (27169/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6949592Z Receiving objects: 91% (27471/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6956718Z Receiving objects: 92% (27773/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.6963136Z Receiving objects: 93% (28074/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.7073935Z Receiving objects: 94% (28376/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.7088884Z Receiving objects: 95% (28678/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.7099708Z Receiving objects: 96% (28980/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.7118105Z Receiving objects: 97% (29282/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.7133607Z Receiving objects: 98% (29584/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.7173628Z Receiving objects: 99% (29886/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.7174745Z remote: Total 30187 (delta 255), reused 486 (delta 215), pack-reused 29641 2024-06-26T07:17:34.7192526Z Receiving objects: 100% (30187/30187), 225.06 MiB | 75.07 MiB/s 2024-06-26T07:17:34.7193756Z Receiving objects: 100% (30187/30187), 245.25 MiB | 73.39 MiB/s, done. 2024-06-26T07:17:34.7219686Z Resolving deltas: 0% (0/16204) 2024-06-26T07:17:34.7233095Z Resolving deltas: 1% (164/16204) 2024-06-26T07:17:34.7242854Z Resolving deltas: 2% (407/16204) 2024-06-26T07:17:34.7257563Z Resolving deltas: 3% (507/16204) 2024-06-26T07:17:34.7278737Z Resolving deltas: 4% (650/16204) 2024-06-26T07:17:34.7288390Z Resolving deltas: 5% (823/16204) 2024-06-26T07:17:34.7296886Z Resolving deltas: 6% (974/16204) 2024-06-26T07:17:34.7306981Z Resolving deltas: 7% (1143/16204) 2024-06-26T07:17:34.7316066Z Resolving deltas: 8% (1299/16204) 2024-06-26T07:17:34.7324238Z Resolving deltas: 9% (1475/16204) 2024-06-26T07:17:34.7335415Z Resolving deltas: 10% (1623/16204) 2024-06-26T07:17:34.7345937Z Resolving deltas: 11% (1783/16204) 2024-06-26T07:17:34.7354833Z Resolving deltas: 12% (1945/16204) 2024-06-26T07:17:34.7367395Z Resolving deltas: 13% (2110/16204) 2024-06-26T07:17:34.7376293Z Resolving deltas: 14% (2270/16204) 2024-06-26T07:17:34.7395014Z Resolving deltas: 15% (2438/16204) 2024-06-26T07:17:34.7410141Z Resolving deltas: 16% (2619/16204) 2024-06-26T07:17:34.7424957Z Resolving deltas: 17% (2759/16204) 2024-06-26T07:17:34.7454532Z Resolving deltas: 18% (2920/16204) 2024-06-26T07:17:34.7481349Z Resolving deltas: 19% (3089/16204) 2024-06-26T07:17:34.7499558Z Resolving deltas: 20% (3241/16204) 2024-06-26T07:17:34.7501900Z Resolving deltas: 21% (3518/16204) 2024-06-26T07:17:34.7512993Z Resolving deltas: 22% (3571/16204) 2024-06-26T07:17:34.7524385Z Resolving deltas: 23% (3776/16204) 2024-06-26T07:17:34.7528791Z Resolving deltas: 24% (4003/16204) 2024-06-26T07:17:34.7536705Z Resolving deltas: 25% (4090/16204) 2024-06-26T07:17:34.7579032Z Resolving deltas: 26% (4215/16204) 2024-06-26T07:17:34.7594331Z Resolving deltas: 27% (4394/16204) 2024-06-26T07:17:34.7606361Z Resolving deltas: 28% (4559/16204) 2024-06-26T07:17:34.7619067Z Resolving deltas: 29% (4702/16204) 2024-06-26T07:17:34.7636562Z Resolving deltas: 30% (4867/16204) 2024-06-26T07:17:34.7647996Z Resolving deltas: 31% (5066/16204) 2024-06-26T07:17:34.7656679Z Resolving deltas: 32% (5200/16204) 2024-06-26T07:17:34.7666235Z Resolving deltas: 33% (5373/16204) 2024-06-26T07:17:34.7676535Z Resolving deltas: 34% (5517/16204) 2024-06-26T07:17:34.7695737Z Resolving deltas: 35% (5674/16204) 2024-06-26T07:17:34.7711352Z Resolving deltas: 36% (5836/16204) 2024-06-26T07:17:34.7729958Z Resolving deltas: 37% (5999/16204) 2024-06-26T07:17:34.7736037Z Resolving deltas: 38% (6190/16204) 2024-06-26T07:17:34.7756763Z Resolving deltas: 39% (6329/16204) 2024-06-26T07:17:34.7767941Z Resolving deltas: 40% (6491/16204) 2024-06-26T07:17:34.7774836Z Resolving deltas: 41% (6690/16204) 2024-06-26T07:17:34.7795370Z Resolving deltas: 42% (6831/16204) 2024-06-26T07:17:34.7801164Z Resolving deltas: 43% (7017/16204) 2024-06-26T07:17:34.7810607Z Resolving deltas: 44% (7145/16204) 2024-06-26T07:17:34.7819576Z Resolving deltas: 45% (7330/16204) 2024-06-26T07:17:34.7841952Z Resolving deltas: 46% (7467/16204) 2024-06-26T07:17:34.7858023Z Resolving deltas: 47% (7624/16204) 2024-06-26T07:17:34.7871020Z Resolving deltas: 48% (7778/16204) 2024-06-26T07:17:34.7896086Z Resolving deltas: 49% (7940/16204) 2024-06-26T07:17:34.7903632Z Resolving deltas: 50% (8116/16204) 2024-06-26T07:17:34.7910320Z Resolving deltas: 51% (8265/16204) 2024-06-26T07:17:34.7919106Z Resolving deltas: 52% (8431/16204) 2024-06-26T07:17:34.7938414Z Resolving deltas: 53% (8594/16204) 2024-06-26T07:17:34.7938831Z Resolving deltas: 54% (8904/16204) 2024-06-26T07:17:34.7946922Z Resolving deltas: 55% (8913/16204) 2024-06-26T07:17:34.7953285Z Resolving deltas: 56% (9108/16204) 2024-06-26T07:17:34.7967000Z Resolving deltas: 57% (9240/16204) 2024-06-26T07:17:34.7973971Z Resolving deltas: 58% (9403/16204) 2024-06-26T07:17:34.7982905Z Resolving deltas: 59% (9613/16204) 2024-06-26T07:17:34.7984853Z Resolving deltas: 60% (9883/16204) 2024-06-26T07:17:34.7990301Z Resolving deltas: 61% (9909/16204) 2024-06-26T07:17:34.8002020Z Resolving deltas: 62% (10052/16204) 2024-06-26T07:17:34.8013212Z Resolving deltas: 63% (10336/16204) 2024-06-26T07:17:34.8019146Z Resolving deltas: 65% (10554/16204) 2024-06-26T07:17:34.8024284Z Resolving deltas: 66% (10718/16204) 2024-06-26T07:17:34.8043737Z Resolving deltas: 67% (10874/16204) 2024-06-26T07:17:34.8048515Z Resolving deltas: 68% (11169/16204) 2024-06-26T07:17:34.8055870Z Resolving deltas: 69% (11287/16204) 2024-06-26T07:17:34.8072450Z Resolving deltas: 70% (11350/16204) 2024-06-26T07:17:34.8077747Z Resolving deltas: 71% (11514/16204) 2024-06-26T07:17:34.8089318Z Resolving deltas: 72% (11668/16204) 2024-06-26T07:17:34.8103657Z Resolving deltas: 73% (11829/16204) 2024-06-26T07:17:34.8117326Z Resolving deltas: 74% (11992/16204) 2024-06-26T07:17:34.8150453Z Resolving deltas: 75% (12171/16204) 2024-06-26T07:17:34.8153779Z Resolving deltas: 78% (12710/16204) 2024-06-26T07:17:34.8166307Z Resolving deltas: 79% (12803/16204) 2024-06-26T07:17:34.8176944Z Resolving deltas: 80% (12969/16204) 2024-06-26T07:17:34.8189539Z Resolving deltas: 81% (13135/16204) 2024-06-26T07:17:34.8196677Z Resolving deltas: 82% (13308/16204) 2024-06-26T07:17:34.8204466Z Resolving deltas: 83% (13450/16204) 2024-06-26T07:17:34.8215111Z Resolving deltas: 84% (13619/16204) 2024-06-26T07:17:34.8235575Z Resolving deltas: 85% (13775/16204) 2024-06-26T07:17:34.8245510Z Resolving deltas: 86% (13936/16204) 2024-06-26T07:17:34.8253104Z Resolving deltas: 87% (14098/16204) 2024-06-26T07:17:34.8260492Z Resolving deltas: 88% (14271/16204) 2024-06-26T07:17:34.8268307Z Resolving deltas: 89% (14437/16204) 2024-06-26T07:17:34.8277993Z Resolving deltas: 90% (14597/16204) 2024-06-26T07:17:34.8285272Z Resolving deltas: 91% (14763/16204) 2024-06-26T07:17:34.8297386Z Resolving deltas: 92% (14914/16204) 2024-06-26T07:17:34.8301747Z Resolving deltas: 93% (15136/16204) 2024-06-26T07:17:34.8310117Z Resolving deltas: 94% (15246/16204) 2024-06-26T07:17:34.8317375Z Resolving deltas: 95% (15401/16204) 2024-06-26T07:17:34.8322912Z Resolving deltas: 96% (15556/16204) 2024-06-26T07:17:34.8328164Z Resolving deltas: 97% (15718/16204) 2024-06-26T07:17:34.8336589Z Resolving deltas: 98% (15880/16204) 2024-06-26T07:17:34.8368299Z Resolving deltas: 99% (16049/16204) 2024-06-26T07:17:34.8368727Z Resolving deltas: 100% (16204/16204) 2024-06-26T07:17:34.8369168Z Resolving deltas: 100% (16204/16204), done. 2024-06-26T07:17:36.4398491Z + pushd torchbench 2024-06-26T07:17:36.4398906Z ~/workspace/torchbench ~/workspace 2024-06-26T07:17:36.4399427Z + git checkout 23512dbebd44a11eb84afbf53c3c071dd105297e 2024-06-26T07:17:36.5024642Z Note: switching to '23512dbebd44a11eb84afbf53c3c071dd105297e'. 2024-06-26T07:17:36.5025156Z 2024-06-26T07:17:36.5025685Z You are in 'detached HEAD' state. You can look around, make experimental 2024-06-26T07:17:36.5026872Z changes and commit them, and you can discard any commits you make in this 2024-06-26T07:17:36.5027810Z state without impacting any branches by switching back to a branch. 2024-06-26T07:17:36.5028356Z 2024-06-26T07:17:36.5028681Z If you want to create a new branch to retain commits you create, you may 2024-06-26T07:17:36.5029471Z do so (now or later) by using -c with the switch command. Example: 2024-06-26T07:17:36.5029904Z 2024-06-26T07:17:36.5030082Z git switch -c 2024-06-26T07:17:36.5030359Z 2024-06-26T07:17:36.5030502Z Or undo this operation with: 2024-06-26T07:17:36.5030757Z 2024-06-26T07:17:36.5030882Z git switch - 2024-06-26T07:17:36.5031059Z 2024-06-26T07:17:36.5031394Z Turn off this advice by setting config variable advice.detachedHead to false 2024-06-26T07:17:36.5031890Z 2024-06-26T07:17:36.5032054Z HEAD is now at 23512dbe Pin numpy version 2024-06-26T07:17:36.5032505Z + '[' '' ']' 2024-06-26T07:17:36.5032867Z + python install.py --continue_on_fail 2024-06-26T07:17:38.5035478Z checking packages torch, torchvision, torchaudio are installed...OK 2024-06-26T07:17:54.3932494Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch...OK 2024-06-26T07:17:56.8274183Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/Background_Matting...OK 2024-06-26T07:17:59.4468793Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/LearningToPaint...OK 2024-06-26T07:18:02.0430287Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/Super_SloMo...OK 2024-06-26T07:18:02.0586140Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/alexnet...OK 2024-06-26T07:18:11.0004101Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_edgecnn...OK 2024-06-26T07:18:14.3931682Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_gcn...OK 2024-06-26T07:18:17.8000259Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_gin...OK 2024-06-26T07:18:21.1956624Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_sage...OK 2024-06-26T07:18:21.1958659Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/cm3leon_generate...SKIP - No install.py is found 2024-06-26T07:18:23.0427333Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/dcgan...OK 2024-06-26T07:18:26.6896397Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/demucs...OK 2024-06-26T07:18:26.7048177Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/densenet121...OK 2024-06-26T07:19:21.4428001Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_c4...OK 2024-06-26T07:19:30.3326898Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_dc5...OK 2024-06-26T07:19:38.2598511Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_fpn...OK 2024-06-26T07:19:45.6301970Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_c4...OK 2024-06-26T07:19:54.2394405Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_dc5...OK 2024-06-26T07:20:01.8845868Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_fpn...OK 2024-06-26T07:20:09.0425961Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fcos_r_50_fpn...OK 2024-06-26T07:20:16.5818640Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn...OK 2024-06-26T07:20:24.3465971Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_101_c4...OK 2024-06-26T07:20:32.0234938Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_101_fpn...OK 2024-06-26T07:20:39.6487496Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_50_c4...OK 2024-06-26T07:20:47.1081061Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_50_fpn...OK 2024-06-26T07:20:51.2728727Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/dlrm...OK 2024-06-26T07:21:32.7779318Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/doctr_det_predictor...OK 2024-06-26T07:21:39.5316422Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/doctr_reco_predictor...OK 2024-06-26T07:21:46.4351106Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/drq...OK 2024-06-26T07:21:57.4604746Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/fastNLP_Bert...OK 2024-06-26T07:21:59.7087370Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/functorch_dp_cifar10...OK 2024-06-26T07:22:02.2131932Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/functorch_maml_omniglot...OK 2024-06-26T07:22:10.4249223Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Albert...OK 2024-06-26T07:22:18.5776295Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bart...OK 2024-06-26T07:22:25.3373333Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert...OK 2024-06-26T07:22:34.0582499Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert_large...OK 2024-06-26T07:22:41.1984399Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_BigBird...OK 2024-06-26T07:22:47.5362662Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_DistilBert...OK 2024-06-26T07:22:54.9101063Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_GPT2...OK 2024-06-26T07:23:11.3201288Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_GPT2_large...OK 2024-06-26T07:23:18.7909828Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Longformer...OK 2024-06-26T07:23:24.1768522Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Reformer...OK 2024-06-26T07:23:30.3399745Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5...OK 2024-06-26T07:23:38.2337687Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_base...OK 2024-06-26T07:23:38.2345079Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_generate...SKIP - No install.py is found 2024-06-26T07:23:51.6156713Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_large...OK 2024-06-26T07:23:56.9542582Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Whisper...OK 2024-06-26T07:23:56.9544888Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_clip...SKIP - No install.py is found 2024-06-26T07:24:05.6022116Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_distil_whisper...OK 2024-06-26T07:24:07.9243014Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/lennard_jones...OK 2024-06-26T07:24:10.2772737Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama...OK 2024-06-26T07:24:49.9697202Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama_v2_7b_16h...OK 2024-06-26T07:26:10.2779265Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llava...OK 2024-06-26T07:26:10.2941332Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/maml...OK 2024-06-26T07:26:12.9013608Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/maml_omniglot...OK 2024-06-26T07:26:12.9172982Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mnasnet1_0...OK 2024-06-26T07:26:12.9329947Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v2...OK 2024-06-26T07:26:12.9481200Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v2_quantized_qat...OK 2024-06-26T07:26:12.9631066Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v3_large...OK 2024-06-26T07:26:12.9778997Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moco...OK 2024-06-26T07:26:33.4920813Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moondream...OK 2024-06-26T07:26:33.4922423Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nanogpt...SKIP - No install.py is found 2024-06-26T07:26:35.8429557Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nvidia_deeprecommender...OK 2024-06-26T07:26:39.2778368Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/opacus_cifar10...OK 2024-06-26T07:26:39.2936833Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/phlippe_densenet...OK 2024-06-26T07:26:39.3100846Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/phlippe_resnet...OK 2024-06-26T07:26:39.3254610Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_equation_of_state...OK 2024-06-26T07:26:39.3414217Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_isoneutral_mixing...OK 2024-06-26T07:26:39.3570893Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_turbulent_kinetic_energy...OK 2024-06-26T07:26:46.6993524Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_CycleGAN_and_pix2pix...OK 2024-06-26T07:26:49.3464719Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_stargan...OK 2024-06-26T07:26:54.0873792Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_unet...OK 2024-06-26T07:26:54.1033910Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet152...OK 2024-06-26T07:26:54.1192018Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet18...OK 2024-06-26T07:26:54.1348557Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet50...OK 2024-06-26T07:26:54.1503486Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet50_quantized_qat...OK 2024-06-26T07:26:54.1656022Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnext50_32x4d...OK 2024-06-26T07:27:04.4941607Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/sam...OK 2024-06-26T07:27:16.5930755Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/sam_fast...OK 2024-06-26T07:27:16.6088954Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/shufflenet_v2_x1_0...OK 2024-06-26T07:27:16.6094229Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/simple_gpt...SKIP - No install.py is found 2024-06-26T07:27:16.6098125Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/simple_gpt_tp_manual...SKIP - No install.py is found 2024-06-26T07:27:20.3880654Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/soft_actor_critic...OK 2024-06-26T07:27:23.2940868Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/speech_transformer...OK 2024-06-26T07:27:23.3095863Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/squeezenet1_1...OK 2024-06-26T07:27:43.0402713Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/stable_diffusion_text_encoder...OK 2024-06-26T07:27:49.9824703Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/stable_diffusion_unet...OK 2024-06-26T07:27:54.4111697Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tacotron2...OK 2024-06-26T07:28:02.0634103Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_efficientdet...OK 2024-06-26T07:28:02.0787690Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_efficientnet...OK 2024-06-26T07:28:02.0936981Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_nfnet...OK 2024-06-26T07:28:02.1103387Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_regnet...OK 2024-06-26T07:28:02.1258483Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_resnest...OK 2024-06-26T07:28:02.1407730Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer...OK 2024-06-26T07:28:02.1559731Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer_large...OK 2024-06-26T07:28:02.1706249Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vovnet...OK 2024-06-26T07:28:13.8613125Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/torch_multimodal_clip...OK 2024-06-26T07:28:25.8225083Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tts_angular...OK 2024-06-26T07:28:25.8380873Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/vgg16...OK 2024-06-26T07:28:28.8776534Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/vision_maskrcnn...OK 2024-06-26T07:28:32.1571074Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/yolov3...OK 2024-06-26T07:28:32.5846746Z + popd 2024-06-26T07:28:32.5847056Z ~/workspace 2024-06-26T07:28:32.5847426Z + [[ aot_inductor_torchbench != *cpu_inductor* ]] 2024-06-26T07:28:32.5847947Z + install_torchrec_and_fbgemm 2024-06-26T07:28:32.5848435Z + local torchrec_commit 2024-06-26T07:28:32.5849461Z ++ get_pinned_commit torchrec 2024-06-26T07:28:32.5850054Z ++ cat .github/ci_commit_pins/torchrec.txt 2024-06-26T07:28:32.6020597Z + torchrec_commit=6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-06-26T07:28:32.6021278Z + local fbgemm_commit 2024-06-26T07:28:32.6021943Z ++ get_pinned_commit fbgemm 2024-06-26T07:28:32.6022349Z ++ cat .github/ci_commit_pins/fbgemm.txt 2024-06-26T07:28:32.6030536Z + fbgemm_commit=de731af65b4f04696e85c729e3282450b51b95fd 2024-06-26T07:28:32.6031217Z + pip_uninstall torchrec-nightly 2024-06-26T07:28:32.6031682Z + pip uninstall -y torchrec-nightly 2024-06-26T07:28:32.9722470Z WARNING: Skipping torchrec-nightly as it is not installed. 2024-06-26T07:28:33.0140480Z + pip_uninstall fbgemm-gpu-nightly 2024-06-26T07:28:33.0141056Z + pip uninstall -y fbgemm-gpu-nightly 2024-06-26T07:28:33.3651292Z WARNING: Skipping fbgemm-gpu-nightly as it is not installed. 2024-06-26T07:28:33.4075721Z + pip_install setuptools-git-versioning scikit-build pyre-extensions 2024-06-26T07:28:33.4076761Z + pip install --progress-bar off setuptools-git-versioning scikit-build pyre-extensions 2024-06-26T07:28:33.8336553Z Collecting setuptools-git-versioning 2024-06-26T07:28:33.8507836Z Downloading setuptools_git_versioning-2.0.0-py3-none-any.whl.metadata (5.8 kB) 2024-06-26T07:28:33.9620112Z Collecting scikit-build 2024-06-26T07:28:33.9650530Z Downloading scikit_build-0.18.0-py3-none-any.whl.metadata (17 kB) 2024-06-26T07:28:34.0826386Z Collecting pyre-extensions 2024-06-26T07:28:34.0856577Z Downloading pyre_extensions-0.0.30-py3-none-any.whl.metadata (4.0 kB) 2024-06-26T07:28:34.1499589Z Requirement already satisfied: packaging in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from setuptools-git-versioning) (24.1) 2024-06-26T07:28:34.1502502Z Requirement already satisfied: setuptools in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from setuptools-git-versioning) (69.5.1) 2024-06-26T07:28:34.2160145Z Collecting toml>=0.10.2 (from setuptools-git-versioning) 2024-06-26T07:28:34.2192297Z Downloading toml-0.10.2-py2.py3-none-any.whl.metadata (7.1 kB) 2024-06-26T07:28:34.2953558Z Requirement already satisfied: distro in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from scikit-build) (1.9.0) 2024-06-26T07:28:34.2964635Z Requirement already satisfied: tomli in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from scikit-build) (2.0.1) 2024-06-26T07:28:34.2969819Z Requirement already satisfied: wheel>=0.32.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from scikit-build) (0.43.0) 2024-06-26T07:28:34.3583940Z Collecting typing-inspect (from pyre-extensions) 2024-06-26T07:28:34.3620042Z Downloading typing_inspect-0.9.0-py3-none-any.whl.metadata (1.5 kB) 2024-06-26T07:28:34.4263867Z Requirement already satisfied: typing-extensions in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pyre-extensions) (4.12.2) 2024-06-26T07:28:34.4492321Z Requirement already satisfied: mypy-extensions>=0.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from typing-inspect->pyre-extensions) (1.0.0) 2024-06-26T07:28:34.4543077Z Downloading setuptools_git_versioning-2.0.0-py3-none-any.whl (10 kB) 2024-06-26T07:28:34.5342595Z Downloading scikit_build-0.18.0-py3-none-any.whl (85 kB) 2024-06-26T07:28:34.6072113Z Downloading pyre_extensions-0.0.30-py3-none-any.whl (12 kB) 2024-06-26T07:28:34.6914077Z Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) 2024-06-26T07:28:34.7706835Z Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB) 2024-06-26T07:28:37.0210083Z Installing collected packages: typing-inspect, toml, scikit-build, setuptools-git-versioning, pyre-extensions 2024-06-26T07:28:37.3351973Z Successfully installed pyre-extensions-0.0.30 scikit-build-0.18.0 setuptools-git-versioning-2.0.0 toml-0.10.2 typing-inspect-0.9.0 2024-06-26T07:28:37.4556333Z + CUDA_PATH=/usr/local/cuda-12.1 2024-06-26T07:28:37.4557748Z + pip_install --no-use-pep517 --user 'git+https://github.com/pytorch/FBGEMM.git@de731af65b4f04696e85c729e3282450b51b95fd#egg=fbgemm-gpu&subdirectory=fbgemm_gpu' 2024-06-26T07:28:37.4559680Z + pip install --progress-bar off --no-use-pep517 --user 'git+https://github.com/pytorch/FBGEMM.git@de731af65b4f04696e85c729e3282450b51b95fd#egg=fbgemm-gpu&subdirectory=fbgemm_gpu' 2024-06-26T07:28:37.7984438Z Collecting fbgemm-gpu 2024-06-26T07:28:37.7986295Z Cloning https://github.com/pytorch/FBGEMM.git (to revision de731af65b4f04696e85c729e3282450b51b95fd) to /tmp/pip-install-drihwljw/fbgemm-gpu_71ea2563f2724e69aca5fb6f1f176b01 2024-06-26T07:28:37.8003317Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/FBGEMM.git /tmp/pip-install-drihwljw/fbgemm-gpu_71ea2563f2724e69aca5fb6f1f176b01 2024-06-26T07:28:38.8445911Z Running command git rev-parse -q --verify 'sha^de731af65b4f04696e85c729e3282450b51b95fd' 2024-06-26T07:28:38.8461003Z Running command git fetch -q https://github.com/pytorch/FBGEMM.git de731af65b4f04696e85c729e3282450b51b95fd 2024-06-26T07:28:39.2443002Z Running command git checkout -q de731af65b4f04696e85c729e3282450b51b95fd 2024-06-26T07:28:39.8358763Z Resolved https://github.com/pytorch/FBGEMM.git to commit de731af65b4f04696e85c729e3282450b51b95fd 2024-06-26T07:28:39.8359853Z Running command git submodule update --init --recursive -q 2024-06-26T07:28:47.7241881Z Preparing metadata (setup.py) ... [?25l- done 2024-06-26T07:28:47.7262945Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbgemm-gpu) (1.21.2) 2024-06-26T07:28:47.7269565Z Building wheels for collected packages: fbgemm-gpu 2024-06-26T08:10:50.0515588Z Building wheel for fbgemm-gpu (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done 2024-06-26T08:10:50.2330993Z [?25h Created wheel for fbgemm-gpu: filename=fbgemm_gpu-0.4.1rc0.post421-cp310-cp310-linux_x86_64.whl size=252634486 sha256=9adad20a73b7a3d9bc0b649a30152c01bc2f6917caa0d6849e71c880b87d0ea2 2024-06-26T08:10:50.2333068Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/d8/d9/f3/f5c260aba9c9f0a533444c12621a0c2ceb8fbcffce2b5beb39 2024-06-26T08:10:50.2351843Z Successfully built fbgemm-gpu 2024-06-26T08:10:52.4199092Z Installing collected packages: fbgemm-gpu 2024-06-26T08:10:55.5773471Z Successfully installed fbgemm-gpu-0.4.1rc0.post421 2024-06-26T08:10:56.2017894Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/torchrec.git@6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-06-26T08:10:56.2020310Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/torchrec.git@6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-06-26T08:10:56.5457037Z Collecting git+https://github.com/pytorch/torchrec.git@6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-06-26T08:10:56.5460199Z Cloning https://github.com/pytorch/torchrec.git (to revision 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2) to /tmp/pip-req-build-7s46uhfs 2024-06-26T08:10:56.5477950Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/torchrec.git /tmp/pip-req-build-7s46uhfs 2024-06-26T08:10:57.1888423Z Running command git rev-parse -q --verify 'sha^6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2' 2024-06-26T08:10:57.1906768Z Running command git fetch -q https://github.com/pytorch/torchrec.git 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-06-26T08:10:57.4641860Z Running command git checkout -q 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-06-26T08:10:57.7832209Z Resolved https://github.com/pytorch/torchrec.git to commit 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-06-26T08:10:58.0196892Z Preparing metadata (setup.py) ... [?25l- done 2024-06-26T08:10:58.0245957Z [?25hRequirement already satisfied: iopath in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (0.1.9) 2024-06-26T08:10:58.0249929Z Requirement already satisfied: pandas in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (2.0.3) 2024-06-26T08:10:58.0253889Z Requirement already satisfied: tabulate in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (0.9.0) 2024-06-26T08:10:58.0791998Z Collecting torchmetrics (from torchrec==0.3.2) 2024-06-26T08:10:58.0977458Z Downloading torchmetrics-1.4.0.post0-py3-none-any.whl.metadata (19 kB) 2024-06-26T08:10:58.1051453Z Requirement already satisfied: tqdm in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (4.66.4) 2024-06-26T08:10:58.1054545Z Requirement already satisfied: fbgemm-gpu in /var/lib/jenkins/.local/lib/python3.10/site-packages (from torchrec==0.3.2) (0.4.1rc0.post421) 2024-06-26T08:10:58.1067159Z Requirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbgemm-gpu->torchrec==0.3.2) (1.21.2) 2024-06-26T08:10:58.1088851Z Requirement already satisfied: portalocker in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from iopath->torchrec==0.3.2) (2.10.0) 2024-06-26T08:10:58.1776013Z Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pandas->torchrec==0.3.2) (2.9.0.post0) 2024-06-26T08:10:58.1780769Z Requirement already satisfied: pytz>=2020.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pandas->torchrec==0.3.2) (2024.1) 2024-06-26T08:10:58.1786267Z Requirement already satisfied: tzdata>=2022.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pandas->torchrec==0.3.2) (2024.1) 2024-06-26T08:10:58.2592653Z Requirement already satisfied: packaging>17.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchmetrics->torchrec==0.3.2) (24.1) 2024-06-26T08:10:58.2597821Z Requirement already satisfied: torch>=1.10.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchmetrics->torchrec==0.3.2) (2.5.0a0+gitb8c4c54) 2024-06-26T08:10:58.2799842Z Collecting lightning-utilities>=0.8.0 (from torchmetrics->torchrec==0.3.2) 2024-06-26T08:10:58.2832584Z Downloading lightning_utilities-0.11.2-py3-none-any.whl.metadata (4.7 kB) 2024-06-26T08:10:58.3007818Z Requirement already satisfied: setuptools in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from lightning-utilities>=0.8.0->torchmetrics->torchrec==0.3.2) (69.5.1) 2024-06-26T08:10:58.3010336Z Requirement already satisfied: typing-extensions in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from lightning-utilities>=0.8.0->torchmetrics->torchrec==0.3.2) (4.12.2) 2024-06-26T08:10:58.3043577Z Requirement already satisfied: six>=1.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->torchrec==0.3.2) (1.16.0) 2024-06-26T08:10:58.3115352Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (3.13.1) 2024-06-26T08:10:58.3119094Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (1.12.1) 2024-06-26T08:10:58.3122294Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (2.8.8) 2024-06-26T08:10:58.3125285Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (3.1.4) 2024-06-26T08:10:58.3128365Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (2024.2.0) 2024-06-26T08:10:58.3807989Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch>=1.10.0->torchmetrics->torchrec==0.3.2) (2.1.5) 2024-06-26T08:10:58.4122452Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch>=1.10.0->torchmetrics->torchrec==0.3.2) (1.3.0) 2024-06-26T08:10:58.4273599Z Downloading torchmetrics-1.4.0.post0-py3-none-any.whl (868 kB) 2024-06-26T08:10:58.4779699Z Downloading lightning_utilities-0.11.2-py3-none-any.whl (26 kB) 2024-06-26T08:10:58.4909705Z Building wheels for collected packages: torchrec 2024-06-26T08:10:58.9278158Z Building wheel for torchrec (setup.py) ... [?25l- \ | done 2024-06-26T08:10:58.9287225Z [?25h Created wheel for torchrec: filename=torchrec-0.3.2-py3-none-any.whl size=374488 sha256=ad0ed813fe16fdc3834443546df84aacdf8fbe1d354fb703b5d9989dfc2cf65c 2024-06-26T08:10:58.9288839Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/8e/a1/47/39ede01672ba82c08fb521bfc057cc4347e4b0e951738c8ca8 2024-06-26T08:10:58.9311270Z Successfully built torchrec 2024-06-26T08:11:01.0878906Z Installing collected packages: lightning-utilities, torchmetrics, torchrec 2024-06-26T08:11:01.6570932Z Successfully installed lightning-utilities-0.11.2 torchmetrics-1.4.0.post0 torchrec-0.3.2 2024-06-26T08:11:01.7907554Z ++ pwd 2024-06-26T08:11:01.7908754Z + PYTHONPATH=/var/lib/jenkins/workspace/torchbench 2024-06-26T08:11:01.7910098Z + test_dynamo_benchmark torchbench 0 2024-06-26T08:11:01.7911041Z ++ pwd 2024-06-26T08:11:01.7913299Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T08:11:01.7914044Z + local suite=torchbench 2024-06-26T08:11:01.7914514Z + shift 2024-06-26T08:11:01.7914870Z + local shard_id=0 2024-06-26T08:11:01.7915221Z + shift 2024-06-26T08:11:01.7915579Z + [[ aot_inductor_torchbench == *perf_compare* ]] 2024-06-26T08:11:01.7917533Z + [[ aot_inductor_torchbench == *perf* ]] 2024-06-26T08:11:01.7918243Z + [[ aot_inductor_torchbench == *cpu_inductor* ]] 2024-06-26T08:11:01.7919109Z + [[ aot_inductor_torchbench == *aot_inductor* ]] 2024-06-26T08:11:01.7919956Z + test_single_dynamo_benchmark inference torchbench 0 --inference --bfloat16 2024-06-26T08:11:01.7920785Z ++ pwd 2024-06-26T08:11:01.7921261Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T08:11:01.7921933Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-06-26T08:11:01.7994200Z + local name=inference 2024-06-26T08:11:01.7994635Z + shift 2024-06-26T08:11:01.7995036Z + local suite=torchbench 2024-06-26T08:11:01.7995464Z + shift 2024-06-26T08:11:01.7995875Z + local shard_id=0 2024-06-26T08:11:01.7996192Z + shift 2024-06-26T08:11:01.7996522Z + partition_flags=() 2024-06-26T08:11:01.7996924Z + local partition_flags 2024-06-26T08:11:01.7997327Z + [[ -n 2 ]] 2024-06-26T08:11:01.7997636Z + [[ -n 0 ]] 2024-06-26T08:11:01.7998225Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-06-26T08:11:01.7998920Z + [[ aot_inductor_torchbench == *perf_compare* ]] 2024-06-26T08:11:01.7999419Z + [[ aot_inductor_torchbench == *perf* ]] 2024-06-26T08:11:01.7999904Z + [[ aot_inductor_torchbench == *aot_inductor* ]] 2024-06-26T08:11:01.8000429Z + export TORCHINDUCTOR_ABI_COMPATIBLE=1 2024-06-26T08:11:01.8000901Z + TORCHINDUCTOR_ABI_COMPATIBLE=1 2024-06-26T08:11:01.8002533Z + python benchmarks/dynamo/torchbench.py --ci --accuracy --timing --explain --export-aot-inductor --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv 2024-06-26T08:11:08.1171162Z 2024-06-26T08:11:08.6735449Z loading model: 0it [00:00, ?it/s]/var/lib/jenkins/.local/lib/python3.10/site-packages/torchrec/optim/apply_optimizer_in_backward.py:47: DeprecationWarning: `TorchScript` support for functional optimizers is deprecated and will be removed in a future PyTorch release. Consider using the `torch.compile` optimizer instead. 2024-06-26T08:11:08.6737482Z from torch.distributed.optim import _apply_optimizer_in_backward 2024-06-26T08:11:08.8633258Z 2024-06-26T08:11:08.8633628Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:11:08.8634197Z cuda eval torchrec_dlrm 2024-06-26T08:11:08.8644052Z Traceback (most recent call last): 2024-06-26T08:11:08.8644990Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2321, in validate_model 2024-06-26T08:11:08.8645999Z self.model_iter_fn(model, example_inputs) 2024-06-26T08:11:08.8646777Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 447, in forward_pass 2024-06-26T08:11:08.8647760Z return mod(*inputs) 2024-06-26T08:11:08.8649014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:11:08.8649944Z return self._call_impl(*args, **kwargs) 2024-06-26T08:11:08.8650894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:11:08.8651738Z return forward_call(*args, **kwargs) 2024-06-26T08:11:08.8652620Z File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchrec/models/dlrm.py", line 893, in forward 2024-06-26T08:11:08.8653564Z logits = self.model(batch.dense_features, batch.sparse_features) 2024-06-26T08:11:08.8654679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:11:08.8655638Z return self._call_impl(*args, **kwargs) 2024-06-26T08:11:08.8656576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:11:08.8657421Z return forward_call(*args, **kwargs) 2024-06-26T08:11:08.8658301Z File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchrec/models/dlrm.py", line 572, in forward 2024-06-26T08:11:08.8659109Z concatenated_dense = self.inter_arch( 2024-06-26T08:11:08.8660085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:11:08.8660984Z return self._call_impl(*args, **kwargs) 2024-06-26T08:11:08.8661923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:11:08.8662944Z return forward_call(*args, **kwargs) 2024-06-26T08:11:08.8663819Z File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchrec/models/dlrm.py", line 284, in forward 2024-06-26T08:11:08.8664852Z return self.crossnet(combined_values.reshape([B, -1])) 2024-06-26T08:11:08.8666004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:11:08.8667086Z return self._call_impl(*args, **kwargs) 2024-06-26T08:11:08.8668021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:11:08.8668861Z return forward_call(*args, **kwargs) 2024-06-26T08:11:08.8669767Z File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchrec/modules/crossnet.py", line 175, in forward 2024-06-26T08:11:08.8670720Z x_l_v = torch.nn.functional.linear(x_l, self.V_kernels[layer]) 2024-06-26T08:11:08.8671573Z RuntimeError: expected mat1 and mat2 to have the same dtype, but got: float != c10::BFloat16 2024-06-26T08:11:08.8672143Z 2024-06-26T08:11:08.8672440Z The above exception was the direct cause of the following exception: 2024-06-26T08:11:08.8672901Z 2024-06-26T08:11:08.8673049Z Traceback (most recent call last): 2024-06-26T08:11:08.8673716Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4229, in run 2024-06-26T08:11:08.8674392Z ) = runner.load_model( 2024-06-26T08:11:08.8675058Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 380, in load_model 2024-06-26T08:11:08.8675837Z self.validate_model(model, example_inputs) 2024-06-26T08:11:08.8676610Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2323, in validate_model 2024-06-26T08:11:08.8677387Z raise RuntimeError("Eager run failed") from e 2024-06-26T08:11:08.8677861Z RuntimeError: Eager run failed 2024-06-26T08:11:08.8678135Z 2024-06-26T08:11:08.8678254Z eager_fail_to_run 2024-06-26T08:11:12.1639338Z 2024-06-26T08:11:14.2018174Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:11:14.2018861Z loading model: 0it [00:02, ?it/s] 2024-06-26T08:11:14.2082686Z cuda eval BERT_pytorch 2024-06-26T08:11:15.2776562Z ERROR:common: 2024-06-26T08:11:15.2776991Z Traceback (most recent call last): 2024-06-26T08:11:15.2778023Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2616, in check_accuracy 2024-06-26T08:11:15.2779105Z optimized_model_iter_fn = optimize_ctx( 2024-06-26T08:11:15.2780145Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1222, in export_aot_inductor 2024-06-26T08:11:15.2781104Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-06-26T08:11:15.2781999Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1192, in load 2024-06-26T08:11:15.2782736Z gm = torch.export._trace._export_to_torch_ir( 2024-06-26T08:11:15.2783856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-06-26T08:11:15.2784858Z gm_torch_level, _ = torch._dynamo.export( 2024-06-26T08:11:15.2785772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1384, in inner 2024-06-26T08:11:15.2786777Z result_traced = opt_f(*args, **kwargs) 2024-06-26T08:11:15.2787776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:11:15.2788693Z return self._call_impl(*args, **kwargs) 2024-06-26T08:11:15.2789680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:11:15.2790538Z return forward_call(*args, **kwargs) 2024-06-26T08:11:15.2791415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 434, in _fn 2024-06-26T08:11:15.2792407Z return fn(*args, **kwargs) 2024-06-26T08:11:15.2793324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:11:15.2794230Z return self._call_impl(*args, **kwargs) 2024-06-26T08:11:15.2795282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:11:15.2796129Z return forward_call(*args, **kwargs) 2024-06-26T08:11:15.2797045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1121, in __call__ 2024-06-26T08:11:15.2797903Z return self._torchdynamo_orig_callable( 2024-06-26T08:11:15.2798840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 472, in __call__ 2024-06-26T08:11:15.2799658Z return _compile( 2024-06-26T08:11:15.2800489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_utils_internal.py", line 85, in wrapper_function 2024-06-26T08:11:15.2801478Z return StrobelightCompileTimeProfiler.profile_compile_time( 2024-06-26T08:11:15.2802685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_strobelight/compile_time_profiler.py", line 129, in profile_compile_time 2024-06-26T08:11:15.2803646Z return func(*args, **kwargs) 2024-06-26T08:11:15.2804280Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner 2024-06-26T08:11:15.2804918Z return func(*args, **kwds) 2024-06-26T08:11:15.2805788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 817, in _compile 2024-06-26T08:11:15.2806746Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-06-26T08:11:15.2807765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 233, in time_wrapper 2024-06-26T08:11:15.2808559Z r = func(*args, **kwargs) 2024-06-26T08:11:15.2809460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 636, in compile_inner 2024-06-26T08:11:15.2810385Z out_code = transform_code_object(code, transform) 2024-06-26T08:11:15.2811503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1270, in transform_code_object 2024-06-26T08:11:15.2812489Z transformations(instructions, code_options) 2024-06-26T08:11:15.2813427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 178, in _fn 2024-06-26T08:11:15.2814216Z return fn(*args, **kwargs) 2024-06-26T08:11:15.2815091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 582, in transform 2024-06-26T08:11:15.2815892Z tracer.run() 2024-06-26T08:11:15.2816705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2476, in run 2024-06-26T08:11:15.2817499Z super().run() 2024-06-26T08:11:15.2818313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 904, in run 2024-06-26T08:11:15.2819107Z while self.step(): 2024-06-26T08:11:15.2819940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 816, in step 2024-06-26T08:11:15.2820812Z self.dispatch_table[inst.opcode](self, inst) 2024-06-26T08:11:15.2821782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in wrapper 2024-06-26T08:11:15.2822616Z return inner_fn(self, inst) 2024-06-26T08:11:15.2823541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1480, in CALL_FUNCTION 2024-06-26T08:11:15.2824466Z self.call_function(fn, args, {}) 2024-06-26T08:11:15.2825436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 754, in call_function 2024-06-26T08:11:15.2826379Z self.push(fn.call_function(self, args, kwargs)) 2024-06-26T08:11:15.2827577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 353, in call_function 2024-06-26T08:11:15.2828530Z return super().call_function(tx, args, kwargs) 2024-06-26T08:11:15.2829641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 295, in call_function 2024-06-26T08:11:15.2830594Z return super().call_function(tx, args, kwargs) 2024-06-26T08:11:15.2831628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 95, in call_function 2024-06-26T08:11:15.2832731Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-06-26T08:11:15.2833994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 760, in inline_user_function_return 2024-06-26T08:11:15.2835147Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-06-26T08:11:15.2836315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2691, in inline_call 2024-06-26T08:11:15.2837252Z return cls.inline_call_(parent, func, args, kwargs) 2024-06-26T08:11:15.2838294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2807, in inline_call_ 2024-06-26T08:11:15.2839160Z tracer.run() 2024-06-26T08:11:15.2840014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 904, in run 2024-06-26T08:11:15.2840816Z while self.step(): 2024-06-26T08:11:15.2841649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 816, in step 2024-06-26T08:11:15.2842533Z self.dispatch_table[inst.opcode](self, inst) 2024-06-26T08:11:15.2843506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in wrapper 2024-06-26T08:11:15.2844340Z return inner_fn(self, inst) 2024-06-26T08:11:15.2845276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1480, in CALL_FUNCTION 2024-06-26T08:11:15.2846164Z self.call_function(fn, args, {}) 2024-06-26T08:11:15.2847130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 754, in call_function 2024-06-26T08:11:15.2848049Z self.push(fn.call_function(self, args, kwargs)) 2024-06-26T08:11:15.2849092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 353, in call_function 2024-06-26T08:11:15.2850082Z return super().call_function(tx, args, kwargs) 2024-06-26T08:11:15.2851114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 295, in call_function 2024-06-26T08:11:15.2852049Z return super().call_function(tx, args, kwargs) 2024-06-26T08:11:15.2853075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 95, in call_function 2024-06-26T08:11:15.2854153Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-06-26T08:11:15.2855392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 760, in inline_user_function_return 2024-06-26T08:11:15.2856535Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-06-26T08:11:15.2857680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2691, in inline_call 2024-06-26T08:11:15.2858624Z return cls.inline_call_(parent, func, args, kwargs) 2024-06-26T08:11:15.2859686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2807, in inline_call_ 2024-06-26T08:11:15.2860662Z tracer.run() 2024-06-26T08:11:15.2861454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 904, in run 2024-06-26T08:11:15.2862339Z while self.step(): 2024-06-26T08:11:15.2863169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 816, in step 2024-06-26T08:11:15.2864034Z self.dispatch_table[inst.opcode](self, inst) 2024-06-26T08:11:15.2865135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1596, in STORE_ATTR 2024-06-26T08:11:15.2865951Z assert ( 2024-06-26T08:11:15.2866393Z AssertionError: Mutating module attribute mask during export. 2024-06-26T08:11:15.2866920Z 2024-06-26T08:11:15.2867053Z from user code: 2024-06-26T08:11:15.2867914Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch/bert_pytorch/model/bert.py", line 50, in forward 2024-06-26T08:11:15.2868871Z x = transformer.forward(x, mask) 2024-06-26T08:11:15.2869920Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch/bert_pytorch/model/transformer.py", line 47, in forward 2024-06-26T08:11:15.2870922Z self.lambda_module.set_mask(mask) 2024-06-26T08:11:15.2871919Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch/bert_pytorch/model/transformer.py", line 16, in set_mask 2024-06-26T08:11:15.2872877Z self.mask = mask 2024-06-26T08:11:15.2873096Z 2024-06-26T08:11:15.2873413Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-06-26T08:11:15.2873874Z 2024-06-26T08:11:15.2874172Z TorchDynamo optimized model failed to run because of following error 2024-06-26T08:11:15.2874748Z fail_to_run 2024-06-26T08:11:15.2875023Z TIMING: 2024-06-26T08:11:15.2875607Z STATS: call_* op count: 0 | FakeTensor.__torch_dispatch__:12 | FakeTensorMode.__torch_dispatch__:43 2024-06-26T08:11:15.2876495Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-06-26T08:11:18.5502089Z 2024-06-26T08:11:21.3031322Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:11:21.3031817Z loading model: 0it [00:02, ?it/s] 2024-06-26T08:11:21.3085563Z cuda eval Background_Matting 2024-06-26T08:11:21.3088781Z pass_due_to_skip 2024-06-26T08:11:21.3612422Z TIMING: 2024-06-26T08:11:21.3612901Z STATS: call_* op count: 0 2024-06-26T08:11:21.3613717Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-06-26T08:11:24.6336042Z 2024-06-26T08:11:27.2751509Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:11:27.2751992Z loading model: 0it [00:02, ?it/s] 2024-06-26T08:11:27.2778889Z cuda eval LearningToPaint 2024-06-26T08:11:42.7199208Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp: In member function ‘void torch::aot_inductor::AOTInductorModel::run_impl(AtenTensorOpaque**, AtenTensorOpaque**, torch::aot_inductor::DeviceStreamType, AOTIProxyExecutorHandle)’: 2024-06-26T08:11:42.7202419Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1570:10: warning: variable ‘L__self___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7204037Z 1570 | auto L__self___bn1_weight = constants_->at(1); 2024-06-26T08:11:42.7204540Z | ^~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7206073Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1571:10: warning: variable ‘L__self___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7207572Z 1571 | auto L__self___bn1_bias = constants_->at(2); 2024-06-26T08:11:42.7208064Z | ^~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7209679Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1573:10: warning: variable ‘getattr_L__self___layer1___0___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7211364Z 1573 | auto getattr_L__self___layer1___0___bn1_weight = constants_->at(4); 2024-06-26T08:11:42.7212231Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7213891Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1574:10: warning: variable ‘getattr_L__self___layer1___0___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7215650Z 1574 | auto getattr_L__self___layer1___0___bn1_bias = constants_->at(5); 2024-06-26T08:11:42.7218802Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7221089Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1576:10: warning: variable ‘getattr_L__self___layer1___0___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7223672Z 1576 | auto getattr_L__self___layer1___0___bn2_weight = constants_->at(7); 2024-06-26T08:11:42.7224456Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7226161Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1577:10: warning: variable ‘getattr_L__self___layer1___0___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7228220Z 1577 | auto getattr_L__self___layer1___0___bn2_bias = constants_->at(8); 2024-06-26T08:11:42.7228825Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7230525Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1579:10: warning: variable ‘getattr_L__self___layer1___0___shortcut_1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7232267Z 1579 | auto getattr_L__self___layer1___0___shortcut_1_weight = constants_->at(10); 2024-06-26T08:11:42.7232918Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7234621Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1580:10: warning: variable ‘getattr_L__self___layer1___0___shortcut_1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7236335Z 1580 | auto getattr_L__self___layer1___0___shortcut_1_bias = constants_->at(11); 2024-06-26T08:11:42.7236987Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7238669Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1582:10: warning: variable ‘getattr_L__self___layer1___1___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7240357Z 1582 | auto getattr_L__self___layer1___1___bn1_weight = constants_->at(13); 2024-06-26T08:11:42.7240971Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7242624Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1583:10: warning: variable ‘getattr_L__self___layer1___1___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7244319Z 1583 | auto getattr_L__self___layer1___1___bn1_bias = constants_->at(14); 2024-06-26T08:11:42.7244939Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7246629Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1585:10: warning: variable ‘getattr_L__self___layer1___1___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7248372Z 1585 | auto getattr_L__self___layer1___1___bn2_weight = constants_->at(16); 2024-06-26T08:11:42.7248995Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7250667Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1586:10: warning: variable ‘getattr_L__self___layer1___1___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7252586Z 1586 | auto getattr_L__self___layer1___1___bn2_bias = constants_->at(17); 2024-06-26T08:11:42.7253202Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7254995Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1588:10: warning: variable ‘getattr_L__self___layer2___0___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7256704Z 1588 | auto getattr_L__self___layer2___0___bn1_weight = constants_->at(19); 2024-06-26T08:11:42.7257321Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7259047Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1589:10: warning: variable ‘getattr_L__self___layer2___0___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7260740Z 1589 | auto getattr_L__self___layer2___0___bn1_bias = constants_->at(20); 2024-06-26T08:11:42.7261344Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7263027Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1591:10: warning: variable ‘getattr_L__self___layer2___0___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7264824Z 1591 | auto getattr_L__self___layer2___0___bn2_weight = constants_->at(22); 2024-06-26T08:11:42.7265446Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7267413Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1592:10: warning: variable ‘getattr_L__self___layer2___0___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7269093Z 1592 | auto getattr_L__self___layer2___0___bn2_bias = constants_->at(23); 2024-06-26T08:11:42.7269710Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7271404Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1594:10: warning: variable ‘getattr_L__self___layer2___0___shortcut_1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7273153Z 1594 | auto getattr_L__self___layer2___0___shortcut_1_weight = constants_->at(25); 2024-06-26T08:11:42.7273810Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7275498Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1595:10: warning: variable ‘getattr_L__self___layer2___0___shortcut_1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7277217Z 1595 | auto getattr_L__self___layer2___0___shortcut_1_bias = constants_->at(26); 2024-06-26T08:11:42.7277924Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7279596Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1597:10: warning: variable ‘getattr_L__self___layer2___1___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7281290Z 1597 | auto getattr_L__self___layer2___1___bn1_weight = constants_->at(28); 2024-06-26T08:11:42.7281901Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7283560Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1598:10: warning: variable ‘getattr_L__self___layer2___1___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7285220Z 1598 | auto getattr_L__self___layer2___1___bn1_bias = constants_->at(29); 2024-06-26T08:11:42.7285822Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7287611Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1600:10: warning: variable ‘getattr_L__self___layer2___1___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7289285Z 1600 | auto getattr_L__self___layer2___1___bn2_weight = constants_->at(31); 2024-06-26T08:11:42.7289977Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7291638Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1601:10: warning: variable ‘getattr_L__self___layer2___1___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7293309Z 1601 | auto getattr_L__self___layer2___1___bn2_bias = constants_->at(32); 2024-06-26T08:11:42.7293909Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7295577Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1603:10: warning: variable ‘getattr_L__self___layer3___0___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7297274Z 1603 | auto getattr_L__self___layer3___0___bn1_weight = constants_->at(34); 2024-06-26T08:11:42.7297887Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7299543Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1604:10: warning: variable ‘getattr_L__self___layer3___0___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7301216Z 1604 | auto getattr_L__self___layer3___0___bn1_bias = constants_->at(35); 2024-06-26T08:11:42.7301818Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7303476Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1606:10: warning: variable ‘getattr_L__self___layer3___0___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7305274Z 1606 | auto getattr_L__self___layer3___0___bn2_weight = constants_->at(37); 2024-06-26T08:11:42.7305893Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7307757Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1607:10: warning: variable ‘getattr_L__self___layer3___0___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7309442Z 1607 | auto getattr_L__self___layer3___0___bn2_bias = constants_->at(38); 2024-06-26T08:11:42.7310047Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7311730Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1609:10: warning: variable ‘getattr_L__self___layer3___0___shortcut_1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7313473Z 1609 | auto getattr_L__self___layer3___0___shortcut_1_weight = constants_->at(40); 2024-06-26T08:11:42.7314125Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7315827Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1610:10: warning: variable ‘getattr_L__self___layer3___0___shortcut_1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7317554Z 1610 | auto getattr_L__self___layer3___0___shortcut_1_bias = constants_->at(41); 2024-06-26T08:11:42.7318251Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7319925Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1612:10: warning: variable ‘getattr_L__self___layer3___1___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7321595Z 1612 | auto getattr_L__self___layer3___1___bn1_weight = constants_->at(43); 2024-06-26T08:11:42.7322331Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7323976Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1613:10: warning: variable ‘getattr_L__self___layer3___1___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7325730Z 1613 | auto getattr_L__self___layer3___1___bn1_bias = constants_->at(44); 2024-06-26T08:11:42.7326354Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7328008Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1615:10: warning: variable ‘getattr_L__self___layer3___1___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7329694Z 1615 | auto getattr_L__self___layer3___1___bn2_weight = constants_->at(46); 2024-06-26T08:11:42.7330312Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7331971Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1616:10: warning: variable ‘getattr_L__self___layer3___1___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7333633Z 1616 | auto getattr_L__self___layer3___1___bn2_bias = constants_->at(47); 2024-06-26T08:11:42.7334242Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7335891Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1618:10: warning: variable ‘getattr_L__self___layer4___0___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7337565Z 1618 | auto getattr_L__self___layer4___0___bn1_weight = constants_->at(49); 2024-06-26T08:11:42.7338185Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7339836Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1619:10: warning: variable ‘getattr_L__self___layer4___0___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7341492Z 1619 | auto getattr_L__self___layer4___0___bn1_bias = constants_->at(50); 2024-06-26T08:11:42.7342101Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7343756Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1621:10: warning: variable ‘getattr_L__self___layer4___0___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7345526Z 1621 | auto getattr_L__self___layer4___0___bn2_weight = constants_->at(52); 2024-06-26T08:11:42.7346145Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7348008Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1622:10: warning: variable ‘getattr_L__self___layer4___0___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7349696Z 1622 | auto getattr_L__self___layer4___0___bn2_bias = constants_->at(53); 2024-06-26T08:11:42.7350308Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7352023Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1624:10: warning: variable ‘getattr_L__self___layer4___0___shortcut_1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7353786Z 1624 | auto getattr_L__self___layer4___0___shortcut_1_weight = constants_->at(55); 2024-06-26T08:11:42.7354446Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7356154Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1625:10: warning: variable ‘getattr_L__self___layer4___0___shortcut_1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7358073Z 1625 | auto getattr_L__self___layer4___0___shortcut_1_bias = constants_->at(56); 2024-06-26T08:11:42.7358725Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7360476Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1627:10: warning: variable ‘getattr_L__self___layer4___1___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7362170Z 1627 | auto getattr_L__self___layer4___1___bn1_weight = constants_->at(58); 2024-06-26T08:11:42.7362798Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7364458Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1628:10: warning: variable ‘getattr_L__self___layer4___1___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7366144Z 1628 | auto getattr_L__self___layer4___1___bn1_bias = constants_->at(59); 2024-06-26T08:11:42.7366754Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7368423Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1630:10: warning: variable ‘getattr_L__self___layer4___1___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7370109Z 1630 | auto getattr_L__self___layer4___1___bn2_weight = constants_->at(61); 2024-06-26T08:11:42.7370732Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7372397Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1631:10: warning: variable ‘getattr_L__self___layer4___1___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7374069Z 1631 | auto getattr_L__self___layer4___1___bn2_bias = constants_->at(62); 2024-06-26T08:11:42.7374688Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7376292Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1634:10: warning: variable ‘L__self___bn1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7377872Z 1634 | auto L__self___bn1_running_mean = constants_->at(65); 2024-06-26T08:11:42.7378402Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7379982Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1635:10: warning: variable ‘L__self___bn1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7381540Z 1635 | auto L__self___bn1_running_var = constants_->at(66); 2024-06-26T08:11:42.7382071Z | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7383740Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1636:10: warning: variable ‘getattr_L__self___layer1___0___bn1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7385596Z 1636 | auto getattr_L__self___layer1___0___bn1_running_mean = constants_->at(67); 2024-06-26T08:11:42.7386266Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7388350Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1637:10: warning: variable ‘getattr_L__self___layer1___0___bn1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7390106Z 1637 | auto getattr_L__self___layer1___0___bn1_running_var = constants_->at(68); 2024-06-26T08:11:42.7390754Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7392458Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1638:10: warning: variable ‘getattr_L__self___layer1___0___bn2_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7394361Z 1638 | auto getattr_L__self___layer1___0___bn2_running_mean = constants_->at(69); 2024-06-26T08:11:42.7395017Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7396801Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1639:10: warning: variable ‘getattr_L__self___layer1___0___bn2_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7398540Z 1639 | auto getattr_L__self___layer1___0___bn2_running_var = constants_->at(70); 2024-06-26T08:11:42.7399188Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7400925Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1640:10: warning: variable ‘getattr_L__self___layer1___0___shortcut_1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7402727Z 1640 | auto getattr_L__self___layer1___0___shortcut_1_running_mean = constants_->at(71); 2024-06-26T08:11:42.7403427Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7405202Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1641:10: warning: variable ‘getattr_L__self___layer1___0___shortcut_1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7407001Z 1641 | auto getattr_L__self___layer1___0___shortcut_1_running_var = constants_->at(72); 2024-06-26T08:11:42.7407695Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7409421Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1642:10: warning: variable ‘getattr_L__self___layer1___1___bn1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7411167Z 1642 | auto getattr_L__self___layer1___1___bn1_running_mean = constants_->at(73); 2024-06-26T08:11:42.7411826Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7413532Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1643:10: warning: variable ‘getattr_L__self___layer1___1___bn1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7415264Z 1643 | auto getattr_L__self___layer1___1___bn1_running_var = constants_->at(74); 2024-06-26T08:11:42.7415911Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7417645Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1644:10: warning: variable ‘getattr_L__self___layer1___1___bn2_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7419430Z 1644 | auto getattr_L__self___layer1___1___bn2_running_mean = constants_->at(75); 2024-06-26T08:11:42.7420089Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7421802Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1645:10: warning: variable ‘getattr_L__self___layer1___1___bn2_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7423521Z 1645 | auto getattr_L__self___layer1___1___bn2_running_var = constants_->at(76); 2024-06-26T08:11:42.7424286Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7425997Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1646:10: warning: variable ‘getattr_L__self___layer2___0___bn1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7428054Z 1646 | auto getattr_L__self___layer2___0___bn1_running_mean = constants_->at(77); 2024-06-26T08:11:42.7428703Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7430504Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1647:10: warning: variable ‘getattr_L__self___layer2___0___bn1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7432231Z 1647 | auto getattr_L__self___layer2___0___bn1_running_var = constants_->at(78); 2024-06-26T08:11:42.7432871Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7434578Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1648:10: warning: variable ‘getattr_L__self___layer2___0___bn2_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7436321Z 1648 | auto getattr_L__self___layer2___0___bn2_running_mean = constants_->at(79); 2024-06-26T08:11:42.7436972Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7438669Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1649:10: warning: variable ‘getattr_L__self___layer2___0___bn2_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7440412Z 1649 | auto getattr_L__self___layer2___0___bn2_running_var = constants_->at(80); 2024-06-26T08:11:42.7441057Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7442794Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1650:10: warning: variable ‘getattr_L__self___layer2___0___shortcut_1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7444593Z 1650 | auto getattr_L__self___layer2___0___shortcut_1_running_mean = constants_->at(81); 2024-06-26T08:11:42.7445286Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7447019Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1651:10: warning: variable ‘getattr_L__self___layer2___0___shortcut_1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7448807Z 1651 | auto getattr_L__self___layer2___0___shortcut_1_running_var = constants_->at(82); 2024-06-26T08:11:42.7449485Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7451183Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1652:10: warning: variable ‘getattr_L__self___layer2___1___bn1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7452923Z 1652 | auto getattr_L__self___layer2___1___bn1_running_mean = constants_->at(83); 2024-06-26T08:11:42.7453584Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7455270Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1653:10: warning: variable ‘getattr_L__self___layer2___1___bn1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7457000Z 1653 | auto getattr_L__self___layer2___1___bn1_running_var = constants_->at(84); 2024-06-26T08:11:42.7457636Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7459326Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1654:10: warning: variable ‘getattr_L__self___layer2___1___bn2_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7461055Z 1654 | auto getattr_L__self___layer2___1___bn2_running_mean = constants_->at(85); 2024-06-26T08:11:42.7461701Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7463485Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1655:10: warning: variable ‘getattr_L__self___layer2___1___bn2_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7465374Z 1655 | auto getattr_L__self___layer2___1___bn2_running_var = constants_->at(86); 2024-06-26T08:11:42.7466018Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7467940Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1656:10: warning: variable ‘getattr_L__self___layer3___0___bn1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7469672Z 1656 | auto getattr_L__self___layer3___0___bn1_running_mean = constants_->at(87); 2024-06-26T08:11:42.7470313Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7472012Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1657:10: warning: variable ‘getattr_L__self___layer3___0___bn1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7473723Z 1657 | auto getattr_L__self___layer3___0___bn1_running_var = constants_->at(88); 2024-06-26T08:11:42.7474371Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7476072Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1658:10: warning: variable ‘getattr_L__self___layer3___0___bn2_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7477826Z 1658 | auto getattr_L__self___layer3___0___bn2_running_mean = constants_->at(89); 2024-06-26T08:11:42.7478473Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7480167Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1659:10: warning: variable ‘getattr_L__self___layer3___0___bn2_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7481909Z 1659 | auto getattr_L__self___layer3___0___bn2_running_var = constants_->at(90); 2024-06-26T08:11:42.7482548Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7484279Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1660:10: warning: variable ‘getattr_L__self___layer3___0___shortcut_1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7486072Z 1660 | auto getattr_L__self___layer3___0___shortcut_1_running_mean = constants_->at(91); 2024-06-26T08:11:42.7486751Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7488539Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1661:10: warning: variable ‘getattr_L__self___layer3___0___shortcut_1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7490321Z 1661 | auto getattr_L__self___layer3___0___shortcut_1_running_var = constants_->at(92); 2024-06-26T08:11:42.7491006Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7492721Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1662:10: warning: variable ‘getattr_L__self___layer3___1___bn1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7494438Z 1662 | auto getattr_L__self___layer3___1___bn1_running_mean = constants_->at(93); 2024-06-26T08:11:42.7495090Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7496785Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1663:10: warning: variable ‘getattr_L__self___layer3___1___bn1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7498634Z 1663 | auto getattr_L__self___layer3___1___bn1_running_var = constants_->at(94); 2024-06-26T08:11:42.7499285Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7501065Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1664:10: warning: variable ‘getattr_L__self___layer3___1___bn2_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7502795Z 1664 | auto getattr_L__self___layer3___1___bn2_running_mean = constants_->at(95); 2024-06-26T08:11:42.7503449Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7505232Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1665:10: warning: variable ‘getattr_L__self___layer3___1___bn2_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7507182Z 1665 | auto getattr_L__self___layer3___1___bn2_running_var = constants_->at(96); 2024-06-26T08:11:42.7507936Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7509936Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1666:10: warning: variable ‘getattr_L__self___layer4___0___bn1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7511948Z 1666 | auto getattr_L__self___layer4___0___bn1_running_mean = constants_->at(97); 2024-06-26T08:11:42.7512654Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7514629Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1667:10: warning: variable ‘getattr_L__self___layer4___0___bn1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7516637Z 1667 | auto getattr_L__self___layer4___0___bn1_running_var = constants_->at(98); 2024-06-26T08:11:42.7517334Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7519320Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1668:10: warning: variable ‘getattr_L__self___layer4___0___bn2_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7521339Z 1668 | auto getattr_L__self___layer4___0___bn2_running_mean = constants_->at(99); 2024-06-26T08:11:42.7522049Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7524017Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1669:10: warning: variable ‘getattr_L__self___layer4___0___bn2_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7526035Z 1669 | auto getattr_L__self___layer4___0___bn2_running_var = constants_->at(100); 2024-06-26T08:11:42.7526746Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7528768Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1670:10: warning: variable ‘getattr_L__self___layer4___0___shortcut_1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7530866Z 1670 | auto getattr_L__self___layer4___0___shortcut_1_running_mean = constants_->at(101); 2024-06-26T08:11:42.7531624Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7533652Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1671:10: warning: variable ‘getattr_L__self___layer4___0___shortcut_1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7535582Z 1671 | auto getattr_L__self___layer4___0___shortcut_1_running_var = constants_->at(102); 2024-06-26T08:11:42.7536274Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7538053Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1672:10: warning: variable ‘getattr_L__self___layer4___1___bn1_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7539790Z 1672 | auto getattr_L__self___layer4___1___bn1_running_mean = constants_->at(103); 2024-06-26T08:11:42.7540441Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7542139Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1673:10: warning: variable ‘getattr_L__self___layer4___1___bn1_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7543853Z 1673 | auto getattr_L__self___layer4___1___bn1_running_var = constants_->at(104); 2024-06-26T08:11:42.7544602Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7546307Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1674:10: warning: variable ‘getattr_L__self___layer4___1___bn2_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7548162Z 1674 | auto getattr_L__self___layer4___1___bn2_running_mean = constants_->at(105); 2024-06-26T08:11:42.7548824Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:42.7550513Z /tmp/tmp76y1ccx0/c3l5tlhbskil3fvrrf22ly7bpuriex6ahe27zq7ly74fqsdphn7n/c442ejxo2xdf3yykze6inpbd7i5jf5x2tttvu2g6vhfnsmunrmcc.cpp:1675:10: warning: variable ‘getattr_L__self___layer4___1___bn2_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:11:42.7552236Z 1675 | auto getattr_L__self___layer4___1___bn2_running_var = constants_->at(106); 2024-06-26T08:11:42.7552888Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:11:43.4459878Z pass 2024-06-26T08:11:43.4460470Z TIMING: backend_compile:0.0002 entire_frame_compile:0.40927 2024-06-26T08:11:43.4461548Z STATS: call_* op count: 71 | FakeTensor.__torch_dispatch__:1073 | FakeTensorMode.__torch_dispatch__:8488 | ProxyTorchDispatchMode.__torch_dispatch__:1808 2024-06-26T08:11:43.4462704Z Dynamo produced 1 graphs covering 71 ops with 0 graph breaks (0 unique) 2024-06-26T08:11:47.2534823Z 2024-06-26T08:11:48.5206423Z loading model: 0it [00:00, ?it/s]Downloading: "https://download.pytorch.org/models/vgg16-397923af.pth" to /var/lib/jenkins/.cache/torch/hub/checkpoints/vgg16-397923af.pth 2024-06-26T08:11:48.5431373Z 2024-06-26T08:11:48.5431625Z 2024-06-26T08:11:48.6435895Z 0% 0.00/528M [00:00= tmp1 2024-06-26T08:13:48.5752918Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] tmp4 = tmp0 < tmp2 2024-06-26T08:13:48.5754003Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] tmp5 = ops.load(buf2, i0 * Max(1, u2) + Identity(i1)) 2024-06-26T08:13:48.5755147Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] tmp6 = ops.masked(tmp4, tmp5, 0.0) 2024-06-26T08:13:48.5756252Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] tmp7 = ops.index_expr(u2, torch.int64) 2024-06-26T08:13:48.5757387Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] tmp8 = ops.index_expr(10000 + u2, torch.int64) 2024-06-26T08:13:48.5758456Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] tmp9 = tmp0 >= tmp7 2024-06-26T08:13:48.5759413Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] tmp10 = tmp0 < tmp8 2024-06-26T08:13:48.5760701Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] tmp11 = ops.index_expr(Identity(i1 + -1 * u2), dtype=torch.int64) 2024-06-26T08:13:48.5761939Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] tmp12 = ops.masked(tmp9, tmp11, 0.0) 2024-06-26T08:13:48.5763072Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] tmp13 = ops.where(tmp4, tmp6, tmp12) 2024-06-26T08:13:48.5764079Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] return tmp13 2024-06-26T08:13:48.5764920Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] , 2024-06-26T08:13:48.5765772Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] ranges=[2, u2 + 10000], 2024-06-26T08:13:48.5766711Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] origin_node=cat_2, 2024-06-26T08:13:48.5767593Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] origins={cat_2} 2024-06-26T08:13:48.5768403Z C0626 08:13:48.572000 140302596915840 torch/_inductor/scheduler.py:854] )) 2024-06-26T08:13:48.5769446Z ERROR:common: 2024-06-26T08:13:48.5769866Z Traceback (most recent call last): 2024-06-26T08:13:48.5770907Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2616, in check_accuracy 2024-06-26T08:13:48.5771944Z optimized_model_iter_fn = optimize_ctx( 2024-06-26T08:13:48.5773006Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1222, in export_aot_inductor 2024-06-26T08:13:48.5774030Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-06-26T08:13:48.5774893Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1198, in load 2024-06-26T08:13:48.5775654Z so_path = torch._inductor.aot_compile( 2024-06-26T08:13:48.5776613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/__init__.py", line 105, in aot_compile 2024-06-26T08:13:48.5777428Z return compile_fx_aot( 2024-06-26T08:13:48.5778317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1188, in compile_fx_aot 2024-06-26T08:13:48.5779191Z compiled_lib_path = compile_fx( 2024-06-26T08:13:48.5779825Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner 2024-06-26T08:13:48.5780455Z return func(*args, **kwds) 2024-06-26T08:13:48.5781556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1298, in compile_fx 2024-06-26T08:13:48.5782371Z return compile_fx( 2024-06-26T08:13:48.5782931Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner 2024-06-26T08:13:48.5783559Z return func(*args, **kwds) 2024-06-26T08:13:48.5784600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1340, in compile_fx 2024-06-26T08:13:48.5785429Z return compile_fx( 2024-06-26T08:13:48.5785997Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner 2024-06-26T08:13:48.5786907Z return func(*args, **kwds) 2024-06-26T08:13:48.5787795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1545, in compile_fx 2024-06-26T08:13:48.5788706Z return inference_compiler(unlifted_gm, example_inputs_) 2024-06-26T08:13:48.5789685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 233, in time_wrapper 2024-06-26T08:13:48.5790480Z r = func(*args, **kwargs) 2024-06-26T08:13:48.5791389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 1455, in fw_compiler_base 2024-06-26T08:13:48.5792241Z return inner_compile( 2024-06-26T08:13:48.5792832Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner 2024-06-26T08:13:48.5793472Z return func(*args, **kwds) 2024-06-26T08:13:48.5794368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/repro/after_aot.py", line 84, in debug_wrapper 2024-06-26T08:13:48.5795296Z inner_compiled_fn = compiler_fn(gm, example_inputs) 2024-06-26T08:13:48.5796223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/debug.py", line 304, in inner 2024-06-26T08:13:48.5797011Z return fn(*args, **kwargs) 2024-06-26T08:13:48.5797604Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner 2024-06-26T08:13:48.5798231Z return func(*args, **kwds) 2024-06-26T08:13:48.5798825Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner 2024-06-26T08:13:48.5799468Z return func(*args, **kwds) 2024-06-26T08:13:48.5800324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 433, in wrapper 2024-06-26T08:13:48.5801186Z return fn(*args, **kwargs) 2024-06-26T08:13:48.5802041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 233, in time_wrapper 2024-06-26T08:13:48.5802846Z r = func(*args, **kwargs) 2024-06-26T08:13:48.5803738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 546, in compile_fx_inner 2024-06-26T08:13:48.5804623Z compiled_graph = fx_codegen_and_compile( 2024-06-26T08:13:48.5805299Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner 2024-06-26T08:13:48.5805939Z return func(*args, **kwds) 2024-06-26T08:13:48.5806876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 868, in fx_codegen_and_compile 2024-06-26T08:13:48.5807788Z compiled_fn = graph.compile_to_fn() 2024-06-26T08:13:48.5808718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/graph.py", line 1726, in compile_to_fn 2024-06-26T08:13:48.5809609Z code, linemap = self.codegen_with_cpp_wrapper() 2024-06-26T08:13:48.5810649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/graph.py", line 1542, in codegen_with_cpp_wrapper 2024-06-26T08:13:48.5811612Z compiled = self.compile_to_module().call 2024-06-26T08:13:48.5812534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 233, in time_wrapper 2024-06-26T08:13:48.5813330Z r = func(*args, **kwargs) 2024-06-26T08:13:48.5814227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/graph.py", line 1676, in compile_to_module 2024-06-26T08:13:48.5815335Z self.codegen_with_cpp_wrapper() if self.cpp_wrapper else self.codegen() 2024-06-26T08:13:48.5816381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/graph.py", line 1631, in codegen 2024-06-26T08:13:48.5817169Z self.scheduler.codegen() 2024-06-26T08:13:48.5818129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 233, in time_wrapper 2024-06-26T08:13:48.5818929Z r = func(*args, **kwargs) 2024-06-26T08:13:48.5819793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/scheduler.py", line 2767, in codegen 2024-06-26T08:13:48.5820646Z self.get_backend(device).codegen_node(node) 2024-06-26T08:13:48.5821786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/cuda_combined_scheduling.py", line 72, in codegen_node 2024-06-26T08:13:48.5822796Z return self._triton_scheduling.codegen_node(node) 2024-06-26T08:13:48.5823811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 1151, in codegen_node 2024-06-26T08:13:48.5824934Z return self.codegen_node_schedule(node_schedule, buf_accesses, numel, rnumel) 2024-06-26T08:13:48.5826125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 1319, in codegen_node_schedule 2024-06-26T08:13:48.5827388Z self.codegen_node_schedule_with_kernel(node_schedule, kernel) 2024-06-26T08:13:48.5828559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 1421, in codegen_node_schedule_with_kernel 2024-06-26T08:13:48.5829501Z node.codegen(index_vars) 2024-06-26T08:13:48.5830331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/scheduler.py", line 852, in codegen 2024-06-26T08:13:48.5831168Z self._body(*index_vars) 2024-06-26T08:13:48.5831967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/ir.py", line 6411, in __call__ 2024-06-26T08:13:48.5832756Z result = self.root_block() 2024-06-26T08:13:48.5833567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/ir.py", line 6573, in __call__ 2024-06-26T08:13:48.5834496Z return InterpreterShim(graph, submodules).run(V.get_ops_handler()) 2024-06-26T08:13:48.5835467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/ir.py", line 6311, in run 2024-06-26T08:13:48.5836231Z return super().run(*args, **kwargs) 2024-06-26T08:13:48.5837069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/interpreter.py", line 146, in run 2024-06-26T08:13:48.5837843Z self.env[node] = self.run_node(node) 2024-06-26T08:13:48.5838702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/ir.py", line 6307, in run_node 2024-06-26T08:13:48.5839471Z return super().run_node(n) 2024-06-26T08:13:48.5840304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/interpreter.py", line 203, in run_node 2024-06-26T08:13:48.5841154Z return getattr(self, n.op)(n.target, args, kwargs) 2024-06-26T08:13:48.5842101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/interpreter.py", line 320, in call_module 2024-06-26T08:13:48.5842886Z return submod(*args, **kwargs) 2024-06-26T08:13:48.5843695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/ir.py", line 6489, in shim 2024-06-26T08:13:48.5844475Z return V.ops.masked(mask, subblock, other) 2024-06-26T08:13:48.5845410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/common.py", line 1739, in inner 2024-06-26T08:13:48.5846482Z value = getattr(parent_handler, name)(*args, **kwargs) # type: ignore[has-type] 2024-06-26T08:13:48.5847588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/triton.py", line 1025, in masked 2024-06-26T08:13:48.5848514Z result = body() 2024-06-26T08:13:48.5849274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/ir.py", line 6573, in __call__ 2024-06-26T08:13:48.5850183Z return InterpreterShim(graph, submodules).run(V.get_ops_handler()) 2024-06-26T08:13:48.5851148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/ir.py", line 6311, in run 2024-06-26T08:13:48.5851957Z return super().run(*args, **kwargs) 2024-06-26T08:13:48.5852798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/interpreter.py", line 146, in run 2024-06-26T08:13:48.5853566Z self.env[node] = self.run_node(node) 2024-06-26T08:13:48.5854426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/ir.py", line 6307, in run_node 2024-06-26T08:13:48.5855186Z return super().run_node(n) 2024-06-26T08:13:48.5856009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/interpreter.py", line 203, in run_node 2024-06-26T08:13:48.5856857Z return getattr(self, n.op)(n.target, args, kwargs) 2024-06-26T08:13:48.5857796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/interpreter.py", line 297, in call_method 2024-06-26T08:13:48.5858681Z return getattr(self_obj, target)(*args_tail, **kwargs) 2024-06-26T08:13:48.5859633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/sizevars.py", line 827, in load 2024-06-26T08:13:48.5860485Z return self._inner.load(name, self._simplify(index)) 2024-06-26T08:13:48.5861447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/common.py", line 1857, in load 2024-06-26T08:13:48.5862253Z out = self.load(name, index) 2024-06-26T08:13:48.5863114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/triton.py", line 1662, in load 2024-06-26T08:13:48.5863991Z indexing = self.indexing(index, block_ptr=True) 2024-06-26T08:13:48.5865045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/triton.py", line 1245, in indexing 2024-06-26T08:13:48.5865905Z index = self.prepare_indexing(index) 2024-06-26T08:13:48.5867066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 668, in prepare_indexing 2024-06-26T08:13:48.5867953Z index = self.simplify_indexing(index) 2024-06-26T08:13:48.5868932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 342, in simplify_indexing 2024-06-26T08:13:48.5869867Z index = self.combine_contiguous_dims(index, tree) 2024-06-26T08:13:48.5870922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 458, in combine_contiguous_dims 2024-06-26T08:13:48.5871924Z return self._combine_contiguous_dims(index, tree) 2024-06-26T08:13:48.5872973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 466, in _combine_contiguous_dims 2024-06-26T08:13:48.5873923Z index_vars, sizes = tree.vars_and_sizes(index) 2024-06-26T08:13:48.5874900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 203, in vars_and_sizes 2024-06-26T08:13:48.5875892Z nodes.sort(key=lambda x: V.graph.sizevars.size_hint(x.divisor)) 2024-06-26T08:13:48.5876929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/codegen/simd.py", line 203, in 2024-06-26T08:13:48.5877872Z nodes.sort(key=lambda x: V.graph.sizevars.size_hint(x.divisor)) 2024-06-26T08:13:48.5878889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/sizevars.py", line 488, in size_hint 2024-06-26T08:13:48.5879668Z return int(out) 2024-06-26T08:13:48.5880396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/sympy/core/expr.py", line 320, in __int__ 2024-06-26T08:13:48.5881202Z raise TypeError("Cannot convert symbols to int") 2024-06-26T08:13:48.5881713Z TypeError: Cannot convert symbols to int 2024-06-26T08:13:48.5882421Z TorchDynamo optimized model failed to run because of following error 2024-06-26T08:13:48.5882978Z fail_to_run 2024-06-26T08:13:48.5885072Z TIMING: backend_compile:9e-05 entire_frame_compile:0.63667 2024-06-26T08:13:48.5886489Z STATS: call_* op count: 109 | FakeTensorMode.__torch_dispatch__:2880 | FakeTensor.__torch_dispatch__:214 | attempt fast:45 | fast is_contiguous:45 | ProxyTorchDispatchMode.__torch_dispatch__:599 2024-06-26T08:13:48.5887814Z Dynamo produced 1 graphs covering 109 ops with 0 graph breaks (0 unique) 2024-06-26T08:13:52.2076503Z 2024-06-26T08:13:53.9092294Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:13:53.9092891Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:13:53.9097157Z cuda eval basic_gnn_gin 2024-06-26T08:14:03.1832949Z pass 2024-06-26T08:14:03.1927499Z TIMING: backend_compile:8e-05 entire_frame_compile:0.36493 2024-06-26T08:14:03.1928613Z STATS: call_* op count: 55 | FakeTensorMode.__torch_dispatch__:1841 | ProxyTorchDispatchMode.__torch_dispatch__:474 | FakeTensor.__torch_dispatch__:103 2024-06-26T08:14:03.1929760Z Dynamo produced 1 graphs covering 55 ops with 0 graph breaks (0 unique) 2024-06-26T08:14:06.7349314Z 2024-06-26T08:14:08.4629738Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:14:08.4630289Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:14:08.4632536Z cuda eval basic_gnn_sage 2024-06-26T08:14:17.8062220Z pass 2024-06-26T08:14:17.8156436Z TIMING: backend_compile:8e-05 entire_frame_compile:0.30821 2024-06-26T08:14:17.8157601Z STATS: call_* op count: 58 | FakeTensorMode.__torch_dispatch__:1837 | ProxyTorchDispatchMode.__torch_dispatch__:466 | FakeTensor.__torch_dispatch__:138 2024-06-26T08:14:17.8158714Z Dynamo produced 1 graphs covering 58 ops with 0 graph breaks (0 unique) 2024-06-26T08:14:21.3217800Z 2024-06-26T08:14:21.7573766Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:14:21.7574247Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:14:21.7580112Z cuda eval dcgan 2024-06-26T08:14:31.4760064Z /tmp/tmp76y1ccx0/cczfur6bebhwsy3riytfcocs2qnzekeapeidnjau25lir3jzu4nm/csewirsb6txds6pa7iub5etu7jjpdawfojyyfpogqbu3eqffwu4y.cpp: In member function ‘void torch::aot_inductor::AOTInductorModel::run_impl(AtenTensorOpaque**, AtenTensorOpaque**, torch::aot_inductor::DeviceStreamType, AOTIProxyExecutorHandle)’: 2024-06-26T08:14:31.4767887Z /tmp/tmp76y1ccx0/cczfur6bebhwsy3riytfcocs2qnzekeapeidnjau25lir3jzu4nm/csewirsb6txds6pa7iub5etu7jjpdawfojyyfpogqbu3eqffwu4y.cpp:749:10: warning: variable ‘L__self___main_3_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:14:31.4769511Z 749 | auto L__self___main_3_bias = constants_->at(3); 2024-06-26T08:14:31.4770025Z | ^~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:14:31.4771644Z /tmp/tmp76y1ccx0/cczfur6bebhwsy3riytfcocs2qnzekeapeidnjau25lir3jzu4nm/csewirsb6txds6pa7iub5etu7jjpdawfojyyfpogqbu3eqffwu4y.cpp:752:10: warning: variable ‘L__self___main_6_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:14:31.4773289Z 752 | auto L__self___main_6_bias = constants_->at(6); 2024-06-26T08:14:31.4773843Z | ^~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:14:31.4775515Z /tmp/tmp76y1ccx0/cczfur6bebhwsy3riytfcocs2qnzekeapeidnjau25lir3jzu4nm/csewirsb6txds6pa7iub5etu7jjpdawfojyyfpogqbu3eqffwu4y.cpp:755:10: warning: variable ‘L__self___main_9_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:14:31.4777147Z 755 | auto L__self___main_9_bias = constants_->at(9); 2024-06-26T08:14:31.4777690Z | ^~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:14:31.4779372Z /tmp/tmp76y1ccx0/cczfur6bebhwsy3riytfcocs2qnzekeapeidnjau25lir3jzu4nm/csewirsb6txds6pa7iub5etu7jjpdawfojyyfpogqbu3eqffwu4y.cpp:757:10: warning: variable ‘L__self___main_3_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:14:31.4781074Z 757 | auto L__self___main_3_running_mean = constants_->at(11); 2024-06-26T08:14:31.4781632Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:14:31.4783534Z /tmp/tmp76y1ccx0/cczfur6bebhwsy3riytfcocs2qnzekeapeidnjau25lir3jzu4nm/csewirsb6txds6pa7iub5etu7jjpdawfojyyfpogqbu3eqffwu4y.cpp:758:10: warning: variable ‘L__self___main_3_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:14:31.4785229Z 758 | auto L__self___main_3_running_var = constants_->at(12); 2024-06-26T08:14:31.4785769Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:14:31.4787808Z /tmp/tmp76y1ccx0/cczfur6bebhwsy3riytfcocs2qnzekeapeidnjau25lir3jzu4nm/csewirsb6txds6pa7iub5etu7jjpdawfojyyfpogqbu3eqffwu4y.cpp:759:10: warning: variable ‘L__self___main_6_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:14:31.4789441Z 759 | auto L__self___main_6_running_mean = constants_->at(13); 2024-06-26T08:14:31.4789996Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:14:31.4791962Z /tmp/tmp76y1ccx0/cczfur6bebhwsy3riytfcocs2qnzekeapeidnjau25lir3jzu4nm/csewirsb6txds6pa7iub5etu7jjpdawfojyyfpogqbu3eqffwu4y.cpp:760:10: warning: variable ‘L__self___main_6_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:14:31.4793567Z 760 | auto L__self___main_6_running_var = constants_->at(14); 2024-06-26T08:14:31.4794115Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:14:31.4795727Z /tmp/tmp76y1ccx0/cczfur6bebhwsy3riytfcocs2qnzekeapeidnjau25lir3jzu4nm/csewirsb6txds6pa7iub5etu7jjpdawfojyyfpogqbu3eqffwu4y.cpp:761:10: warning: variable ‘L__self___main_9_running_mean’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:14:31.4797326Z 761 | auto L__self___main_9_running_mean = constants_->at(15); 2024-06-26T08:14:31.4797883Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:14:31.4799472Z /tmp/tmp76y1ccx0/cczfur6bebhwsy3riytfcocs2qnzekeapeidnjau25lir3jzu4nm/csewirsb6txds6pa7iub5etu7jjpdawfojyyfpogqbu3eqffwu4y.cpp:762:10: warning: variable ‘L__self___main_9_running_var’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:14:31.4801064Z 762 | auto L__self___main_9_running_var = constants_->at(16); 2024-06-26T08:14:31.4801618Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:14:31.9545315Z pass 2024-06-26T08:14:31.9546019Z TIMING: backend_compile:7e-05 entire_frame_compile:0.13068 2024-06-26T08:14:31.9547356Z STATS: call_* op count: 13 | FakeTensor.__torch_dispatch__:211 | FakeTensorMode.__torch_dispatch__:1666 | ProxyTorchDispatchMode.__torch_dispatch__:320 2024-06-26T08:14:31.9548504Z Dynamo produced 1 graphs covering 13 ops with 0 graph breaks (0 unique) 2024-06-26T08:14:35.4408145Z 2024-06-26T08:14:38.0965283Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:14:38.0965762Z loading model: 0it [00:02, ?it/s] 2024-06-26T08:14:38.0984143Z cuda eval demucs 2024-06-26T08:17:51.6709065Z pass 2024-06-26T08:17:51.6808082Z TIMING: backend_compile:0.00019 entire_frame_compile:1.91512 2024-06-26T08:17:51.6811887Z STATS: call_* op count: 67 | FakeTensorMode.__torch_dispatch__:164976 | FakeTensor.__torch_dispatch__:15527 | ProxyTorchDispatchMode.__torch_dispatch__:38531 2024-06-26T08:17:51.6813277Z Dynamo produced 1 graphs covering 67 ops with 0 graph breaks (0 unique) 2024-06-26T08:17:58.1821692Z 2024-06-26T08:17:58.4280988Z loading model: 0it [00:00, ?it/s]Downloading: "https://download.pytorch.org/models/densenet121-a639ec97.pth" to /var/lib/jenkins/.cache/torch/hub/checkpoints/densenet121-a639ec97.pth 2024-06-26T08:17:58.4501186Z 2024-06-26T08:17:58.4501196Z 2024-06-26T08:17:58.5356831Z 0% 0.00/30.8M [00:00 2024-06-26T08:21:08.0523711Z [ 2024-06-26T08:21:08.0524508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/detection/core.py", line 92, in 2024-06-26T08:21:08.0525525Z cv2.morphologyEx(bmap[..., idx], cv2.MORPH_OPEN, self._opening_kernel) 2024-06-26T08:21:08.0525984Z 2024-06-26T08:21:08.0526300Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-06-26T08:21:08.0526755Z 2024-06-26T08:21:08.0527057Z TorchDynamo optimized model failed to run because of following error 2024-06-26T08:21:08.0527627Z fail_to_run 2024-06-26T08:21:08.0828365Z TIMING: 2024-06-26T08:21:08.0828987Z STATS: call_* op count: 0 | FakeTensor.__torch_dispatch__:605 | FakeTensorMode.__torch_dispatch__:3633 2024-06-26T08:21:08.0829889Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-06-26T08:21:11.5187471Z 2024-06-26T08:21:14.4887291Z loading model: 0it [00:00, ?it/s]WARNING:common:Model doctr_reco_predictor does not support bfloat16, running with amp instead 2024-06-26T08:21:14.7028819Z 2024-06-26T08:21:14.7029159Z loading model: 0it [00:03, ?it/s] 2024-06-26T08:21:14.7029868Z WARNING:common:Model doctr_reco_predictor does not support bfloat16, running with amp instead 2024-06-26T08:21:14.7030622Z cuda eval doctr_reco_predictor 2024-06-26T08:21:14.8630572Z WARNING:common:Model doctr_reco_predictor does not support bfloat16, running with amp instead 2024-06-26T08:21:15.9815837Z E0626 08:21:15.980000 140067371594368 torch/fx/experimental/recording.py:281] [0/0] failed while running evaluate_expr(*(Ne(u0, 123), None), **{'fx_node': ne}) 2024-06-26T08:21:15.9892728Z ERROR:common: 2024-06-26T08:21:15.9893119Z Traceback (most recent call last): 2024-06-26T08:21:15.9894479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/tensor.py", line 1040, in evaluate_expr 2024-06-26T08:21:15.9895715Z return guard_scalar(self.sym_num) 2024-06-26T08:21:15.9897304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 788, in guard_scalar 2024-06-26T08:21:15.9898228Z return guard_bool(a) 2024-06-26T08:21:15.9899506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 961, in guard_bool 2024-06-26T08:21:15.9900945Z return a.node.guard_bool("", 0) # NB: uses Python backtrace 2024-06-26T08:21:15.9902408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/sym_node.py", line 414, in guard_bool 2024-06-26T08:21:15.9904012Z r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node) 2024-06-26T08:21:15.9905620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/recording.py", line 275, in wrapper 2024-06-26T08:21:15.9907014Z return event.run(self) 2024-06-26T08:21:15.9908034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/recording.py", line 162, in run 2024-06-26T08:21:15.9908896Z return self.f(*args, **kwargs) 2024-06-26T08:21:15.9909902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5147, in evaluate_expr 2024-06-26T08:21:15.9910848Z concrete_val = compute_concrete_val() 2024-06-26T08:21:15.9911914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5128, in compute_concrete_val 2024-06-26T08:21:15.9912892Z return self.size_hint(orig_expr) 2024-06-26T08:21:15.9913883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 4666, in size_hint 2024-06-26T08:21:15.9914904Z raise self._make_data_dependent_error(result_expr, expr) 2024-06-26T08:21:15.9916252Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Ne(u0, 123) (unhinted: Ne(u0, 123)). (Size-like symbols: none) 2024-06-26T08:21:15.9917236Z 2024-06-26T08:21:15.9917520Z Potential framework code culprit (scroll up for full backtrace): 2024-06-26T08:21:15.9918688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/tensor.py", line 1040, in evaluate_expr 2024-06-26T08:21:15.9919573Z return guard_scalar(self.sym_num) 2024-06-26T08:21:15.9919861Z 2024-06-26T08:21:15.9920074Z For more information, run with TORCH_LOGS="dynamic" 2024-06-26T08:21:15.9920861Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-06-26T08:21:15.9921833Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-06-26T08:21:15.9923027Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-06-26T08:21:15.9923819Z 2024-06-26T08:21:15.9923978Z User Stack (most recent call last): 2024-06-26T08:21:15.9924434Z (snipped, see stack below for prefix) 2024-06-26T08:21:15.9925439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 224, in forward 2024-06-26T08:21:15.9926387Z out["preds"] = self.postprocessor(logits) 2024-06-26T08:21:15.9927407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 99, in __call__ 2024-06-26T08:21:15.9928562Z return self.ctc_best_path(logits=logits, vocab=self.vocab, blank=len(self.vocab)) 2024-06-26T08:21:15.9929777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 78, in ctc_best_path 2024-06-26T08:21:15.9930805Z words = [ 2024-06-26T08:21:15.9931692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-06-26T08:21:15.9932777Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-06-26T08:21:15.9934008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-06-26T08:21:15.9935081Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-06-26T08:21:15.9935556Z 2024-06-26T08:21:15.9935815Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-06-26T08:21:15.9936230Z 2024-06-26T08:21:15.9936525Z During handling of the above exception, another exception occurred: 2024-06-26T08:21:15.9936968Z 2024-06-26T08:21:15.9937120Z Traceback (most recent call last): 2024-06-26T08:21:15.9937825Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2616, in check_accuracy 2024-06-26T08:21:15.9938588Z optimized_model_iter_fn = optimize_ctx( 2024-06-26T08:21:15.9939400Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1222, in export_aot_inductor 2024-06-26T08:21:15.9940330Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-06-26T08:21:15.9941168Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1192, in load 2024-06-26T08:21:15.9941976Z gm = torch.export._trace._export_to_torch_ir( 2024-06-26T08:21:15.9943296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-06-26T08:21:15.9944554Z gm_torch_level, _ = torch._dynamo.export( 2024-06-26T08:21:15.9945679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1384, in inner 2024-06-26T08:21:15.9946672Z result_traced = opt_f(*args, **kwargs) 2024-06-26T08:21:15.9947686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:21:15.9948641Z return self._call_impl(*args, **kwargs) 2024-06-26T08:21:15.9949571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:21:15.9950411Z return forward_call(*args, **kwargs) 2024-06-26T08:21:15.9951277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 434, in _fn 2024-06-26T08:21:15.9952051Z return fn(*args, **kwargs) 2024-06-26T08:21:15.9952967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:21:15.9953859Z return self._call_impl(*args, **kwargs) 2024-06-26T08:21:15.9954782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:21:15.9955613Z return forward_call(*args, **kwargs) 2024-06-26T08:21:15.9956526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1121, in __call__ 2024-06-26T08:21:15.9957381Z return self._torchdynamo_orig_callable( 2024-06-26T08:21:15.9958346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 472, in __call__ 2024-06-26T08:21:15.9959139Z return _compile( 2024-06-26T08:21:15.9959969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_utils_internal.py", line 85, in wrapper_function 2024-06-26T08:21:15.9960941Z return StrobelightCompileTimeProfiler.profile_compile_time( 2024-06-26T08:21:15.9962128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_strobelight/compile_time_profiler.py", line 129, in profile_compile_time 2024-06-26T08:21:15.9963070Z return func(*args, **kwargs) 2024-06-26T08:21:15.9963684Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 79, in inner 2024-06-26T08:21:15.9964316Z return func(*args, **kwds) 2024-06-26T08:21:15.9965290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 817, in _compile 2024-06-26T08:21:15.9966227Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-06-26T08:21:15.9967246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 233, in time_wrapper 2024-06-26T08:21:15.9968048Z r = func(*args, **kwargs) 2024-06-26T08:21:15.9969005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 636, in compile_inner 2024-06-26T08:21:15.9969906Z out_code = transform_code_object(code, transform) 2024-06-26T08:21:15.9971019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1270, in transform_code_object 2024-06-26T08:21:15.9972001Z transformations(instructions, code_options) 2024-06-26T08:21:15.9972916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 178, in _fn 2024-06-26T08:21:15.9973701Z return fn(*args, **kwargs) 2024-06-26T08:21:15.9974569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 582, in transform 2024-06-26T08:21:15.9975361Z tracer.run() 2024-06-26T08:21:15.9976160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2476, in run 2024-06-26T08:21:15.9976950Z super().run() 2024-06-26T08:21:15.9977739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 904, in run 2024-06-26T08:21:15.9978583Z while self.step(): 2024-06-26T08:21:15.9979404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 816, in step 2024-06-26T08:21:15.9980261Z self.dispatch_table[inst.opcode](self, inst) 2024-06-26T08:21:15.9981211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in wrapper 2024-06-26T08:21:15.9982034Z return inner_fn(self, inst) 2024-06-26T08:21:15.9982957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1480, in CALL_FUNCTION 2024-06-26T08:21:15.9983821Z self.call_function(fn, args, {}) 2024-06-26T08:21:15.9984844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 754, in call_function 2024-06-26T08:21:15.9985773Z self.push(fn.call_function(self, args, kwargs)) 2024-06-26T08:21:15.9986999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/user_defined.py", line 785, in call_function 2024-06-26T08:21:15.9987977Z return self.call_method(tx, "__call__", args, kwargs) 2024-06-26T08:21:15.9989088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/user_defined.py", line 637, in call_method 2024-06-26T08:21:15.9990135Z return UserMethodVariable(method, self, source=source).call_function( 2024-06-26T08:21:15.9991274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 353, in call_function 2024-06-26T08:21:15.9992197Z return super().call_function(tx, args, kwargs) 2024-06-26T08:21:15.9993207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 295, in call_function 2024-06-26T08:21:15.9994129Z return super().call_function(tx, args, kwargs) 2024-06-26T08:21:15.9995137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 95, in call_function 2024-06-26T08:21:15.9996193Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-06-26T08:21:15.9997412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 760, in inline_user_function_return 2024-06-26T08:21:15.9998582Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-06-26T08:21:15.9999832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2691, in inline_call 2024-06-26T08:21:16.0000749Z return cls.inline_call_(parent, func, args, kwargs) 2024-06-26T08:21:16.0001769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2807, in inline_call_ 2024-06-26T08:21:16.0002597Z tracer.run() 2024-06-26T08:21:16.0003447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 904, in run 2024-06-26T08:21:16.0004240Z while self.step(): 2024-06-26T08:21:16.0005053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 816, in step 2024-06-26T08:21:16.0005903Z self.dispatch_table[inst.opcode](self, inst) 2024-06-26T08:21:16.0006871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in wrapper 2024-06-26T08:21:16.0007704Z return inner_fn(self, inst) 2024-06-26T08:21:16.0008706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1533, in CALL_FUNCTION_KW 2024-06-26T08:21:16.0009603Z self.call_function(fn, args, kwargs) 2024-06-26T08:21:16.0010593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 754, in call_function 2024-06-26T08:21:16.0011508Z self.push(fn.call_function(self, args, kwargs)) 2024-06-26T08:21:16.0012537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 295, in call_function 2024-06-26T08:21:16.0013471Z return super().call_function(tx, args, kwargs) 2024-06-26T08:21:16.0014476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 95, in call_function 2024-06-26T08:21:16.0015533Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-06-26T08:21:16.0016752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 760, in inline_user_function_return 2024-06-26T08:21:16.0017874Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-06-26T08:21:16.0019061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2691, in inline_call 2024-06-26T08:21:16.0019986Z return cls.inline_call_(parent, func, args, kwargs) 2024-06-26T08:21:16.0020999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2807, in inline_call_ 2024-06-26T08:21:16.0021829Z tracer.run() 2024-06-26T08:21:16.0022621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 904, in run 2024-06-26T08:21:16.0023408Z while self.step(): 2024-06-26T08:21:16.0024305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 816, in step 2024-06-26T08:21:16.0025169Z self.dispatch_table[inst.opcode](self, inst) 2024-06-26T08:21:16.0026129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in wrapper 2024-06-26T08:21:16.0027135Z return inner_fn(self, inst) 2024-06-26T08:21:16.0028060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1480, in CALL_FUNCTION 2024-06-26T08:21:16.0028926Z self.call_function(fn, args, {}) 2024-06-26T08:21:16.0029865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 754, in call_function 2024-06-26T08:21:16.0030782Z self.push(fn.call_function(self, args, kwargs)) 2024-06-26T08:21:16.0031786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 95, in call_function 2024-06-26T08:21:16.0032847Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-06-26T08:21:16.0034197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 760, in inline_user_function_return 2024-06-26T08:21:16.0035309Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-06-26T08:21:16.0036441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2691, in inline_call 2024-06-26T08:21:16.0037421Z return cls.inline_call_(parent, func, args, kwargs) 2024-06-26T08:21:16.0038513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2807, in inline_call_ 2024-06-26T08:21:16.0039360Z tracer.run() 2024-06-26T08:21:16.0040149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 904, in run 2024-06-26T08:21:16.0040934Z while self.step(): 2024-06-26T08:21:16.0041754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 816, in step 2024-06-26T08:21:16.0042619Z self.dispatch_table[inst.opcode](self, inst) 2024-06-26T08:21:16.0043571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 510, in wrapper 2024-06-26T08:21:16.0044378Z return inner_fn(self, inst) 2024-06-26T08:21:16.0045314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1480, in CALL_FUNCTION 2024-06-26T08:21:16.0046180Z self.call_function(fn, args, {}) 2024-06-26T08:21:16.0047121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 754, in call_function 2024-06-26T08:21:16.0048053Z self.push(fn.call_function(self, args, kwargs)) 2024-06-26T08:21:16.0049123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 95, in call_function 2024-06-26T08:21:16.0050197Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-06-26T08:21:16.0051435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 760, in inline_user_function_return 2024-06-26T08:21:16.0052565Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-06-26T08:21:16.0053705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2691, in inline_call 2024-06-26T08:21:16.0054651Z return cls.inline_call_(parent, func, args, kwargs) 2024-06-26T08:21:16.0055680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2807, in inline_call_ 2024-06-26T08:21:16.0056522Z tracer.run() 2024-06-26T08:21:16.0057311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 904, in run 2024-06-26T08:21:16.0058125Z while self.step(): 2024-06-26T08:21:16.0058948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 816, in step 2024-06-26T08:21:16.0059826Z self.dispatch_table[inst.opcode](self, inst) 2024-06-26T08:21:16.0060777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 464, in inner 2024-06-26T08:21:16.0061652Z eval_result = value.evaluate_expr(self.output) 2024-06-26T08:21:16.0062672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/tensor.py", line 1042, in evaluate_expr 2024-06-26T08:21:16.0063544Z raise UserError( # noqa: B904 2024-06-26T08:21:16.0064895Z torch._dynamo.exc.UserError: Consider annotating your code using torch._check*(). Could not guard on data-dependent expression Ne(u0, 123) (unhinted: Ne(u0, 123)). (Size-like symbols: none) 2024-06-26T08:21:16.0065911Z 2024-06-26T08:21:16.0066187Z Potential framework code culprit (scroll up for full backtrace): 2024-06-26T08:21:16.0067471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/variables/tensor.py", line 1040, in evaluate_expr 2024-06-26T08:21:16.0068494Z return guard_scalar(self.sym_num) 2024-06-26T08:21:16.0068787Z 2024-06-26T08:21:16.0068995Z For more information, run with TORCH_LOGS="dynamic" 2024-06-26T08:21:16.0069790Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-06-26T08:21:16.0070746Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-06-26T08:21:16.0072001Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-06-26T08:21:16.0072809Z 2024-06-26T08:21:16.0072954Z User Stack (most recent call last): 2024-06-26T08:21:16.0073397Z (snipped, see stack below for prefix) 2024-06-26T08:21:16.0074397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 224, in forward 2024-06-26T08:21:16.0075334Z out["preds"] = self.postprocessor(logits) 2024-06-26T08:21:16.0076353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 99, in __call__ 2024-06-26T08:21:16.0077468Z return self.ctc_best_path(logits=logits, vocab=self.vocab, blank=len(self.vocab)) 2024-06-26T08:21:16.0078747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 78, in ctc_best_path 2024-06-26T08:21:16.0079653Z words = [ 2024-06-26T08:21:16.0080546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-06-26T08:21:16.0081638Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-06-26T08:21:16.0082817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-06-26T08:21:16.0083890Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-06-26T08:21:16.0084366Z 2024-06-26T08:21:16.0084623Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-06-26T08:21:16.0085762Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-06-26T08:21:16.0086538Z 2024-06-26T08:21:16.0086661Z from user code: 2024-06-26T08:21:16.0087532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 224, in forward 2024-06-26T08:21:16.0088460Z out["preds"] = self.postprocessor(logits) 2024-06-26T08:21:16.0089465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 99, in __call__ 2024-06-26T08:21:16.0090558Z return self.ctc_best_path(logits=logits, vocab=self.vocab, blank=len(self.vocab)) 2024-06-26T08:21:16.0091787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 78, in ctc_best_path 2024-06-26T08:21:16.0092681Z words = [ 2024-06-26T08:21:16.0093556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-06-26T08:21:16.0094640Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-06-26T08:21:16.0095800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-06-26T08:21:16.0096857Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-06-26T08:21:16.0097329Z 2024-06-26T08:21:16.0097625Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-06-26T08:21:16.0098078Z 2024-06-26T08:21:16.0098374Z TorchDynamo optimized model failed to run because of following error 2024-06-26T08:21:16.0098936Z fail_to_run 2024-06-26T08:21:16.0099205Z TIMING: 2024-06-26T08:21:16.0099785Z STATS: call_* op count: 0 | FakeTensor.__torch_dispatch__:260 | FakeTensorMode.__torch_dispatch__:5942 2024-06-26T08:21:16.0100668Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-06-26T08:21:19.4549879Z 2024-06-26T08:21:20.8141535Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:21:20.8142276Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:21:20.8148872Z cuda eval drq 2024-06-26T08:21:21.1670551Z ERROR:common: 2024-06-26T08:21:21.1672878Z Traceback (most recent call last): 2024-06-26T08:21:21.1674211Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2616, in check_accuracy 2024-06-26T08:21:21.1675204Z optimized_model_iter_fn = optimize_ctx( 2024-06-26T08:21:21.1676387Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1222, in export_aot_inductor 2024-06-26T08:21:21.1677785Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-06-26T08:21:21.1679039Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1192, in load 2024-06-26T08:21:21.1680099Z gm = torch.export._trace._export_to_torch_ir( 2024-06-26T08:21:21.1681702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-06-26T08:21:21.1683044Z gm_torch_level, _ = torch._dynamo.export( 2024-06-26T08:21:21.1684486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1490, in inner 2024-06-26T08:21:21.1685718Z graph = rewrite_signature( 2024-06-26T08:21:21.1687165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1045, in rewrite_signature 2024-06-26T08:21:21.1688799Z check_user_input_output(flat_results_traced, UserErrorType.INVALID_OUTPUT) 2024-06-26T08:21:21.1690684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1031, in check_user_input_output 2024-06-26T08:21:21.1691969Z raise UserError( 2024-06-26T08:21:21.1693693Z torch._dynamo.exc.UserError: It looks like one of the outputs with type `` is not supported or pytree-flattenable. 2024-06-26T08:21:21.1698853Z Exported graphs outputs can only contain the following supported types: [, , , , , , , , , , , , , , , , , , ]. 2024-06-26T08:21:21.1703955Z If you are using a custom class object, please register a pytree_flatten/unflatten function using `torch.utils._pytree.register_pytree_node` or `torch.export.register_dataclass`. 2024-06-26T08:21:21.1706093Z TorchDynamo optimized model failed to run because of following error 2024-06-26T08:21:21.1707157Z fail_to_run 2024-06-26T08:21:21.1707905Z TIMING: backend_compile:7e-05 entire_frame_compile:0.18558 2024-06-26T08:21:21.1709215Z STATS: call_* op count: 29 | FakeTensorMode.__torch_dispatch__:236 | FakeTensor.__torch_dispatch__:64 2024-06-26T08:21:21.1710668Z Dynamo produced 1 graphs covering 29 ops with 0 graph breaks (0 unique) 2024-06-26T08:21:24.5160414Z 2024-06-26T08:21:25.2353924Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:21:25.2354388Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:21:25.2374199Z cuda eval functorch_dp_cifar10 2024-06-26T08:21:39.8492771Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp: In member function ‘void torch::aot_inductor::AOTInductorModel::run_impl(AtenTensorOpaque**, AtenTensorOpaque**, torch::aot_inductor::DeviceStreamType, AOTIProxyExecutorHandle)’: 2024-06-26T08:21:39.8496416Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1169:10: warning: variable ‘L__self___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8498208Z 1169 | auto L__self___bn1_weight = constants_->at(1); 2024-06-26T08:21:39.8498708Z | ^~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8500252Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1170:10: warning: variable ‘L__self___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8501877Z 1170 | auto L__self___bn1_bias = constants_->at(2); 2024-06-26T08:21:39.8502369Z | ^~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8504147Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1172:10: warning: variable ‘getattr_L__self___layer1___0___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8505851Z 1172 | auto getattr_L__self___layer1___0___bn1_weight = constants_->at(4); 2024-06-26T08:21:39.8506752Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8509677Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1173:10: warning: variable ‘getattr_L__self___layer1___0___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8511406Z 1173 | auto getattr_L__self___layer1___0___bn1_bias = constants_->at(5); 2024-06-26T08:21:39.8512029Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8513780Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1175:10: warning: variable ‘getattr_L__self___layer1___0___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8515506Z 1175 | auto getattr_L__self___layer1___0___bn2_weight = constants_->at(7); 2024-06-26T08:21:39.8516128Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8517797Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1176:10: warning: variable ‘getattr_L__self___layer1___0___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8519495Z 1176 | auto getattr_L__self___layer1___0___bn2_bias = constants_->at(8); 2024-06-26T08:21:39.8520115Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8521819Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1178:10: warning: variable ‘getattr_L__self___layer1___1___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8531701Z 1178 | auto getattr_L__self___layer1___1___bn1_weight = constants_->at(10); 2024-06-26T08:21:39.8532612Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8534446Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1179:10: warning: variable ‘getattr_L__self___layer1___1___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8536169Z 1179 | auto getattr_L__self___layer1___1___bn1_bias = constants_->at(11); 2024-06-26T08:21:39.8536786Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8538496Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1181:10: warning: variable ‘getattr_L__self___layer1___1___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8540210Z 1181 | auto getattr_L__self___layer1___1___bn2_weight = constants_->at(13); 2024-06-26T08:21:39.8540841Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8542540Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1182:10: warning: variable ‘getattr_L__self___layer1___1___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8544511Z 1182 | auto getattr_L__self___layer1___1___bn2_bias = constants_->at(14); 2024-06-26T08:21:39.8545124Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8547212Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1184:10: warning: variable ‘getattr_L__self___layer2___0___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8548968Z 1184 | auto getattr_L__self___layer2___0___bn1_weight = constants_->at(16); 2024-06-26T08:21:39.8549589Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8551273Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1185:10: warning: variable ‘getattr_L__self___layer2___0___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8552963Z 1185 | auto getattr_L__self___layer2___0___bn1_bias = constants_->at(17); 2024-06-26T08:21:39.8553600Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8555313Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1187:10: warning: variable ‘getattr_L__self___layer2___0___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8557030Z 1187 | auto getattr_L__self___layer2___0___bn2_weight = constants_->at(19); 2024-06-26T08:21:39.8557639Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8559307Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1188:10: warning: variable ‘getattr_L__self___layer2___0___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8560981Z 1188 | auto getattr_L__self___layer2___0___bn2_bias = constants_->at(20); 2024-06-26T08:21:39.8561579Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8563313Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1190:10: warning: variable ‘getattr_L__self___layer2___0___downsample_1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8565152Z 1190 | auto getattr_L__self___layer2___0___downsample_1_weight = constants_->at(22); 2024-06-26T08:21:39.8565843Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8567586Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1191:10: warning: variable ‘getattr_L__self___layer2___0___downsample_1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8569367Z 1191 | auto getattr_L__self___layer2___0___downsample_1_bias = constants_->at(23); 2024-06-26T08:21:39.8570027Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8571741Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1193:10: warning: variable ‘getattr_L__self___layer2___1___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8573446Z 1193 | auto getattr_L__self___layer2___1___bn1_weight = constants_->at(25); 2024-06-26T08:21:39.8574059Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8575735Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1194:10: warning: variable ‘getattr_L__self___layer2___1___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8577419Z 1194 | auto getattr_L__self___layer2___1___bn1_bias = constants_->at(26); 2024-06-26T08:21:39.8578023Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8579704Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1196:10: warning: variable ‘getattr_L__self___layer2___1___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8581494Z 1196 | auto getattr_L__self___layer2___1___bn2_weight = constants_->at(28); 2024-06-26T08:21:39.8582105Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8583888Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1197:10: warning: variable ‘getattr_L__self___layer2___1___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8585683Z 1197 | auto getattr_L__self___layer2___1___bn2_bias = constants_->at(29); 2024-06-26T08:21:39.8586290Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8588263Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1199:10: warning: variable ‘getattr_L__self___layer3___0___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8589982Z 1199 | auto getattr_L__self___layer3___0___bn1_weight = constants_->at(31); 2024-06-26T08:21:39.8590601Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8592292Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1200:10: warning: variable ‘getattr_L__self___layer3___0___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8594005Z 1200 | auto getattr_L__self___layer3___0___bn1_bias = constants_->at(32); 2024-06-26T08:21:39.8594633Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8596318Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1202:10: warning: variable ‘getattr_L__self___layer3___0___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8598038Z 1202 | auto getattr_L__self___layer3___0___bn2_weight = constants_->at(34); 2024-06-26T08:21:39.8598649Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8600317Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1203:10: warning: variable ‘getattr_L__self___layer3___0___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8602001Z 1203 | auto getattr_L__self___layer3___0___bn2_bias = constants_->at(35); 2024-06-26T08:21:39.8602608Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8604381Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1205:10: warning: variable ‘getattr_L__self___layer3___0___downsample_1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8606160Z 1205 | auto getattr_L__self___layer3___0___downsample_1_weight = constants_->at(37); 2024-06-26T08:21:39.8606830Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8608564Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1206:10: warning: variable ‘getattr_L__self___layer3___0___downsample_1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8610331Z 1206 | auto getattr_L__self___layer3___0___downsample_1_bias = constants_->at(38); 2024-06-26T08:21:39.8610986Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8612688Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1208:10: warning: variable ‘getattr_L__self___layer3___1___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8614443Z 1208 | auto getattr_L__self___layer3___1___bn1_weight = constants_->at(40); 2024-06-26T08:21:39.8615050Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8616851Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1209:10: warning: variable ‘getattr_L__self___layer3___1___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8618530Z 1209 | auto getattr_L__self___layer3___1___bn1_bias = constants_->at(41); 2024-06-26T08:21:39.8619193Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8620873Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1211:10: warning: variable ‘getattr_L__self___layer3___1___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8622572Z 1211 | auto getattr_L__self___layer3___1___bn2_weight = constants_->at(43); 2024-06-26T08:21:39.8623182Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8624991Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1212:10: warning: variable ‘getattr_L__self___layer3___1___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8626866Z 1212 | auto getattr_L__self___layer3___1___bn2_bias = constants_->at(44); 2024-06-26T08:21:39.8627477Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8629188Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1214:10: warning: variable ‘getattr_L__self___layer4___0___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8630887Z 1214 | auto getattr_L__self___layer4___0___bn1_weight = constants_->at(46); 2024-06-26T08:21:39.8631502Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8633170Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1215:10: warning: variable ‘getattr_L__self___layer4___0___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8634855Z 1215 | auto getattr_L__self___layer4___0___bn1_bias = constants_->at(47); 2024-06-26T08:21:39.8635456Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8637135Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1217:10: warning: variable ‘getattr_L__self___layer4___0___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8638843Z 1217 | auto getattr_L__self___layer4___0___bn2_weight = constants_->at(49); 2024-06-26T08:21:39.8639458Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8641115Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1218:10: warning: variable ‘getattr_L__self___layer4___0___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8642805Z 1218 | auto getattr_L__self___layer4___0___bn2_bias = constants_->at(50); 2024-06-26T08:21:39.8643408Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8645185Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1220:10: warning: variable ‘getattr_L__self___layer4___0___downsample_1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8646965Z 1220 | auto getattr_L__self___layer4___0___downsample_1_weight = constants_->at(52); 2024-06-26T08:21:39.8647631Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8649365Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1221:10: warning: variable ‘getattr_L__self___layer4___0___downsample_1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8651249Z 1221 | auto getattr_L__self___layer4___0___downsample_1_bias = constants_->at(53); 2024-06-26T08:21:39.8651906Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8653688Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1223:10: warning: variable ‘getattr_L__self___layer4___1___bn1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8655382Z 1223 | auto getattr_L__self___layer4___1___bn1_weight = constants_->at(55); 2024-06-26T08:21:39.8655994Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8657671Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1224:10: warning: variable ‘getattr_L__self___layer4___1___bn1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8659344Z 1224 | auto getattr_L__self___layer4___1___bn1_bias = constants_->at(56); 2024-06-26T08:21:39.8659956Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8661621Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1226:10: warning: variable ‘getattr_L__self___layer4___1___bn2_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8663323Z 1226 | auto getattr_L__self___layer4___1___bn2_weight = constants_->at(58); 2024-06-26T08:21:39.8663931Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:39.8665686Z /tmp/tmp76y1ccx0/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cpxtukpdttqcienhlibyqeuiicfcaowe3n26pnt52yanssf5rbrz.cpp:1227:10: warning: variable ‘getattr_L__self___layer4___1___bn2_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:39.8667567Z 1227 | auto getattr_L__self___layer4___1___bn2_bias = constants_->at(59); 2024-06-26T08:21:39.8668168Z | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-06-26T08:21:40.5300778Z pass 2024-06-26T08:21:40.5336327Z TIMING: backend_compile:7e-05 entire_frame_compile:0.33948 2024-06-26T08:21:40.5337646Z STATS: call_* op count: 69 | FakeTensor.__torch_dispatch__:926 | FakeTensorMode.__torch_dispatch__:5564 | ProxyTorchDispatchMode.__torch_dispatch__:1397 2024-06-26T08:21:40.5338755Z Dynamo produced 1 graphs covering 69 ops with 0 graph breaks (0 unique) 2024-06-26T08:21:44.1839741Z 2024-06-26T08:21:44.6455647Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:21:44.6456127Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:21:44.6461718Z cuda eval functorch_maml_omniglot 2024-06-26T08:21:54.6955104Z /tmp/tmp76y1ccx0/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/ck62xpwefo45wbwkhv5w4c3hzznrzkscm4gjb73hiqhnteqz7muw.cpp: In member function ‘void torch::aot_inductor::AOTInductorModel::run_impl(AtenTensorOpaque**, AtenTensorOpaque**, torch::aot_inductor::DeviceStreamType, AOTIProxyExecutorHandle)’: 2024-06-26T08:21:54.6958123Z /tmp/tmp76y1ccx0/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/ck62xpwefo45wbwkhv5w4c3hzznrzkscm4gjb73hiqhnteqz7muw.cpp:714:10: warning: variable ‘fn_1_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:54.6959585Z 714 | auto fn_1_weight = constants_->at(2); 2024-06-26T08:21:54.6960056Z | ^~~~~~~~~~~ 2024-06-26T08:21:54.6961522Z /tmp/tmp76y1ccx0/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/ck62xpwefo45wbwkhv5w4c3hzznrzkscm4gjb73hiqhnteqz7muw.cpp:715:10: warning: variable ‘fn_1_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:54.6962965Z 715 | auto fn_1_bias = constants_->at(3); 2024-06-26T08:21:54.6963466Z | ^~~~~~~~~ 2024-06-26T08:21:54.6964916Z /tmp/tmp76y1ccx0/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/ck62xpwefo45wbwkhv5w4c3hzznrzkscm4gjb73hiqhnteqz7muw.cpp:718:10: warning: variable ‘fn_5_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:54.6966376Z 718 | auto fn_5_weight = constants_->at(6); 2024-06-26T08:21:54.6967100Z | ^~~~~~~~~~~ 2024-06-26T08:21:54.6968558Z /tmp/tmp76y1ccx0/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/ck62xpwefo45wbwkhv5w4c3hzznrzkscm4gjb73hiqhnteqz7muw.cpp:719:10: warning: variable ‘fn_5_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:54.6969985Z 719 | auto fn_5_bias = constants_->at(7); 2024-06-26T08:21:54.6970530Z | ^~~~~~~~~ 2024-06-26T08:21:54.6971987Z /tmp/tmp76y1ccx0/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/ck62xpwefo45wbwkhv5w4c3hzznrzkscm4gjb73hiqhnteqz7muw.cpp:722:10: warning: variable ‘fn_9_weight’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:54.6973442Z 722 | auto fn_9_weight = constants_->at(10); 2024-06-26T08:21:54.6973904Z | ^~~~~~~~~~~ 2024-06-26T08:21:54.6975353Z /tmp/tmp76y1ccx0/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/ck62xpwefo45wbwkhv5w4c3hzznrzkscm4gjb73hiqhnteqz7muw.cpp:723:10: warning: variable ‘fn_9_bias’ set but not used [-Wunused-but-set-variable] 2024-06-26T08:21:54.6976791Z 723 | auto fn_9_bias = constants_->at(11); 2024-06-26T08:21:54.6977241Z | ^~~~~~~~~ 2024-06-26T08:21:55.1564686Z pass 2024-06-26T08:21:55.1569734Z TIMING: backend_compile:7e-05 entire_frame_compile:0.12788 2024-06-26T08:21:55.1571942Z STATS: call_* op count: 14 | FakeTensor.__torch_dispatch__:148 | FakeTensorMode.__torch_dispatch__:1557 | ProxyTorchDispatchMode.__torch_dispatch__:282 2024-06-26T08:21:55.1573904Z Dynamo produced 1 graphs covering 14 ops with 0 graph breaks (0 unique) 2024-06-26T08:21:58.6698306Z 2024-06-26T08:21:59.8656217Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:21:59.8656928Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:21:59.8663010Z cuda eval hf_Albert 2024-06-26T08:22:21.8449378Z pass 2024-06-26T08:22:21.8611366Z TIMING: backend_compile:0.00015 entire_frame_compile:1.21018 2024-06-26T08:22:21.8613807Z STATS: call_* op count: 436 | FakeTensorMode.__torch_dispatch__:21455 | FakeTensor.__torch_dispatch__:2978 | ProxyTorchDispatchMode.__torch_dispatch__:6078 2024-06-26T08:22:21.8615007Z Dynamo produced 1 graphs covering 436 ops with 0 graph breaks (0 unique) 2024-06-26T08:22:25.6699912Z 2024-06-26T08:22:29.3201404Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:22:29.3201896Z loading model: 0it [00:03, ?it/s] 2024-06-26T08:22:29.3264530Z cuda eval hf_Bart 2024-06-26T08:23:03.5100560Z pass 2024-06-26T08:23:03.5286118Z TIMING: backend_compile:0.00014 entire_frame_compile:1.59988 2024-06-26T08:23:03.5288835Z STATS: call_* op count: 451 | FakeTensorMode.__torch_dispatch__:28598 | FakeTensor.__torch_dispatch__:3875 | ProxyTorchDispatchMode.__torch_dispatch__:7744 2024-06-26T08:23:03.5290040Z Dynamo produced 1 graphs covering 451 ops with 0 graph breaks (0 unique) 2024-06-26T08:23:07.5177569Z 2024-06-26T08:23:09.8859929Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:23:09.8860400Z loading model: 0it [00:02, ?it/s] 2024-06-26T08:23:09.8911423Z cuda eval hf_Bert 2024-06-26T08:23:37.5410125Z pass 2024-06-26T08:23:37.5601474Z TIMING: backend_compile:0.00013 entire_frame_compile:2.6871 2024-06-26T08:23:37.5604330Z STATS: call_* op count: 367 | FakeTensorMode.__torch_dispatch__:23323 | FakeTensor.__torch_dispatch__:2959 | ProxyTorchDispatchMode.__torch_dispatch__:6652 2024-06-26T08:23:37.5605600Z Dynamo produced 1 graphs covering 367 ops with 0 graph breaks (0 unique) 2024-06-26T08:23:41.4415428Z 2024-06-26T08:23:45.8144079Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:23:45.8144635Z loading model: 0it [00:04, ?it/s] 2024-06-26T08:23:45.8239281Z cuda eval hf_Bert_large 2024-06-26T08:24:35.5764017Z pass 2024-06-26T08:24:35.6113307Z TIMING: backend_compile:0.00021 entire_frame_compile:5.12722 2024-06-26T08:24:35.6117501Z STATS: call_* op count: 715 | FakeTensorMode.__torch_dispatch__:45187 | FakeTensor.__torch_dispatch__:5755 | ProxyTorchDispatchMode.__torch_dispatch__:13012 2024-06-26T08:24:35.6119852Z Dynamo produced 1 graphs covering 715 ops with 0 graph breaks (0 unique) 2024-06-26T08:24:39.8270687Z 2024-06-26T08:24:42.8228605Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:24:42.8229099Z loading model: 0it [00:02, ?it/s] 2024-06-26T08:24:42.8282090Z cuda eval hf_BigBird 2024-06-26T08:25:38.9947121Z W0626 08:25:38.994000 140570627117696 torch/_inductor/utils.py:1234] DeviceCopy in input program 2024-06-26T08:25:40.1864672Z W0626 08:25:40.185000 140570627117696 torch/_inductor/utils.py:1234] DeviceCopy in input program 2024-06-26T08:25:41.1263730Z W0626 08:25:41.125000 140570627117696 torch/_inductor/utils.py:1234] DeviceCopy in input program 2024-06-26T08:25:41.7169013Z W0626 08:25:41.716000 140570627117696 torch/_inductor/utils.py:1234] DeviceCopy in input program 2024-06-26T08:25:42.3077163Z W0626 08:25:42.307000 140570627117696 torch/_inductor/utils.py:1234] DeviceCopy in input program 2024-06-26T08:25:42.8980641Z W0626 08:25:42.897000 140570627117696 torch/_inductor/utils.py:1234] DeviceCopy in input program 2024-06-26T08:25:43.4879973Z W0626 08:25:43.487000 140570627117696 torch/_inductor/utils.py:1234] DeviceCopy in input program 2024-06-26T08:25:44.0767087Z W0626 08:25:44.076000 140570627117696 torch/_inductor/utils.py:1234] DeviceCopy in input program 2024-06-26T08:25:44.6643494Z W0626 08:25:44.663000 140570627117696 torch/_inductor/utils.py:1234] DeviceCopy in input program 2024-06-26T08:25:45.2526562Z W0626 08:25:45.252000 140570627117696 torch/_inductor/utils.py:1234] DeviceCopy in input program 2024-06-26T08:25:45.8420976Z W0626 08:25:45.841000 140570627117696 torch/_inductor/utils.py:1234] DeviceCopy in input program 2024-06-26T08:25:46.4323972Z W0626 08:25:46.431000 140570627117696 torch/_inductor/utils.py:1234] DeviceCopy in input program 2024-06-26T08:26:53.2300774Z fail_accuracy 2024-06-26T08:26:53.2585023Z TIMING: backend_compile:0.00064 entire_frame_compile:4.8769 2024-06-26T08:26:53.2588073Z STATS: call_* op count: 3347 | FakeTensorMode.__torch_dispatch__:118836 | FakeTensor.__torch_dispatch__:9960 | ProxyTorchDispatchMode.__torch_dispatch__:30534 2024-06-26T08:26:53.2589258Z Dynamo produced 1 graphs covering 3347 ops with 0 graph breaks (0 unique) 2024-06-26T08:26:59.4636186Z 2024-06-26T08:27:01.3507463Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:27:01.3508058Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:27:01.3534667Z cuda eval hf_DistilBert 2024-06-26T08:27:21.3601055Z pass 2024-06-26T08:27:21.3762181Z TIMING: backend_compile:0.0001 entire_frame_compile:0.7306 2024-06-26T08:27:21.3763458Z STATS: call_* op count: 203 | FakeTensor.__torch_dispatch__:1710 | FakeTensorMode.__torch_dispatch__:13433 | ProxyTorchDispatchMode.__torch_dispatch__:3595 2024-06-26T08:27:21.3764830Z Dynamo produced 1 graphs covering 203 ops with 0 graph breaks (0 unique) 2024-06-26T08:27:25.1037014Z 2024-06-26T08:27:28.0883124Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:27:28.0883633Z loading model: 0it [00:02, ?it/s] 2024-06-26T08:27:28.0923258Z cuda eval hf_GPT2 2024-06-26T08:27:58.2896189Z pass 2024-06-26T08:27:58.3103826Z TIMING: backend_compile:0.0002 entire_frame_compile:2.2125 2024-06-26T08:27:58.3105147Z STATS: call_* op count: 622 | FakeTensorMode.__torch_dispatch__:21988 | FakeTensor.__torch_dispatch__:2394 | ProxyTorchDispatchMode.__torch_dispatch__:5857 2024-06-26T08:27:58.3106329Z Dynamo produced 1 graphs covering 622 ops with 0 graph breaks (0 unique) 2024-06-26T08:28:02.2585534Z 2024-06-26T08:28:14.4602398Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:28:14.4602878Z loading model: 0it [00:12, ?it/s] 2024-06-26T08:28:14.4719634Z cuda eval hf_GPT2_large 2024-06-26T08:28:14.4722752Z pass_due_to_skip 2024-06-26T08:28:14.8067251Z TIMING: 2024-06-26T08:28:14.8067572Z STATS: call_* op count: 0 2024-06-26T08:28:14.8068200Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-06-26T08:28:18.0903610Z 2024-06-26T08:28:20.0942109Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:28:20.0943270Z loading model: 0it [00:02, ?it/s] 2024-06-26T08:28:20.0979032Z cuda eval hf_T5 2024-06-26T08:29:00.9568003Z pass 2024-06-26T08:29:01.0101539Z TIMING: backend_compile:0.00022 entire_frame_compile:2.99405 2024-06-26T08:29:01.0104254Z STATS: call_* op count: 827 | FakeTensorMode.__torch_dispatch__:29287 | FakeTensor.__torch_dispatch__:3262 | ProxyTorchDispatchMode.__torch_dispatch__:8288 2024-06-26T08:29:01.0107970Z Dynamo produced 1 graphs covering 827 ops with 0 graph breaks (0 unique) 2024-06-26T08:29:04.9696398Z 2024-06-26T08:29:08.8496568Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:29:08.8497153Z loading model: 0it [00:03, ?it/s] 2024-06-26T08:29:08.8497660Z cuda eval hf_T5_base 2024-06-26T08:29:08.8508047Z Traceback (most recent call last): 2024-06-26T08:29:08.8508900Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2321, in validate_model 2024-06-26T08:29:08.8509664Z self.model_iter_fn(model, example_inputs) 2024-06-26T08:29:08.8510574Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 447, in forward_pass 2024-06-26T08:29:08.8511346Z return mod(*inputs) 2024-06-26T08:29:08.8512370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:29:08.8513512Z return self._call_impl(*args, **kwargs) 2024-06-26T08:29:08.8514700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:29:08.8515881Z return forward_call(*args, **kwargs) 2024-06-26T08:29:08.8516927Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/framework/huggingface/model_factory.py", line 69, in forward 2024-06-26T08:29:08.8518103Z return self.model(input_ids=input_ids, decoder_input_ids=decoder_input_ids) 2024-06-26T08:29:08.8519290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:29:08.8520216Z return self._call_impl(*args, **kwargs) 2024-06-26T08:29:08.8521172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:29:08.8522028Z return forward_call(*args, **kwargs) 2024-06-26T08:29:08.8523026Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/framework/huggingface/model_factory.py", line 69, in forward 2024-06-26T08:29:08.8524179Z return self.model(input_ids=input_ids, decoder_input_ids=decoder_input_ids) 2024-06-26T08:29:08.8525366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:29:08.8526286Z return self._call_impl(*args, **kwargs) 2024-06-26T08:29:08.8527231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:29:08.8528081Z return forward_call(*args, **kwargs) 2024-06-26T08:29:08.8529067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1748, in forward 2024-06-26T08:29:08.8529950Z decoder_outputs = self.decoder( 2024-06-26T08:29:08.8530915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:29:08.8531811Z return self._call_impl(*args, **kwargs) 2024-06-26T08:29:08.8532752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:29:08.8533610Z return forward_call(*args, **kwargs) 2024-06-26T08:29:08.8534580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1115, in forward 2024-06-26T08:29:08.8535448Z layer_outputs = layer_module( 2024-06-26T08:29:08.8536392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:29:08.8537594Z return self._call_impl(*args, **kwargs) 2024-06-26T08:29:08.8538545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:29:08.8539393Z return forward_call(*args, **kwargs) 2024-06-26T08:29:08.8540501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 695, in forward 2024-06-26T08:29:08.8541415Z self_attention_outputs = self.layer[0]( 2024-06-26T08:29:08.8542415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:29:08.8543318Z return self._call_impl(*args, **kwargs) 2024-06-26T08:29:08.8544394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:29:08.8545252Z return forward_call(*args, **kwargs) 2024-06-26T08:29:08.8546223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 602, in forward 2024-06-26T08:29:08.8547548Z attention_output = self.SelfAttention( 2024-06-26T08:29:08.8548560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T08:29:08.8549449Z return self._call_impl(*args, **kwargs) 2024-06-26T08:29:08.8550382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:29:08.8551221Z return forward_call(*args, **kwargs) 2024-06-26T08:29:08.8552177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 562, in forward 2024-06-26T08:29:08.8553255Z attn_weights = nn.functional.softmax(scores.float(), dim=-1).type_as( 2024-06-26T08:29:08.8556624Z torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 768.00 MiB. GPU 0 has a total capacity of 21.98 GiB of which 58.44 MiB is free. Process 10331 has 21.91 GiB memory in use. Of the allocated memory 20.94 GiB is allocated by PyTorch, and 666.49 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) 2024-06-26T08:29:08.8559302Z 2024-06-26T08:29:08.8559615Z The above exception was the direct cause of the following exception: 2024-06-26T08:29:08.8560070Z 2024-06-26T08:29:08.8560223Z Traceback (most recent call last): 2024-06-26T08:29:08.8560882Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4229, in run 2024-06-26T08:29:08.8561558Z ) = runner.load_model( 2024-06-26T08:29:08.8562237Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 380, in load_model 2024-06-26T08:29:08.8563017Z self.validate_model(model, example_inputs) 2024-06-26T08:29:08.8563780Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2323, in validate_model 2024-06-26T08:29:08.8564558Z raise RuntimeError("Eager run failed") from e 2024-06-26T08:29:08.8565037Z RuntimeError: Eager run failed 2024-06-26T08:29:08.8565295Z 2024-06-26T08:29:08.8565418Z eager_fail_to_run 2024-06-26T08:29:12.1702440Z 2024-06-26T08:29:21.4020109Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:29:21.4020627Z loading model: 0it [00:09, ?it/s] 2024-06-26T08:29:21.4476911Z cuda eval hf_T5_large 2024-06-26T08:29:21.4480754Z pass_due_to_skip 2024-06-26T08:29:21.7415290Z TIMING: 2024-06-26T08:29:21.7415727Z STATS: call_* op count: 0 2024-06-26T08:29:21.7416290Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-06-26T08:29:25.0102189Z 2024-06-26T08:29:26.0135674Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:29:26.0136261Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:29:26.0156994Z cuda eval hf_Whisper 2024-06-26T08:29:43.0476645Z pass 2024-06-26T08:29:43.0541738Z TIMING: backend_compile:9e-05 entire_frame_compile:0.45923 2024-06-26T08:29:43.0542944Z STATS: call_* op count: 115 | FakeTensor.__torch_dispatch__:1108 | FakeTensorMode.__torch_dispatch__:7778 | ProxyTorchDispatchMode.__torch_dispatch__:2046 2024-06-26T08:29:43.0544212Z Dynamo produced 1 graphs covering 115 ops with 0 graph breaks (0 unique) 2024-06-26T08:29:46.7514926Z 2024-06-26T08:29:51.1043723Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:29:51.1044204Z loading model: 0it [00:04, ?it/s] 2024-06-26T08:29:51.1139692Z cuda eval hf_distil_whisper 2024-06-26T08:31:01.1196893Z pass 2024-06-26T08:31:01.1471557Z TIMING: backend_compile:0.00018 entire_frame_compile:2.06156 2024-06-26T08:31:01.1472663Z STATS: call_* op count: 635 | FakeTensor.__torch_dispatch__:5948 | FakeTensorMode.__torch_dispatch__:40634 | ProxyTorchDispatchMode.__torch_dispatch__:11226 2024-06-26T08:31:01.1473795Z Dynamo produced 1 graphs covering 635 ops with 0 graph breaks (0 unique) 2024-06-26T08:31:05.4400246Z 2024-06-26T08:31:05.7837071Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:31:05.7837550Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:31:05.7841471Z cuda eval lennard_jones 2024-06-26T08:31:15.7046954Z pass 2024-06-26T08:31:15.7050075Z TIMING: backend_compile:7e-05 entire_frame_compile:0.09372 2024-06-26T08:31:15.7051178Z STATS: call_* op count: 9 | FakeTensor.__torch_dispatch__:77 | FakeTensorMode.__torch_dispatch__:1072 | ProxyTorchDispatchMode.__torch_dispatch__:219 2024-06-26T08:31:15.7052278Z Dynamo produced 1 graphs covering 9 ops with 0 graph breaks (0 unique) 2024-06-26T08:31:16.5687688Z accuracy pass_rate=51.16% 2024-06-26T08:31:16.5692144Z calls_captured gmean=0.00x mean=212.395x 2024-06-26T08:31:16.5695962Z unique_graphs gmean=0.00x mean=0.581x 2024-06-26T08:31:16.5699936Z graph_breaks gmean=0.00x mean=0.000x 2024-06-26T08:31:16.5704387Z unique_graph_breaks gmean=0.00x mean=0.000x 2024-06-26T08:31:16.5708284Z autograd_captures gmean=0.00x mean=0.000x 2024-06-26T08:31:16.5712707Z autograd_compiles gmean=0.00x mean=0.000x 2024-06-26T08:31:16.5716772Z cudagraph_skips gmean=0.00x mean=0.000x 2024-06-26T08:31:17.9052413Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy//aot_inductor_torchbench_inference.csv 2024-06-26T08:31:18.2314805Z torchrec_dlrm XFAIL 2024-06-26T08:31:18.2319153Z BERT_pytorch XFAIL 2024-06-26T08:31:18.2324238Z Background_Matting XFAIL 2024-06-26T08:31:18.2328817Z LearningToPaint PASS 2024-06-26T08:31:18.2333198Z Super_SloMo PASS 2024-06-26T08:31:18.2338292Z alexnet PASS 2024-06-26T08:31:18.2342315Z basic_gnn_edgecnn PASS 2024-06-26T08:31:18.2347255Z basic_gnn_gcn XFAIL 2024-06-26T08:31:18.2352598Z basic_gnn_gin PASS 2024-06-26T08:31:18.2356385Z basic_gnn_sage PASS 2024-06-26T08:31:18.2361068Z dcgan PASS 2024-06-26T08:31:18.2365205Z demucs PASS 2024-06-26T08:31:18.2369757Z densenet121 PASS 2024-06-26T08:31:18.2374127Z detectron2_fasterrcnn_r_101_c4 XFAIL 2024-06-26T08:31:18.2378620Z detectron2_fasterrcnn_r_101_dc5 XFAIL 2024-06-26T08:31:18.2383151Z detectron2_fasterrcnn_r_101_fpn XFAIL 2024-06-26T08:31:18.2388269Z detectron2_fasterrcnn_r_50_c4 XFAIL 2024-06-26T08:31:18.2392683Z detectron2_fasterrcnn_r_50_dc5 XFAIL 2024-06-26T08:31:18.2396967Z detectron2_fasterrcnn_r_50_fpn XFAIL 2024-06-26T08:31:18.2401443Z detectron2_maskrcnn_r_101_c4 XFAIL 2024-06-26T08:31:18.2405960Z detectron2_maskrcnn_r_101_fpn XFAIL 2024-06-26T08:31:18.2410431Z detectron2_maskrcnn_r_50_c4 XFAIL 2024-06-26T08:31:18.2414923Z detectron2_maskrcnn_r_50_fpn XFAIL 2024-06-26T08:31:18.2419463Z dlrm PASS 2024-06-26T08:31:18.2423935Z doctr_det_predictor XFAIL 2024-06-26T08:31:18.2428663Z doctr_reco_predictor XFAIL 2024-06-26T08:31:18.2433003Z drq XFAIL 2024-06-26T08:31:18.2437564Z functorch_dp_cifar10 PASS 2024-06-26T08:31:18.2441769Z functorch_maml_omniglot PASS 2024-06-26T08:31:18.2446553Z hf_Albert PASS 2024-06-26T08:31:18.2450710Z hf_Bart PASS 2024-06-26T08:31:18.2455269Z hf_Bert PASS 2024-06-26T08:31:18.2459730Z hf_Bert_large PASS 2024-06-26T08:31:18.2464217Z hf_BigBird XFAIL 2024-06-26T08:31:18.2469102Z hf_DistilBert PASS 2024-06-26T08:31:18.2473624Z hf_GPT2 PASS 2024-06-26T08:31:18.2477978Z hf_GPT2_large XFAIL 2024-06-26T08:31:18.2482446Z hf_T5 PASS 2024-06-26T08:31:18.2486844Z hf_T5_base XFAIL 2024-06-26T08:31:18.2491361Z hf_T5_large XFAIL 2024-06-26T08:31:18.2495883Z hf_Whisper PASS 2024-06-26T08:31:18.2500390Z hf_distil_whisper PASS 2024-06-26T08:31:18.2504913Z lennard_jones PASS 2024-06-26T08:31:18.2958332Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy//aot_inductor_torchbench_inference.csv 2024-06-26T08:31:18.6081042Z torchrec_dlrm PASS 2024-06-26T08:31:18.6084970Z BERT_pytorch PASS 2024-06-26T08:31:18.6089789Z Background_Matting PASS 2024-06-26T08:31:18.6095200Z LearningToPaint PASS 2024-06-26T08:31:18.6100351Z Super_SloMo PASS 2024-06-26T08:31:18.6105945Z alexnet PASS 2024-06-26T08:31:18.6111755Z basic_gnn_edgecnn PASS 2024-06-26T08:31:18.6116912Z basic_gnn_gcn PASS 2024-06-26T08:31:18.6121859Z basic_gnn_gin PASS 2024-06-26T08:31:18.6127115Z basic_gnn_sage PASS 2024-06-26T08:31:18.6132374Z dcgan PASS 2024-06-26T08:31:18.6137508Z demucs PASS 2024-06-26T08:31:18.6142545Z densenet121 PASS 2024-06-26T08:31:18.6148093Z detectron2_fasterrcnn_r_101_c4 PASS 2024-06-26T08:31:18.6153363Z detectron2_fasterrcnn_r_101_dc5 PASS 2024-06-26T08:31:18.6158534Z detectron2_fasterrcnn_r_101_fpn PASS 2024-06-26T08:31:18.6163640Z detectron2_fasterrcnn_r_50_c4 PASS 2024-06-26T08:31:18.6168878Z detectron2_fasterrcnn_r_50_dc5 PASS 2024-06-26T08:31:18.6174003Z detectron2_fasterrcnn_r_50_fpn PASS 2024-06-26T08:31:18.6179117Z detectron2_maskrcnn_r_101_c4 PASS 2024-06-26T08:31:18.6184171Z detectron2_maskrcnn_r_101_fpn PASS 2024-06-26T08:31:18.6189865Z detectron2_maskrcnn_r_50_c4 PASS 2024-06-26T08:31:18.6194917Z detectron2_maskrcnn_r_50_fpn PASS 2024-06-26T08:31:18.6200158Z dlrm PASS 2024-06-26T08:31:18.6205235Z doctr_det_predictor PASS 2024-06-26T08:31:18.6210373Z doctr_reco_predictor PASS 2024-06-26T08:31:18.6215485Z drq PASS 2024-06-26T08:31:18.6220590Z functorch_dp_cifar10 PASS 2024-06-26T08:31:18.6225834Z functorch_maml_omniglot PASS 2024-06-26T08:31:18.6231372Z hf_Albert PASS 2024-06-26T08:31:18.6236310Z hf_Bart PASS 2024-06-26T08:31:18.6241455Z hf_Bert PASS 2024-06-26T08:31:18.6246607Z hf_Bert_large PASS 2024-06-26T08:31:18.6251788Z hf_BigBird PASS 2024-06-26T08:31:18.6256931Z hf_DistilBert PASS 2024-06-26T08:31:18.6262027Z hf_GPT2 PASS 2024-06-26T08:31:18.6267777Z hf_GPT2_large PASS 2024-06-26T08:31:18.6272839Z hf_T5 PASS 2024-06-26T08:31:18.6277862Z hf_T5_base PASS 2024-06-26T08:31:18.6283104Z hf_T5_large PASS 2024-06-26T08:31:18.6288216Z hf_Whisper PASS 2024-06-26T08:31:18.6293519Z hf_distil_whisper PASS 2024-06-26T08:31:18.6298385Z lennard_jones PASS 2024-06-26T08:31:18.6760334Z + cleanup_workspace 2024-06-26T08:31:18.6761391Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-06-26T08:31:18.6763240Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-06-26T08:31:18.6764455Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-06-26T08:31:18.6765115Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-06-26T08:31:18.6765952Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-06-26T08:31:18.6766847Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-06-26T08:31:18.6767540Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-06-26T08:31:19.3041226Z ##[group]Run cat test/**/*_toprint.log || true 2024-06-26T08:31:19.3041761Z cat test/**/*_toprint.log || true 2024-06-26T08:31:19.3049848Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:31:19.3050352Z env: 2024-06-26T08:31:19.3050638Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:19.3051104Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:19.3051848Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:19.3052493Z ##[endgroup] 2024-06-26T08:31:19.3114949Z cat: test/**/*_toprint.log: No such file or directory 2024-06-26T08:31:19.3190873Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-06-26T08:31:19.3191390Z kill "$MONITOR_SCRIPT_PID" 2024-06-26T08:31:19.3198248Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:31:19.3198743Z env: 2024-06-26T08:31:19.3199027Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:19.3199487Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:19.3200228Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:19.3200885Z MONITOR_SCRIPT_PID: 31734 2024-06-26T08:31:19.3201247Z ##[endgroup] 2024-06-26T08:31:19.3390211Z Prepare all required actions 2024-06-26T08:31:19.3390673Z Getting action download info 2024-06-26T08:31:19.5421579Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-06-26T08:31:19.7174450Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-06-26T08:31:19.7174931Z with: 2024-06-26T08:31:19.7175494Z file-suffix: test-aot_inductor_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_26688912665 2024-06-26T08:31:19.7176214Z s3-bucket: gha-artifacts 2024-06-26T08:31:19.7176555Z env: 2024-06-26T08:31:19.7176838Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:19.7177308Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:19.7178065Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:19.7178723Z ##[endgroup] 2024-06-26T08:31:19.7203573Z ##[group]Run # Remove any previous test jsons if they exist 2024-06-26T08:31:19.7204237Z # Remove any previous test jsons if they exist 2024-06-26T08:31:19.7204763Z rm -f test-jsons-*.zip 2024-06-26T08:31:19.7205293Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-06-26T08:31:19.7213111Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:31:19.7213624Z env: 2024-06-26T08:31:19.7213911Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:19.7214370Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:19.7215257Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:19.7216199Z FILE_SUFFIX: test-aot_inductor_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_26688912665 2024-06-26T08:31:19.7216875Z ##[endgroup] 2024-06-26T08:31:19.7385389Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-06-26T08:31:19.7414638Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-06-26T08:31:19.7415437Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-06-26T08:31:19.7421602Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-06-26T08:31:19.7446558Z ##[group]Run # Remove any previous test reports if they exist 2024-06-26T08:31:19.7447268Z # Remove any previous test reports if they exist 2024-06-26T08:31:19.7447805Z rm -f test-reports-*.zip 2024-06-26T08:31:19.7448411Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-06-26T08:31:19.7456042Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:31:19.7456547Z env: 2024-06-26T08:31:19.7456828Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:19.7457294Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:19.7458049Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:19.7458973Z FILE_SUFFIX: test-aot_inductor_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_26688912665 2024-06-26T08:31:19.7459649Z ##[endgroup] 2024-06-26T08:31:19.7618813Z adding: test/test-reports/inference_torchbench.csv (deflated 75%) 2024-06-26T08:31:19.7659893Z ##[group]Run # Remove any previous usage logs if they exist 2024-06-26T08:31:19.7660532Z # Remove any previous usage logs if they exist 2024-06-26T08:31:19.7661037Z rm -f logs-*.zip 2024-06-26T08:31:19.7661713Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-06-26T08:31:19.7662503Z # so check to see if the file exists first 2024-06-26T08:31:19.7663029Z if [ -f 'usage_log.txt' ]; then 2024-06-26T08:31:19.7663557Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-06-26T08:31:19.7664049Z fi 2024-06-26T08:31:19.7664487Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-06-26T08:31:19.7665074Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-06-26T08:31:19.7665558Z fi 2024-06-26T08:31:19.7673217Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:31:19.7673850Z env: 2024-06-26T08:31:19.7674135Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:19.7674590Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:19.7675330Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:19.7676256Z FILE_SUFFIX: test-aot_inductor_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_26688912665 2024-06-26T08:31:19.7676944Z ##[endgroup] 2024-06-26T08:31:19.7745796Z adding: usage_log.txt (deflated 92%) 2024-06-26T08:31:19.7811027Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-06-26T08:31:19.7811779Z # Remove any previous debugging artifacts if they exist 2024-06-26T08:31:19.7812351Z rm -f debug-*.zip 2024-06-26T08:31:19.7812753Z if [ -d 'test/debug' ]; then 2024-06-26T08:31:19.7813273Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-06-26T08:31:19.7813765Z fi 2024-06-26T08:31:19.7821275Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:31:19.7821799Z env: 2024-06-26T08:31:19.7822075Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:19.7822543Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:19.7823301Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:19.7824314Z FILE_SUFFIX: test-aot_inductor_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_26688912665 2024-06-26T08:31:19.7825145Z ##[endgroup] 2024-06-26T08:31:19.7897603Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T08:31:19.7898053Z with: 2024-06-26T08:31:19.7898351Z s3-bucket: gha-artifacts 2024-06-26T08:31:19.7898797Z s3-prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:31:19.7899300Z retention-days: 14 2024-06-26T08:31:19.7899646Z if-no-files-found: warn 2024-06-26T08:31:19.7900009Z path: test-jsons-*.zip 2024-06-26T08:31:19.7900361Z name: artifact 2024-06-26T08:31:19.7900670Z region: us-east-1 2024-06-26T08:31:19.7900969Z env: 2024-06-26T08:31:19.7901259Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:19.7901724Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:19.7902481Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:19.7903149Z ##[endgroup] 2024-06-26T08:31:20.1118711Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T08:31:20.1119436Z With the provided path, there will be 1 file uploaded 2024-06-26T08:31:20.1120143Z Uploading to s3 prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:31:20.1145437Z Starting upload of test-jsons-test-aot_inductor_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_26688912665.zip 2024-06-26T08:31:20.3140106Z Finished upload of test-jsons-test-aot_inductor_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_26688912665.zip 2024-06-26T08:31:20.3289375Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T08:31:20.3289826Z with: 2024-06-26T08:31:20.3290119Z s3-bucket: gha-artifacts 2024-06-26T08:31:20.3290569Z s3-prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:31:20.3291048Z retention-days: 14 2024-06-26T08:31:20.3291383Z if-no-files-found: error 2024-06-26T08:31:20.3291758Z path: test-reports-*.zip 2024-06-26T08:31:20.3292119Z name: artifact 2024-06-26T08:31:20.3292423Z region: us-east-1 2024-06-26T08:31:20.3292737Z env: 2024-06-26T08:31:20.3293013Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:20.3293480Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:20.3294232Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:20.3294877Z ##[endgroup] 2024-06-26T08:31:20.6456018Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T08:31:20.6456852Z With the provided path, there will be 1 file uploaded 2024-06-26T08:31:20.6458074Z Uploading to s3 prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:31:20.6483370Z Starting upload of test-reports-test-aot_inductor_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_26688912665.zip 2024-06-26T08:31:20.7414824Z Finished upload of test-reports-test-aot_inductor_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_26688912665.zip 2024-06-26T08:31:20.7558144Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T08:31:20.7558595Z with: 2024-06-26T08:31:20.7558888Z s3-bucket: gha-artifacts 2024-06-26T08:31:20.7559331Z s3-prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:31:20.7559812Z retention-days: 14 2024-06-26T08:31:20.7560167Z if-no-files-found: ignore 2024-06-26T08:31:20.7560531Z path: logs-*.zip 2024-06-26T08:31:20.7560849Z name: artifact 2024-06-26T08:31:20.7561157Z region: us-east-1 2024-06-26T08:31:20.7561469Z env: 2024-06-26T08:31:20.7561752Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:20.7562220Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:20.7562953Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:20.7563597Z ##[endgroup] 2024-06-26T08:31:21.0721601Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T08:31:21.0722595Z With the provided path, there will be 1 file uploaded 2024-06-26T08:31:21.0723305Z Uploading to s3 prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:31:21.0748279Z Starting upload of logs-test-aot_inductor_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_26688912665.zip 2024-06-26T08:31:21.2607402Z Finished upload of logs-test-aot_inductor_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_26688912665.zip 2024-06-26T08:31:21.2750069Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T08:31:21.2750508Z with: 2024-06-26T08:31:21.2750803Z s3-bucket: gha-artifacts 2024-06-26T08:31:21.2751238Z s3-prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:31:21.2751714Z retention-days: 14 2024-06-26T08:31:21.2752051Z if-no-files-found: ignore 2024-06-26T08:31:21.2752419Z path: debug-*.zip 2024-06-26T08:31:21.2752740Z name: artifact 2024-06-26T08:31:21.2753042Z region: us-east-1 2024-06-26T08:31:21.2753346Z env: 2024-06-26T08:31:21.2753647Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:21.2754125Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:21.2754873Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:21.2755527Z ##[endgroup] 2024-06-26T08:31:21.5794897Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-06-26T08:31:21.5935791Z ##[group]Run # shellcheck disable=SC2156 2024-06-26T08:31:21.5936287Z # shellcheck disable=SC2156 2024-06-26T08:31:21.5937086Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-06-26T08:31:21.5945379Z shell: /usr/bin/bash -e {0} 2024-06-26T08:31:21.5945734Z env: 2024-06-26T08:31:21.5958757Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:21.5959423Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:21.5960207Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:21.5960850Z ##[endgroup] 2024-06-26T08:31:21.7594186Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-06-26T08:31:21.7594747Z with: 2024-06-26T08:31:21.7595002Z env: 2024-06-26T08:31:21.7595274Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:21.7595735Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:21.7596472Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:21.7597126Z ##[endgroup] 2024-06-26T08:31:21.7614523Z ##[group]Run set -eou pipefail 2024-06-26T08:31:21.7614940Z set -eou pipefail 2024-06-26T08:31:21.7615302Z  2024-06-26T08:31:21.7615828Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-06-26T08:31:21.7616476Z for _ in $(seq 1440); do 2024-06-26T08:31:21.7616962Z  # Break if no ssh session exists anymore 2024-06-26T08:31:21.7617480Z  if [ "$(who)" = "" ]; then 2024-06-26T08:31:21.7617896Z  break 2024-06-26T08:31:21.7618238Z  fi 2024-06-26T08:31:21.7618548Z  echo "." 2024-06-26T08:31:21.7618884Z  sleep 5 2024-06-26T08:31:21.7619211Z done 2024-06-26T08:31:21.7626743Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:31:21.7627246Z env: 2024-06-26T08:31:21.7627523Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:21.7627981Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:21.7628747Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:21.7629389Z ##[endgroup] 2024-06-26T08:31:21.7648732Z Holding runner for 2 hours until all ssh sessions have logged out 2024-06-26T08:31:21.7684531Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T08:31:21.7685296Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T08:31:21.7685888Z # shellcheck disable=SC2046 2024-06-26T08:31:21.7686359Z docker stop $(docker ps -q) || true 2024-06-26T08:31:21.7686835Z # Prune all of the docker images 2024-06-26T08:31:21.7687291Z docker system prune -af 2024-06-26T08:31:21.7694176Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:31:21.7694682Z env: 2024-06-26T08:31:21.7694951Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:31:21.7695407Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:31:21.7696265Z DOCKER_CONTAINER_ID: 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:21.7696914Z ##[endgroup] 2024-06-26T08:31:22.2137732Z 80c146ebd76a 2024-06-26T08:31:24.5445950Z Deleted Containers: 2024-06-26T08:31:24.5446634Z 80c146ebd76a6c6f1332a82e94bc38d07b873ea63412979ad293a65347de0494 2024-06-26T08:31:24.5447109Z 2024-06-26T08:31:28.4917906Z Deleted Images: 2024-06-26T08:31:28.4919644Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T08:31:28.4922214Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb 2024-06-26T08:31:28.4924197Z deleted: sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd 2024-06-26T08:31:28.4925527Z deleted: sha256:754ead7012f0b7691885a9660f02e80be6cee8c5665486fb995c1fb06adb6e92 2024-06-26T08:31:28.4926640Z deleted: sha256:fbe4496d8830e11be86d8f222522e7956f32f85bfc06b05ea53961f34a58af74 2024-06-26T08:31:28.4927528Z deleted: sha256:1c83814aa1a1a7e0cc998e3479bdab8e8766b975560cbf00690acf39ed899a80 2024-06-26T08:31:28.4928378Z deleted: sha256:e9ea8e61942c7f8ae4aab97b8d102b01d6bb4b6cf2a605808cd34130a17d8a0f 2024-06-26T08:31:28.4929238Z deleted: sha256:acc5fc32588cd95e082a6ce35e48d0f303ae8f96e3ab65c4e375b85e1f3c4a92 2024-06-26T08:31:28.4930094Z deleted: sha256:a93e53400904d29e4185d3be942ea34f19572ea438962788ae4e9b3d9bf4c498 2024-06-26T08:31:28.4931121Z deleted: sha256:c97092b88b5e4c77c68bbc4b51afab64e38719af1ff58013ec49a696cdd40a6e 2024-06-26T08:31:28.4931953Z deleted: sha256:b132889c90d67ffa53655846603e3fcc2372c23f04dbdbc54690a39d304eb89b 2024-06-26T08:31:28.4932962Z deleted: sha256:915d54081ed607710624abe3f2490114c745858831f4fc13396bd3b4b9a121bc 2024-06-26T08:31:28.4933964Z deleted: sha256:bfa8d446c498c87f535aef102695564de1b22c755e6c63944ae2f5073a2b31ec 2024-06-26T08:31:28.4935000Z deleted: sha256:e71460fb363c29e70bbe357d680aaa7ae694f094ef2f2713609158ad90611744 2024-06-26T08:31:28.4935948Z deleted: sha256:84079bb1b4ee1d0119ae3d3e701de460a1324912297c675425c9a36d30fcdb0f 2024-06-26T08:31:28.4936786Z deleted: sha256:d86ac3ca148d4758cb03b493fdd49109d8f1a2c7c85c4b755493c3c4529553e5 2024-06-26T08:31:28.4937637Z deleted: sha256:532b3ab97c44a673a0d5f62ee46fa6a83248a84de8877754a612355be56ed0a3 2024-06-26T08:31:28.4938476Z deleted: sha256:617e2a35f25ae24500c88c0c9b8475f0687fbcbed3d200013b4c99c6d8941aff 2024-06-26T08:31:28.4939330Z deleted: sha256:b88120bcd5a26a660c09c5df13176e0ef098fb2c8cdb57d74ab0796a3d1194d6 2024-06-26T08:31:28.4940157Z deleted: sha256:c016b2e7764735243205fb76fa9a273bb3dd4bf05532288e3a613ce022131ac3 2024-06-26T08:31:28.4941024Z deleted: sha256:bcaca462c390deddd2995de2d245a7495e7d2e1e94f0edad30a573c83d0588fd 2024-06-26T08:31:28.4941868Z deleted: sha256:9e260eba003d5c25d7b410e5c4c3093845d9e46611896840901c689d7fbf8ee6 2024-06-26T08:31:28.4942796Z deleted: sha256:3e62b86f8b911a9b59c62d55f063e71683ae6a01c22418db5d27930712adaad6 2024-06-26T08:31:28.4943947Z deleted: sha256:905920ccb1d88449b8dde686c34a4b2a8b6f59e78ac44916b189ea3593b9fbbb 2024-06-26T08:31:28.4945099Z deleted: sha256:961201237e03acd5061406238dd7a3c1343ac888d7583212cd0ab5ae1b4f0540 2024-06-26T08:31:28.4945940Z deleted: sha256:70047a2aac62159fa28fb4dbd4567ec2ffbab68fe046967e09acb0eb3e5cb9cb 2024-06-26T08:31:28.4947126Z deleted: sha256:1ec680ad781436ae65fa987f5fa1ed032ecd5c82a6e35eea6707de501f62c0f4 2024-06-26T08:31:28.4947975Z deleted: sha256:ef859623a50e85abf6e6c1dbd6188aa765a66cc7b2bd618a1cbe4825b7b2bd91 2024-06-26T08:31:28.4948829Z deleted: sha256:404ade5cacff59773b93ab438bfa64cd3a171d07f6f168bdc4a5d4bfa0b43526 2024-06-26T08:31:28.4949712Z deleted: sha256:5b27de6ee3d9f2f18217bf18dec725f7b4954c0e48c85d35752634c22d62209b 2024-06-26T08:31:28.4950567Z deleted: sha256:498e39c575b3fa069acaac7e46b91b4e54ae3a05f2ada5c4347091f346f25d4c 2024-06-26T08:31:28.4951380Z deleted: sha256:1405919d2694d65669b1f4c842282021afe89208ce86570e96464903733da2d0 2024-06-26T08:31:28.4952364Z deleted: sha256:d5b41e5d70896c0472fc3e6da0516a17d89cd6635dc75010ecddc31ce9690fad 2024-06-26T08:31:28.4953199Z deleted: sha256:18e9379b62b9d0bb40293d28a652eafaee66a499f66f0776bf14e37a513101b7 2024-06-26T08:31:28.4954034Z deleted: sha256:a781f80ed00ff2926a699fd01a309bb0b4fb7d5655eb652c205bb9c7f811035f 2024-06-26T08:31:28.4954867Z deleted: sha256:c0430973b7cf759075330ee29c79bfd1312c91e6033d9dd883b4b1c29743a171 2024-06-26T08:31:28.4955720Z deleted: sha256:e733b3d66efb9342e5a283376c0e1ddffa47a42fecc26e61ecaf0a3472885d97 2024-06-26T08:31:28.4956566Z deleted: sha256:8929c0b2047f74b84c82c39eff9077a92ae1a828cde00c9ab78fb3e0ed5e0283 2024-06-26T08:31:28.4957395Z deleted: sha256:f5b56155fb58a0a60fe6391d3581cf91610ed69d4f0a321257c19944da296a52 2024-06-26T08:31:28.4958226Z deleted: sha256:b54d2cb355238d1aa91e58df93064447b80f53706d1ca62823acc834f5ac54fe 2024-06-26T08:31:28.4959050Z deleted: sha256:e7311285af9eeddb51aa5629aa98386e89f1456ff32843b6aee6443247edf66c 2024-06-26T08:31:28.4959899Z deleted: sha256:67657ceec347da8dc8085d5731633efeac115ba677fa4b7c5966c4260f7abab0 2024-06-26T08:31:28.4960743Z deleted: sha256:b496e50409dbf414930fc9b4437aa0aa4ce27bc7840e050ae459aaffcf9aca86 2024-06-26T08:31:28.4961579Z deleted: sha256:8f5276170cc4a60d36ad7b964a502306b29546254cedcbcb101ea9123497d2c0 2024-06-26T08:31:28.4962406Z deleted: sha256:53d524d1564745e96a7e515462171b73184d9f1ff5454b32fcfbbca39d361467 2024-06-26T08:31:28.4963248Z deleted: sha256:2acbadb00d69b6aca575f04baba4ea26ae2b5c2d85f09945004ad93a78a8eb75 2024-06-26T08:31:28.4964206Z deleted: sha256:02665efeff878aeb603c1edb2e21cb700c171c025d64f5aae1b2b20b12f3964a 2024-06-26T08:31:28.4965066Z deleted: sha256:19bd23e9a51dfe883a8b8d2ff6d66ff1b3476b8912856406bb79b81762aa6d4b 2024-06-26T08:31:28.4965910Z deleted: sha256:73aa8637bf9299d6298dabd858480c274a62117388f2ca1d5e9eb15c6d1e4155 2024-06-26T08:31:28.4966731Z deleted: sha256:5f8968406305f348b1caebc0e035187d6e9965ace7b9d5f2f21114d37c9fe2e1 2024-06-26T08:31:28.4967574Z deleted: sha256:d827104914db0c503cf5c81f13e2456bf4efce42d4cf1f8d5d70838207278bee 2024-06-26T08:31:28.4968411Z deleted: sha256:867b219724f2f1aaf56f42e3e9f03a5d64bdca8a6a72407efd10328aeea5d510 2024-06-26T08:31:28.4969267Z deleted: sha256:b9ff2a7cceb9237b26bcb17728edf8a558eec5a76f63626e84f80541dfc0f1d4 2024-06-26T08:31:28.4970112Z deleted: sha256:be73f509106819007ce2b0f5a1301ac56ae583cb6c1fb69bee93efa0840368f1 2024-06-26T08:31:28.4970957Z deleted: sha256:2a11c07c915c3fdbb4e73096dea73ea950c970d50fe1a5764060bd62b7b7cf2c 2024-06-26T08:31:28.4971813Z deleted: sha256:5e9989622e2abc1c72f8184343507b45d75e4ff971edb68b96cbcd75ae484ff5 2024-06-26T08:31:28.4972663Z deleted: sha256:05ce642bd9cf5a9205bff107df28b1d6cef7d5f6ab3ea944d28c032ec61270df 2024-06-26T08:31:28.4973494Z deleted: sha256:017702617f6c07b73f2088678f4710b794cfe897a213a5cf122a0e2519eae0ac 2024-06-26T08:31:28.4974332Z deleted: sha256:44b3b37bb1ca39dc5cf356de41a23a5c12f605a1992813e408087fb2b0ad6d1c 2024-06-26T08:31:28.4975177Z deleted: sha256:8c387982e9f27db357a1d149dc149cabc0bd04668ad6956387a5072682dada67 2024-06-26T08:31:28.4976027Z deleted: sha256:10d415dca4e3fbb88a747b40317f3c6333c041ba8f73855f67fb9fe981fae5ee 2024-06-26T08:31:28.4976865Z deleted: sha256:6c65469bf0f53dcd625faefe363b8f3136c08388a6cf64d9158e8576e4ee89b1 2024-06-26T08:31:28.4977713Z deleted: sha256:027db5d8b76deeefbd8c34b485ab7ff14ed2a3e6800425c1ce0d77c2e9fdd4ab 2024-06-26T08:31:28.4978558Z deleted: sha256:24941234ad28cf5c8174d7cf18326c10b4e17f2c2137672ea0322df461597dd7 2024-06-26T08:31:28.4979407Z deleted: sha256:240ea2eebfa8519d6c2d88f4c99e66ce4e038d000f93ce8d03726c01d81180ef 2024-06-26T08:31:28.4980250Z deleted: sha256:09e823cb391a69f6a195e3ca8a41624b5f8b34eee14c6eb36277de814a91966e 2024-06-26T08:31:28.4981068Z deleted: sha256:730479d695169a6593bf0ac16b377491dce65175a6d158e012422367d04577fd 2024-06-26T08:31:28.4981900Z deleted: sha256:b0a30f9f7bd5459ff662453b9346a7ba8ea8db1191528345c1953eab4aa2f54e 2024-06-26T08:31:28.4982743Z deleted: sha256:75cefa87effaa3ad361c74642839923101f0f9982f814f3c9e2ed4ad52f2a008 2024-06-26T08:31:28.4983635Z deleted: sha256:6f97db87ba9512e615e871c6982000602982e1e5a5be334a56e7292251e84e00 2024-06-26T08:31:28.4984589Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-06-26T08:31:28.4985435Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-06-26T08:31:28.4986260Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-06-26T08:31:28.4987300Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-06-26T08:31:28.4988154Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-06-26T08:31:28.4988995Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-06-26T08:31:28.4989865Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-06-26T08:31:28.4990741Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-06-26T08:31:28.4991617Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-06-26T08:31:28.4992472Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-06-26T08:31:28.4993328Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-06-26T08:31:28.4993839Z 2024-06-26T08:31:28.4993997Z Total reclaimed space: 40.92GB 2024-06-26T08:31:28.5056726Z Post job cleanup. 2024-06-26T08:31:28.5105398Z Post job cleanup. 2024-06-26T08:31:28.5922527Z [command]/usr/bin/git version 2024-06-26T08:31:28.5964371Z git version 2.40.1 2024-06-26T08:31:28.6004435Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/12855e21-b5aa-4404-af34-1fd900b6dd12' before making global git config changes 2024-06-26T08:31:28.6005684Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T08:31:28.6010125Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T08:31:28.6042622Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T08:31:28.6070753Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T08:31:28.6294546Z Entering 'android/libs/fbjni' 2024-06-26T08:31:28.6331848Z Entering 'third_party/FP16' 2024-06-26T08:31:28.6368700Z Entering 'third_party/FXdiv' 2024-06-26T08:31:28.6405075Z Entering 'third_party/NNPACK' 2024-06-26T08:31:28.6442560Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T08:31:28.6480545Z Entering 'third_party/XNNPACK' 2024-06-26T08:31:28.6525986Z Entering 'third_party/benchmark' 2024-06-26T08:31:28.6563529Z Entering 'third_party/cpp-httplib' 2024-06-26T08:31:28.6600312Z Entering 'third_party/cpuinfo' 2024-06-26T08:31:28.6637041Z Entering 'third_party/cudnn_frontend' 2024-06-26T08:31:28.6671935Z Entering 'third_party/cutlass' 2024-06-26T08:31:28.6714052Z Entering 'third_party/eigen' 2024-06-26T08:31:28.6752038Z Entering 'third_party/fbgemm' 2024-06-26T08:31:28.6786079Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T08:31:28.6820406Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T08:31:28.6853533Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T08:31:28.6892964Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T08:31:28.6927691Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T08:31:28.6960677Z Entering 'third_party/flatbuffers' 2024-06-26T08:31:28.7000665Z Entering 'third_party/fmt' 2024-06-26T08:31:28.7034448Z Entering 'third_party/foxi' 2024-06-26T08:31:28.7068833Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T08:31:28.7103244Z Entering 'third_party/gloo' 2024-06-26T08:31:28.7139854Z Entering 'third_party/googletest' 2024-06-26T08:31:28.7171995Z Entering 'third_party/ideep' 2024-06-26T08:31:28.7206574Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T08:31:28.7244576Z Entering 'third_party/ittapi' 2024-06-26T08:31:28.7280315Z Entering 'third_party/kineto' 2024-06-26T08:31:28.7315213Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T08:31:28.7349761Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T08:31:28.7385216Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T08:31:28.7420219Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T08:31:28.7454850Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T08:31:28.7490008Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T08:31:28.7528193Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T08:31:28.7562162Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T08:31:28.7596856Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T08:31:28.7631788Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T08:31:28.7668028Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T08:31:28.7701113Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T08:31:28.7735424Z Entering 'third_party/mimalloc' 2024-06-26T08:31:28.7770895Z Entering 'third_party/nccl/nccl' 2024-06-26T08:31:28.7806167Z Entering 'third_party/nlohmann' 2024-06-26T08:31:28.7839958Z Entering 'third_party/onnx' 2024-06-26T08:31:28.7887115Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T08:31:28.7920881Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T08:31:28.7957859Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T08:31:28.7992993Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T08:31:28.8027342Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T08:31:28.8058023Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T08:31:28.8091869Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T08:31:28.8127400Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T08:31:28.8160726Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T08:31:28.8194449Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T08:31:28.8227909Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T08:31:28.8263510Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T08:31:28.8296953Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T08:31:28.8348090Z Entering 'third_party/pocketfft' 2024-06-26T08:31:28.8379925Z Entering 'third_party/protobuf' 2024-06-26T08:31:28.8417331Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T08:31:28.8450698Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T08:31:28.8486066Z Entering 'third_party/psimd' 2024-06-26T08:31:28.8520656Z Entering 'third_party/pthreadpool' 2024-06-26T08:31:28.8558240Z Entering 'third_party/pybind11' 2024-06-26T08:31:28.8594855Z Entering 'third_party/python-peachpy' 2024-06-26T08:31:28.8629822Z Entering 'third_party/sleef' 2024-06-26T08:31:28.8665942Z Entering 'third_party/tensorpipe' 2024-06-26T08:31:28.8701520Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T08:31:28.8736398Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T08:31:28.8768026Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T08:31:28.8801771Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T08:31:28.8835670Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T08:31:28.8886093Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T08:31:28.8904053Z http.https://github.com/.extraheader 2024-06-26T08:31:28.8913054Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-06-26T08:31:28.8937087Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T08:31:28.9139593Z Entering 'android/libs/fbjni' 2024-06-26T08:31:28.9163414Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9187580Z Entering 'third_party/FP16' 2024-06-26T08:31:28.9208772Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9233461Z Entering 'third_party/FXdiv' 2024-06-26T08:31:28.9256506Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9278788Z Entering 'third_party/NNPACK' 2024-06-26T08:31:28.9300798Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9325013Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T08:31:28.9346408Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9371093Z Entering 'third_party/XNNPACK' 2024-06-26T08:31:28.9394365Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9429551Z Entering 'third_party/benchmark' 2024-06-26T08:31:28.9451533Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9475380Z Entering 'third_party/cpp-httplib' 2024-06-26T08:31:28.9497712Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9519339Z Entering 'third_party/cpuinfo' 2024-06-26T08:31:28.9540123Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9564085Z Entering 'third_party/cudnn_frontend' 2024-06-26T08:31:28.9587611Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9611585Z Entering 'third_party/cutlass' 2024-06-26T08:31:28.9634684Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9663996Z Entering 'third_party/eigen' 2024-06-26T08:31:28.9686241Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9713101Z Entering 'third_party/fbgemm' 2024-06-26T08:31:28.9735292Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9759695Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T08:31:28.9780375Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9803799Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T08:31:28.9824310Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9847888Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T08:31:28.9869078Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9895942Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T08:31:28.9916806Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9938453Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T08:31:28.9960080Z http.https://github.com/.extraheader 2024-06-26T08:31:28.9984284Z Entering 'third_party/flatbuffers' 2024-06-26T08:31:29.0005670Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0030276Z Entering 'third_party/fmt' 2024-06-26T08:31:29.0052488Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0075735Z Entering 'third_party/foxi' 2024-06-26T08:31:29.0097623Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0119914Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T08:31:29.0140085Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0164597Z Entering 'third_party/gloo' 2024-06-26T08:31:29.0185032Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0207892Z Entering 'third_party/googletest' 2024-06-26T08:31:29.0229145Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0251871Z Entering 'third_party/ideep' 2024-06-26T08:31:29.0273712Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0297606Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T08:31:29.0319404Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0347838Z Entering 'third_party/ittapi' 2024-06-26T08:31:29.0369671Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0394015Z Entering 'third_party/kineto' 2024-06-26T08:31:29.0414707Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0437893Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T08:31:29.0458956Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0483546Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T08:31:29.0503179Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0529387Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T08:31:29.0550929Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0574110Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T08:31:29.0595563Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0619338Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T08:31:29.0641356Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0663846Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T08:31:29.0686097Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0711575Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T08:31:29.0732523Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0754952Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T08:31:29.0775590Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0799099Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T08:31:29.0820371Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0846028Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T08:31:29.0866991Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0892000Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T08:31:29.0913581Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0935699Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T08:31:29.0957781Z http.https://github.com/.extraheader 2024-06-26T08:31:29.0980740Z Entering 'third_party/mimalloc' 2024-06-26T08:31:29.1003382Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1027334Z Entering 'third_party/nccl/nccl' 2024-06-26T08:31:29.1049455Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1074059Z Entering 'third_party/nlohmann' 2024-06-26T08:31:29.1095790Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1118497Z Entering 'third_party/onnx' 2024-06-26T08:31:29.1140085Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1176003Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T08:31:29.1197757Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1221152Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T08:31:29.1243649Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1269796Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T08:31:29.1292923Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1316598Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T08:31:29.1337589Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1360801Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T08:31:29.1381554Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1404711Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T08:31:29.1425293Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1447216Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T08:31:29.1469302Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1492136Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T08:31:29.1514153Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1537734Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T08:31:29.1559220Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1580281Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T08:31:29.1601210Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1623837Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T08:31:29.1645412Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1670061Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T08:31:29.1691750Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1716178Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T08:31:29.1737312Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1776354Z Entering 'third_party/pocketfft' 2024-06-26T08:31:29.1801266Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1824821Z Entering 'third_party/protobuf' 2024-06-26T08:31:29.1846982Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1872532Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T08:31:29.1894020Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1915581Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T08:31:29.1935249Z http.https://github.com/.extraheader 2024-06-26T08:31:29.1959412Z Entering 'third_party/psimd' 2024-06-26T08:31:29.1980578Z http.https://github.com/.extraheader 2024-06-26T08:31:29.2004245Z Entering 'third_party/pthreadpool' 2024-06-26T08:31:29.2025856Z http.https://github.com/.extraheader 2024-06-26T08:31:29.2048967Z Entering 'third_party/pybind11' 2024-06-26T08:31:29.2071044Z http.https://github.com/.extraheader 2024-06-26T08:31:29.2093997Z Entering 'third_party/python-peachpy' 2024-06-26T08:31:29.2116238Z http.https://github.com/.extraheader 2024-06-26T08:31:29.2137351Z Entering 'third_party/sleef' 2024-06-26T08:31:29.2158272Z http.https://github.com/.extraheader 2024-06-26T08:31:29.2181302Z Entering 'third_party/tensorpipe' 2024-06-26T08:31:29.2203395Z http.https://github.com/.extraheader 2024-06-26T08:31:29.2227716Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T08:31:29.2248932Z http.https://github.com/.extraheader 2024-06-26T08:31:29.2271267Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T08:31:29.2293613Z http.https://github.com/.extraheader 2024-06-26T08:31:29.2317613Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T08:31:29.2337930Z http.https://github.com/.extraheader 2024-06-26T08:31:29.2361956Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T08:31:29.2383209Z http.https://github.com/.extraheader 2024-06-26T08:31:29.2405865Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T08:31:29.2426733Z http.https://github.com/.extraheader 2024-06-26T08:31:29.2526892Z A job completed hook has been configured by the self-hosted runner administrator 2024-06-26T08:31:29.2544123Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T08:31:29.2551216Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:31:29.2551756Z ##[endgroup] 2024-06-26T08:31:30.3369424Z Cleaning up orphan processes